Methods for Studying and Advancing Deep Learning Models: Dynamics of Platforms, Applications, nd Emerging Research Trends
DOI:
https://doi.org/10.53469/jssh.2024.6(12).20Keywords:
deep learning, neural networks, learning models, applications, Generative learning, Hybrid learning, Intelligent systems, Emergent Applications, Machine learning, Artificial IntelligenceAbstract
Deep learning has rapidly emerged as a focal point, captivating both academic research and industrial applications in recent years. This paper provides a concise overview of the pivotal concepts and determinants underpinning the evolution of deep learning. The impact of data augmentation (DA) on deep learning shines through recent demonstrations, leading to heightened accuracy, stability, and mitigated overfitting. Operating within the broader sphere of machine learning, deep learning's prowess lies in its capacity to autonomously uncover intricate patterns and representations from raw data. Its evolution from conventional machine learning approaches has unlocked transformative potential, propelling advancements in areas such as image classification, speech synthesis, and medical diagnosis. However, challenges pertaining to interpretability, data scarcity, and model generalization persist, fostering an active field of research that continually refines deep learning techniques. The ramifications of this technology are profound, as it reshapes industries and fuels innovation, thereby shaping the trajectory of AI-driven solutions. In contrast, strides in image and spectral data analysis have harnessed the power of synthetic data facilitated by sophisticated forward models and generative unsupervised deep learning methods. This paper offers a comprehensive perspective on deep learning techniques, first introducing them at a high level and subsequently delving into their applications in atomistic simulation, materials imaging, spectral analysis, and natural language processing. Across these modalities, we explore theoretical and experimental data applications, prevalent modeling strategies with their strengths and limitations, as well as available software and datasets. The exposition encapsulates six principal deep learning models that dominate contemporary academic research, elucidating their principles and characteristics. Beyond academia, this work examines industrial applications, such as speech and image recognition, artificial intelligence, and anticipates forthcoming trends and challenges in these domains."
References
M. Nielsen. Neural networks and deep learning. 2017. http: //neuralnetworksanddeeplearning.com/.
I. Goodfellow, Y. Bengio, and A. Courville. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org
H.-J. Bae, C.-W. Kim, N. Kim, B. Park, N. Kim, J. B. Seo, and S. M. Lee, “A perlin noise-based augmentation strategy for deep learning with small data samples of hrct images,” Scientific reports, vol. 8, no. 1, p. 17687, 2018.
J. Lemley, S. Bazrafkan, and P. Corcoran, “Smart augmentation learning an optimal data augmentation strategy,” Ieee Access, vol. 5, pp. 5858– 5869, 2017.
H.-C. Shin, N. A. Tenenholtz, J. K. Rogers, C. G. Schwarz, M. L. Senjem, J. L. Gunter, K. P. Andriole, and M. Michalski, “Medical image synthesis for data augmentation and anonymization using generative adversarial networks,” in International Workshop on Simulation and Synthesis in Medical Imaging. Springer, 2018, pp. 1–11.
J. Shijie, W. Ping, J. Peiyi, and H. Siping, “Research on data augmenta tion for image classification based on convolution neural networks,” in 2017 Chinese automation congress (CAC). IEEE, 2017, pp. 4165– 4170.
A. Garcia-Garcia, S. Orts-Escolano, S. Oprea, V. Villena-Martinez, P. Martinez-Gonzalez, and J. Garcia-Rodriguez, “A survey on deep learn ing techniques for image and video semantic segmentation,” Applied Soft Computing, vol. 70, pp. 41–65, 2018.
Jing, Y., Bian, Y., Hu, Z., Wang, L. & Xie, X.-Q. S. Deep learning for drug design: an artificial intelligence paradigm for drug discovery in the big data era. AAPS J. 20, 1–10 (2018).
ZEILER M D㸪 FERGUS R㸬Stochastic pooling for regularization of deep convolutional neural networks
[EB / OL]㸬[2016-01-11]
Sarker IH. A machine learning based robust prediction model for real-life mobile phone data. Internet of Things. 2019;5:180–93.
Sarker IH, Colman A, Han J, Khan AI, Abushark YB, Salah K. Behavdt: a behavioral decision tree learning to build usercentric context-aware predictive model. Mob Netw Appl. 2020;25(3):1151–61.
Hasan RI, Yusuf SM, Alzubaidi L. Review of the state of the art of deep learning for plant diseases: a broad analysis and discussion. Plants. 2020;9(10):1302
Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E. Deep learning applications and challenges in big data analytics. J Big Data. 2015;2(1):1.
Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning, vol. 1. Cambridge: MIT press; 2016.
Shorten C, Khoshgoftaar TM, Furht B. Deep learning applications for COVID-19. J Big Data. 2021;8(1):1– 54.
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classifcation with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
Bhowmick S, Nagarajaiah S, Veeraraghavan A. Vision and deep learning-based algorithms to detect and quantify cracks on concrete surfaces from uav videos. Sensors. 2020;20(21):6299. 32. Goh GB, Hodas NO, Vishnu A. Deep learning for computational chemistry. J Comput Chem. 2017;38(16):1291–307. 33. Li Y, Zhang T, Sun S, Gao X. Accelerating fash calculation through deep learning methods. J Comput Phys. 2019;394:153–65
Yang W, Zhang X, Tian Y, Wang W, Xue JH, Liao Q. Deep learning for single image super-resolution: a brief review. IEEE Trans Multimed. 2019;21(12):3106–21.
Tang J, Li S, Liu P. A review of lane detection methods based on deep learning. Pattern Recogn. 2020;111:107623.
Zhao ZQ, Zheng P, Xu ST, Wu X. Object detection with deep learning: a review. IEEE Trans Neural Netw Learn Syst. 2019;30(11):3212–32.
Chea P, Mandell JC. Current applications and future directions of deep learning in musculoskeletal radiology. Skelet Radiol. 2020;49(2):1–15.
Wu X, Sahoo D, Hoi SC. Recent advances in deep learning for object detection. Neurocomputing. 2020;396:39–64.
Kuutti S, Bowden R, Jin Y, Barber P, Fallah S. A survey of deep learning applications to autonomous vehicle control. IEEE Trans Intell Transp Syst. 2020;22:712–33.
Yolcu G, Oztel I, Kazan S, Oz C, Bunyak F. Deep learning-based face analysis system for monitoring customer inter est. J Ambient Intell Humaniz Comput. 2020;11(1):237–48. 224. Jiao L, Zhang F, Liu F,
Muhammad K, Khan S, Del Ser J, de Albuquerque VHC. Deep learning for multigrade brain tumor classifcation in smart healthcare systems: a prospective survey. IEEE Trans Neural Netw Learn Syst. 2020;32:507–22.
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.
Mukherjee D, Mondal R, Singh PK, Sarkar R, Bhattacharjee D. Ensemconvnet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications. Multimed Tools Appl. 2020;79(41):31663–90.
Zeleznik R, Foldyna B, Eslami P, Weiss J, Alexander I, Taron J, Parmar C, Alvi RM, Banerji D, Uno M, et al. Deep convolutional neural networks to predict cardiovascular risk from computed tomography. Nature Commun. 2021;12(1):1–9.
Wang J, Liu Q, Xie H, Yang Z, Zhou H. Boosted efcientnet: detection of lymph node metastases in breast cancer using convolutional neural networks. Cancers. 2021;13(4):661.
Yu H, Yang LT, Zhang Q, Armstrong D, Deen MJ. Convolutional neural networks for medical image analysis: state-of-the-art, comparisons, improvement and perspectives. Neurocomputing. 2021. https://doi.org/10.1016/j. neucom.2020.04.157
Y. Wu, „„Google‟s neural machine translation system: Bridging the gap between human and machine translation,‟‟ CoRR, Sep. 2016.
Z.-H. Zhou, M.-L. Zhang, S.-J. Huang, and Y.-F. Li,
„„Multi-instance multilabel learning,‟‟ Artif. Intell., vol. 176, no. 1, pp. 2291–2320, 2012.
L. Huang, A. D. Joseph, B. Nelson, B. I. P. Rubinstein, and J. D. Tygar, „„Adversarial machine learning,‟‟ in Proc. 4th ACM Workshop Secur. Artif. Intell., Chicago, IL, USA, 2011, pp. 43–58.
D. Yu and L. Deng, Automatic Speech Recognition: A Deep Learning Approach. London, U.K.: Springer, 2015.
R. Hadsell, S. Chopra, and Y. LeCun, „„Dimensionality reduction by learning an invariant mapping,‟‟ in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2006, pp. 1735–1742.
K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. Bharath. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6):26– 38, Nov 2017.
Z. Zhao, J. Guo, E. Ding, Z. Zhu, and D. Zhao. Terminal replacement prediction based on deep belief networks. In 2015 International Conference on Network and Information Systems for Computers, pages 255–258, Jan 2015.
D. Zhu, H. Jin, Y. Yang, D. Wu, and W. Chen. DeepFlow: deep learning-based malware detection by mining Android application for abnormal usage of sensitive data. In 2017 IEEE Symposium on Computers and Communications (ISCC), pages 438– 443, July 2017.
J. Zhu, Y. Song, D. Jiang, and H. Song. A new deep-q- learningbased transmission scheduling mechanism for the cognitive Internet of Things. IEEE Internet of Things Journal, PP(99):1–1, 2017.
M. Zolotukhin, T. Hämäläinen, T. Kokkonen, and J. Siltanen. Increasing web service availability by detecting application-layer DDoS attacks in encrypted traffic. In 2016 23rd International Conference on Telecommunications (ICT), pages 1–6, May 2016
X. Yuan, P. He, Q. Zhu, R. Rana Bhat, and X. Li. Adversarial Examples: Attacks and Defenses for Deep Learning. ArXiv e-prints, Dec. 2017
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Maria Kushnareva
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
