PARALLEL VA ASINXRON HISOBLASH ASOSIDA NEYRO-TAQSIMLANGAN KOGNITIV ADAPTIV OPTIMALLASHTIRISH MODELI
Choriyev O. B. & Boysariyev Y. J.
Annotatsiya
Ushbu tadqiqotda neyro-taqsimlangan chuqur o‘rganish tizimlari uchun kognitiv adaptiv asinxron optimallashtirish modeli taklif etiladi. Modelning asosiy maqsadi — parallel va asinxron hisoblash yondashuvlarining afzalliklarini birlashtirib, konvergensiya tezligi, barqarorlik hamda energiyadan foydalanish samaradorligini oshirishdir. Taklif etilgan yondashuv inson miyasining o‘z-o‘zini boshqarish tamoyillariga asoslangan bo‘lib, o‘rganish tezligini parametr orqali gradient normasi, yo‘qotish funksiyasi dinamikasi va kechikish (staleness) darajasiga nisbatan kognitiv tarzda moslashtiradi. Matematik asos sifatida Lyapunov barqarorlik nazariyasi qo‘llanilib, modelning eksponensial konvergensiyasi isbotlangan. Amaliy tajribalar MNIST, CIFAR-10 va CIFAR-100 ma’lumotlar to‘plamlarida PyTorch muhitida o‘tkazildi. Natijalarga ko‘ra, kognitiv adaptiv model o‘qitish vaqtini 20–25 % gacha qisqartirib, aniqlikni 99.1 % gacha oshirgan va energiyadan foydalanish samaradorligini (Perf/Watt) 1.5 barobar yaxshilagan. Shuningdek, model kechikkan gradientlar ta’sirini (staleness) kognitiv kompensatsiya mexanizmi orqali kamaytirib, asinxron muhitdagi konvergensiya tebranishlarini silliqlash imkonini berdi. Olingan natijalar kognitiv adaptiv yondashuvning neyro-taqsimlangan tizimlar uchun optimal vaqt–aniqlik–energiyaviy balansni ta’minlash qobiliyatini isbotlaydi. Tadqiqot natijalari kelajakda federativ o‘qitish, bulutli hisoblash klasterlari, aqlli raqamli egizaklar va energiya samarador sun’iy intellekt arxitekturalarini yaratishda qo‘llanilishi mumkin bo‘lgan yangi ilmiy yo‘nalishni shakllantiradi.
Adabiyotlar
- J. Zhang, Y. Zhu, “Neural Network-Based Model Predictive Control for Nonlinear Systems,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 2, pp. 478–489, 2021.
- H. You, C. Ding, et al., “Asynchronous Parallel Stochastic Gradient Descent with Delay Compensation,” IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 12, pp. 2754–2768, 2020.
- A. Amini, M. Soleymani, “An Intelligent Adaptive Fuzzy Neural Controller for Nonlinear Uncertain Systems,” Engineering Applications of Artificial Intelligence, vol. 100, 2021, p. 104151.
- K. S. Narendra, A. M. Annaswamy, Stable Adaptive Systems, Dover Publications, 2019.
- S. Haykin, Neural Networks and Learning Machines, 3rd ed., Pearson Education, 2020.
- J. Dean, G. Corrado, “Large Scale Distributed Deep Networks,” Advances in Neural Information Processing Systems (NIPS), 2012, pp. 1223–1231.
- T. Chen, B. Xu, C. Zhang, “Training Deep Nets with Sublinear Memory Cost,” arXiv preprint arXiv:1604.06174, 2016.
- Yusupbekov, N. R., Usmonov, U. S., & Khayitov, A. K. “Adaptive Intelligent Control Systems for Industrial Automation,” Scientific and Technical Journal of Information Technologies, Mechanics and Optics, vol. 21, no. 5, pp. 815–828, 2021.
- A. Dutta, R. Dabre, A. Kunchukuttan, “Asynchronous and Synchronous Parallel Training of Neural Networks: A Comparative Study,” Journal of Parallel and Distributed Computing, vol. 145, pp. 27–37, 2020.
- Y. Bengio, I. Goodfellow, A. Courville, Deep Learning, MIT Press, 2017.
- M. Goyal, “Energy-Aware Distributed Deep Learning Frameworks,” Future Generation Computer Systems, vol. 136, pp. 180–194, 2022.
- L. Bottou, F. Curtis, J. Nocedal, “Optimization Methods for Large-Scale Machine Learning,” SIAM Review, vol. 60, no. 2, pp. 223–311, 2018.
- C. Zhang, M. Li, D. G. Andersen, A. Smola, “Parameter Server for Distributed Machine Learning,” Journal of Machine Learning Research, vol. 18, no. 1, pp. 1–47, 2017.
- P. Blanchard, E. Moulines, et al., “Distributed Optimization with Sparse Communications,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 6, pp. 1418–1432, 2020.
- F. Niu, B. Recht, C. Re, S. Wright, “Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent,” Advances in Neural Information Processing Systems, vol. 24, 2011.