BERT MODELLARI VA ULARNING TARJIMON DASTURLARIDA QO'LLANILISHI: TAKOMILLASHGAN YONDASHUVLAR

Safoyev N. N.


Annotatsiya

Ushbu maqolada BERT (Bidirectional Encoder Representations from Transformers) modellari va ularning tarjimon dasturlarida qo‘llanilishi haqida so‘nggi tadqiqotlar tahlil qilinadi. BERTning ikki yo‘nalishli (bidirectional) arxitekturasi va transformer texnologiyasi orqali tarjimon tizimlarining samaradorligini sezilarli darajada oshirishi mumkin. Tadqiqotda BERT modelining tarjima sifatini yaxshilashda, ko‘p tilli tizimlarda yuqori sifatli tarjimalar olishda va cross-lingual transfer learning jarayonlarida qanday muvaffaqiyatlarga erishilganligi ko‘rsatilgan. Shuningdek, RoBERTa va ALBERT kabi BERTning optimallashtirilgan versiyalarining tarjima tizimlaridagi o‘rni va imkoniyatlari ham tahlil qilinadi. BERTni tarjimonlikda qo‘llashning ba'zi cheklovlari, masalan, hisoblash resurslariga bo‘lgan talab va murakkab matnlar bilan bog‘liq muammolar ham ko‘rib chiqiladi. Tadqiqot natijalari BERT va uning takomillashgan versiyalarining tarjima tizimlarida keng qo‘llanilishini va kelajakdagi imkoniyatlarini ochib beradi.


Adabiyotlar

  1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. A., Kaiser, Ł., & Polosukhin, I. (2017). Attention is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems.
  2. Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. A. (2018). Unsupervised Machine Translation Using Monolingual Corpora Only. Proceedings of the 6th International Conference on Learning Representations.
  3. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805.
  4. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Le, Q. V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692.
  5. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2020). ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. arXiv preprint arXiv:1909.11942.
  6. Artetxe, M., & Schwenk, H. (2019). Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
  7. Zhang, X., Zong, C., & Yu, L. (2020). BERT for Cross-lingual Transfer Learning in Multilingual Translation Systems. Journal of Artificial Intelligence Research, 67, 309-334.
  8. Kwiatkowski, T., Palomaki, J., Redfield, S., Collins, M., & Parikh, A. (2019). Natural Questions: A Benchmark for Question Answering Research. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.