Применение трансформеров и методов машинного обучения в системе рекомендаций научных руководителей
Аннотация
В настоящей работе предложена система рекомендаций для выбора научного руко- водителя, основанная на архитектуре трансформеров и современных методах ма- шинного обучения. Система анализирует академические данные студента, включая изученные дисциплины и оценки по ним, а также профессиональные характеристи- ки преподавателей. Экспериментальные результаты демонстрируют значительное превосходство предложенного подхода над традиционными методами: при тестиро- вании достигнута точность рекомендаций 0,3230 против 0,1106 (метод на основе частоты положительных оценок) и 0,1637 (подход с использованием классификации через машинное обучение). Полученные результаты подтверждают эффективность системы в оптимизации процесса подбора научного руководителя, что способствует повышению качества научно-исследовательской деятельности студентов за счёт персонализированного сопоставления компетенций.
Литература
J. L. Herlocker et al., “Evaluating collaborative filtering recommender systems,” ACM Transactions on Information Systems, vol. 22, no. 1, pp. 5–53, 2004, doi: 10.1145/963770.963772
M. J. Pazzani and D. Billsus, “Content-Based Recommendation Systems,” in The Adaptive Web, Berlin: Springer Berlin Heidelberg, pp. 325–341, 2007; doi:10.1007/978-3-540-72079-9_10
X. N. Lam et al., “Addressing cold-start problem in recommendation systems,” in Proc. of the 2nd Int. Conf. on Ubiquitous Information Management and Communication, pp. 208–211, 2008, doi:10.1145/1352793.1352837
X. He et al., “Neural Collaborative Filtering,” in Proc. of the 26th International Conference on World WideWeb, pp. 173–182, 2017; doi:10.1145/3038912.3052569
W.-C. Kang and J. McAuley, “Self-Attentive Sequential Recommendation,” in Proc. of the 2018 IEEE Int. Conf. on Data Mining (ICDM), pp. 197–206, 2018, doi:10.1109/icdm.2018.00035
A. Vaswani et al., “Attention is all you need,” Advances in neural information processing systems, vol. 31, pp. 1–15, 2017.
Z. Zhao et al., “Recommender Systems in the Era of Large Language Models (LLMs),” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 11, pp. 6889–6907, 2024; doi:10.1109/tkde.2024.3392335
W. X. Zhao et al., “A survey of large language models,” 2023, arXiv:2303.18223v16 [cs.CL].
V. R. Veerannagari, Developing A Professor Recommendation System, Ames, Io, USA: Iowa State University, 2024.
H. I. Pohan et al., “Recommender System Using Transformer Model: A Systematic Literature Review,” in Proc. of 2022 1st International Conference on Information System & Information Technology (ICISIT), pp. 376–381, 2022, doi:10.1109/icisit54091.2022.9873070
Sun F. et al, “BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer,” in Proc. of the 28th ACM international conference on information and knowledge management, pp. 1441–1450, 2019; doi:10.1145/3357384.3357895.
W.-C. Kang and J. McAuley, “Self-Attentive Sequential Recommendation,” in Proc. of the 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206, 2018; doi:10.1109/icdm.2018.00035
L. Breiman et al., Classification and Regression Trees, Belmont, CA, USA:Wadsworth Int. 1984.
L. Prokhorenkova et al., “CatBoost: unbiased boosting with categorical features,” in Advances in neural information processing systems, vol. 31, pp. 1–11, 2018.
A. P. Bradley, “The use of the area under the ROC curve in the evaluation of machine learning algorithms,” Pattern Recognition, vol. 30, no. 7, pp. 1145–1159, 1997; doi:10.1016/s0031-3203(96)00142-2
V. Solovyev et al., “A BERT-Based Classification Model: The Case of Russian Fairy Tales,” Journal of Language and Education, vol. 10, no. 4, pp. 98–111, 2024; doi:10.17323/jle.2024.24030
N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992, 2019; doi:10.18653/v1/d19-1410
A. Snegirev et al., “The Russian-focused embedders’ exploration: ruMTEB benchmark and Russian embedding model design,” in Proc. of the 2025 Conf. of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 236–254, 2025; doi:10.18653/v1/2025.naacl-long.12
Wang L. et al., “Multilingual e5 text embeddings: A technical report,” 2024, arXiv:2402.05672v1 [cs.CL].
J. Devlin et al., “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 4171–4186, 2019; doi:10.18653/v1/N19-1423
A. Paszke et al., “Pytorch: An imperative style, high-performance deep learning library,” 2019, arXiv:1912.01703v1 [cs.LG].
D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),”2016, arXiv:1606.08415v5 [cs.LG].
Hinton G. E. et al., “Improving neural networks by preventing co-adaptation of feature detectors,” 2012, arXiv:1207.0580v1 [cs.NE].
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016, arXiv:1607.06450v1 [stat.ML].
B. Barz and J. Denzler, “Deep Learning on Small Datasets without Pre-Training using Cosine Loss,” in Proc. of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020; doi:10.1109/wacv45572.2020.9093286
F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of machine Learning research, vol. 12, pp. 2825–2830, 2011.
M. D. Zeiler, “Adadelta: an adaptive learning rate method,” 2012, arXiv:1212.5701v1 [cs.LG].
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” 2017, arXiv:1711.05101v3 [cs.LG].
Материал публикуется под лицензией:
