Overview of Existing Methods for Automatic Generation of Tasks with Conditions in Natural Language
Abstract
The paper considers the main algorithms for generating various school subject problems of closed and open type. Some of these algorythms (i.e. question answering, Visual question answering) use artificial intelligence and some not (i.e. sets of AND/OR tree, templates). It was shown that methods for generating tests using artificial intelligence have a high potential, but they require further development, in particular, the creation of large question-answer database in russian language.
References
V. I. Zagvyazinskii, Teoriya obucheniya: Sovremennaya interpretatsiya: Uchebnoe posobie dlya vuzov [Larning Theory: A Modern Interpretation: A Textbook for Universities], Moscow: Akademia, 2006 (in Russian).
V. V. Kraevskii and A. V. Khutorskoi, Osnovy obucheniya: Didaktika i metodika. Ucheb. posobie dlya stud. vyssh. ucheb. zavedenii [Fundamentals of teaching: Didactics and methodology. Textbook for students higher institution], Moscow: Akademia, 2007 (in Russian).
O. V. Mikhailichenko, Metodika prepodavaniya obshchestvennykh distsiplin v vysshei shkole: uchebnoe posobie [Methods of teaching social disciplines in higher education: textbook], Sumy, Ukraine: SumDPU, 2009 (in Russian).
S. V. Tarasenko and N. Yu. Ryazanova, “Analiz metodov avtomaticheskoi generatsii voprosov na estestvennom yazyke” [Analysis of methods for automatic generation of questions in natural language], Inzhenernyi vestnik, no. 12, pp. 1032–1037, 2015 (in Russian).
V. V. Kruchinin and Yu. V. Morozova, “Modeli generatorov voprosov dlya komp’yuternogo kontrolya znanii,” Open and Distance Education, vol. 2, no. 14, pp. 52–62, 2004 (in Russian).
J. R. Slagle, “A Heuristic Program that Solves Symbolic Integration Problems in Freshman Calculus,” Journal of the ACM, vol. 10, no. 4, pp. 507–520, 1963; doi: 10.1145/321186.321193
V. V. Kruchinin, “Ispol’zovanie derev’ev I/ILI dlya generatsii voprosov i zadach” [Using AND/OR trees to generate questions and problems], Tomsk State University Journal, no. 284, pp. 182–186, 2004 (in Russian).
Yu. A. Zorin, “The interpreter of programming language for design generators of tests based on AND/OR trees,” Proceedings of TUSUR University, no. 1(27), pp. 75–79, 2013 (in Russian).
Yu. A. Zorin, “Using algorythms of combinatorial generation in designing tests generators,” Distantsionnoe i virtual’noe obuchenie, no. 6, pp. 54–59, 2013 (in Russian).
J. A. Gonzalez and P. Munoz, “E-status: An automatic web-based problem generator—Applications to statistics,” Computer Applications in Engineering Education, vol. 14, no. 2, pp. 151–159, 2006; doi: 10.1002/cae.20071
J. Camejo, A. Silva, L. Descal¸co, and P. Oliveira, “Modelmaker, a multidisciplinary web application to build question generator models from basic to higher education,” in EDULEARN16 Proceedings, Barcelona, Spain, 2016, pp. 5095–5103; doi: 10.21125/edulearn.2016.2206
R. Rioja, S. Gutierrez-Santos, A. Pardo, and C. Delgado-Kloos, “A Parametric Exercise Based Tutoring System,” in Proc. of 33rd ASEE/IEEE Frontiers in Education Conference. 5–8 November, 2003, Boulder, CO, 2003, pp. S1B20–S1B26.
V. Minchichova, “Rossiya v Industrii 4.0,” Molodoi uchenyi, no. 24 (314), pp. 196–198, 2020 (in Russian)
B. Das, M. Majumder, S. Phadikar, and A. A. Sekh, ”Automatic question generation and answer assessment: a survey,” Research and Practice in Technology Enhanced Learning, vol. 16, no. 5, 2021; doi: 10.1186/s41039-021-00151-1
M. Agarwal, Cloze and open cloze question generation systems and their evaluation guidelines, Hyderabad, India: International Institute of Information Technology, 2012.
M. Heilman and N. A. Smith, “Good question! statistical ranking for question generation”, in Human Language
Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational
Linguistics, pp. 609–617, 2010.
M. Denkowski and Alon Lavie, “Meteor universal: language specific translation evaluation for any target
language,” in Proc. of the 9th Workshop on Statistical Machine Translation. Baltimore, Maryland, USA, pp. 376–380, 2014; doi: 10.3115/v1/W14-3348
I. Labutov, S. Basu, and L. Vanderwende, “Deep questions without deep understanding,” in Proc. of the 53rd
Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on
Natural Language Processing, 2015, p. 889898.
X. Du, J. Shao, and C. Cardie, “Learning to ask: Neural question generation for reading comprehension,” in Proc. of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada., vol. 1, 2017, pp. 1342–13524; doi: 10.18653/v1/P17-1123
X. Yuan, T. Wang, C. Gulcehre, A. Sordoni, P. Bachman, S. Subramanian, S. Zhang, and A. Trischler, Machine
comprehension by text-to-text neural question generation, 2017. [Online]. Available: https://arxiv.org/abs/1705.
L. Song, Z. Wang, and W. Hamza, A unified query-based generative model for question generation and question answering, 2017. [Online]. Available: (https://arxiv.org/abs/1709.01058)
Q. Zhou, N. Yang, F. Wei, C. Tan, H. Bao, and M. Zhou, “Neural question generation from text: A preliminary study,” in X. Huang, J. Jiang, D. Zhao, Y. Feng, Y. Hong, eds., Natural Language Processing and Chinese Computing (NLPCC 2017), Lecture Notes in Computer Science, vol. 10619, Springer, Cham., 2017, pp. 662–671; doi: 10.1007/978-3-319-73618-1_56
L. Vanderwende, “The importance of being important: Question generation,” in Proc. of the 1st Workshop on the Question Generation Shared Task Evaluation Challenge, Arlington, VA., pp. 1–2, 2008.
Stanford NLP Group, “Squad the stanford question answering dataset SQUad 2.0,” in SQuAD. [Online]. Available: https://rajpurkar.github.io/SQuAD-explorer/
M. Heilman, Automatic factual question generation from text, Doctoral dissertation, Carnegie Mellon University, Schenley Park Pittsburgh, PA, USA, 2011.
K. Vachev, et al. “Generating Answer Candidates for Quizzes and Answer-Aware Question Generators,” 2021.
[Online]. Available: https://arxiv.org/pdf/2108.12898.pdf
D. Dip, Automatic Question Generator from Text, 2020. [Online]. Available: https://github.com/dipta-dhar/
Automatic-Question-Generator#
S. Jain, Question Generation, 2020. [Online]. Available: https://github.com/Sanskar-Jain/Automatic-Question-Generator/blob/master/NLP%20Project.pdf
K. Vachev, Question Generation with use of NLP, 2021. [Online]. Available: ttps://github.com/KristiyanVachev/
Question-Generation
Orzhan, Generate questions based on text in Russian with use of Rugpt3, 2021. [Online]. Available: https://github.com/orzhan/rugpt3-question-generation
A. V. Manuilov and V. I. Rodionov, “Osnovy khimii. Internet uchebnik,” in hemi.nsu.ru, 2017 (in Russian). [Online]. Available: http://www.hemi.nsu.ru/ucheb161.html
V. I. Zvonnikov and M. B. Chelyshkova, Sovremennye sredstva otsenivaniya rezul’tatov obucheniya, Moscow:
Akademiya, 2007 (in Russian).
D. Gordon, A. Kembhavi, M. Rastegari, J. Redmon, D. Fox, and A. Farhadi, “Iqa: Visual question answering in
interactive environments,” in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4089–4098.
I. V. Serban, A. Garc ˊia-Duran, C. Gulcehre, S. Ahn, S. Chandar, A. Courville, and Y. Bengio, “Generating Factoid
Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus,” in Proc. of the 54th
Annual Meeting of the Association for Computational Linguistics, vol. 1, 2016, pp. 588–598.
A. Santoro, F. Hill, D. Barrett, A. Morcos, and T. Lillicrap, “Measuring abstract reasoning in neural networks,” in
Proc. of Int.l Conf. on Machine Learning, 2018, pp. 4477–4486.
S. Gaba, Visual question answering, 2020. [Online]. Available: https://github.com/SatyamGaba/visual_question_answering
M. M. Kondragunta, Generating natural questions from image, 2018. [Online]. Available: https://github.com/
gitlost-murali/Natural-Questions-Generation-from-Images
This work is licensed under a Creative Commons Attribution 4.0 International License.
