Intelligent voice assistant as an example of inclusive design methodology implementation

Introduction. The development of artificial intelligence methods and technologies aimed at speech recognition contributes to the creation of specialised programmes – namely voice assistants – which are capable of conducting a dialogue in natural language. Such services are of particular relevance in inclusive education in order to support students with visual impairments. The research problem lies in the individualised support of students’ independent work based on a voice assistant and is determined by the contradiction between the widespread use of various question-answer systems in business and everyday life (including those with a voice interface), on the one hand, and insufficient knowledge their didactic possibilities, on the other hand.

Aim. The present research aims to investigate and practically implement the methodology of inclusive design of digital educational resources on the example of creating an intelligent voice assistant for students’ independent work in the course “Computer Networks”.

Methodology and research methods. The current research is based on the methodology of inclusive design in combination with an ontological approach in relation to the creation of digital educational resources, speech recognition methods, methods for designing intelligent systems and knowledge bases, methods and technologies for designing and implementing automated learning systems with feedback. To recognise questions, search for answers, and support dialogues carried out by a voice assistant, the authors applied natural language text analysis methods and classification models created using machine learning methods.

Results. The authors have developed and substantiated the requirements that a digital educational resource must meet in accordance with the principles of inclusive design, linking an ontological approach to content development, automatic individualised support for students, and monitoring the achievement of educational results. According to the formulated requirements, the authors have developed an interactive computer service – an intelligent voice assistant that provides support for students’ independent work when performing practical tasks, using the “Computer Networks” course as an example. The service supports voice input and subsequent interpretation of questions, search for answers in the knowledge base with voice output of the result, and implements the execution of operations according to certain rules.

Scientific novelty. The authors have clarified the content of the concept of “inclusive design” in the context of digital educational resources, when the key feature is the focus on continuous improvement of the didactic capabilities of a particular product. The authors have shown that this can be achieved through a conceptually based content structure and initially provided feedback. This approach has proven to be effective in designing an intelligent voice assistant to answer student questions and to automatically perform operations on a computer.

Practical significance. The use of a voice assistant by students of the Institute of Mathematics and Computer Science of the University of Tyumen in the process of studying the course “Computer Networks” demonstrated the relevance of developing similar question-answering systems to accompany the independent work of students, including those with visual impairments in online and blended learning. The developed service is universal and can be used with any knowledge base that provides answers to students’ questions.

1. Standard I. S. EN 17161:2019 – Design for all [Internet]. 2019 [cited 2023 Aug 03]; Available from: https://universaldesign.ie/products-services/i-s-en-17161-2019-design-for-all-accessibility-followinga-design-for-all-approach-in-products-goods-and-services-extending-the-range-of-users/

2. Clarkson P. J., Coleman R. History of inclusive design in the UK. Applied Ergonomics. 2015; 46: 235–247. DOI: 10.1016/j.apergo.2013.03.002

3. Dong H., McGinley C., Nickpour F., Cifter A. S. Designing for designers: Insights into the knowledge users of inclusive design. Applied Ergonomics. 2015; 46: 284–291. DOI: 10.1016/j.apergo.2013.03.003

4. Kotov S. V. Inclusive education: Transformation of the educational system. Izvestiya Yuzhnogo federal’nogo universiteta. Pedagogicheskie nauki = The World of Academia: Culture, Education [Internet]. 2016 [cited 2023 Aug 03]; 12: 30–36. Available from: https://www.elibrary.ru/download/elibrary_28140896_65083749.pdf (In Russ.)

5. Lavrentieva Z. I., Lavrentieva O. A. Universal design as a way of organizing the educational space of an inclusive school. Izvestiya Voronezhskogo gosudarstvennogo pedagogicheskogo universiteta = Proceedings of the Voronezh State Pedagogical University [Internet]. 2020 [cited 2023 Aug 03]; 1: 85–88. Available from: http://izvestia.vspu.ac.ru/content/izvestia_2020_v286_N1/85-88.pdf (In Russ.)

6. Oliveira A. R. P., Munster M. A., Gonçalves A. G. Universal design for learning and inclusive education: A systematic review in the international literature. Revista Brasileira de Educação Especial. 2019; 25: 675–690. DOI: 10.1590/s1413-65382519000400009

7. Kovyazina E. S., Volosnikova L. M. Inclusive education quality management based on conception of universal design for learning. Pedagogika. Voprosy teorii i praktiki = Pedagogy. Theory & Practice. 2020; 4: 437–442. DOI: 10.30853/ped200104 (In Russ.)

8. Persson H., Åhman H., Yngling A. A., Gulliksen J. Universal design, inclusive design, accessible design, design for all: Different concepts – one goal? On the concept of accessibility – historical, methodological and philosophical aspects. Universal Access in the Information Society. 2015; 14 (4): 505–526. DOI: 10.1007/s10209-014-0358-z

9. Moriña A. Inclusive education in higher education: Challenges and opportunities. European Journal of Special Needs Education. 2017: 32 (1): 3–17. DOI: 10.1080/08856257.2016.1254964

10. Mohammed P. S., Nell’Watson E. Towards inclusive education in the age of artificial intelligence: Perspectives, challenges, and opportunities. In: Knox J., Wang Y., Gallagher M. (Eds.). Artificial intelligence and inclusive education. Singapore: Springer; 2019. p. 17–37. DOI: 10.1007/978-981-13-8161-4_2

11. Pacheco E., Lips M., Yoong P. Transition 2.0: Digital technologies, higher education, and vision impairment. The Internet and Higher Education. 2018; 37: 1–10. DOI: 10.1016/j.heduc.2017.11.001

12. Farhan W., Razmak J. A comparative study of an assistive e-learning interface among students with and without visual and hearing impairments. Disability and Rehabilitation: Assistive Technology. 2022; 17 (4): 431–441. DOI: 10.1080/17483107.2020.1786733

13. Azeta A. A., Daramola O. A voice-based e-examination framework for visually impaired students in open and distance learning. Turkish Online Journal of Distance Education. 2018; 19 (2): 34–46. DOI: 10.17718/tojde.415635

14. Dharmasena I. H. H. N., Jayakody J. A. D. C. A. Voice-based online examination system for visually impaired students. In: Proceedings of the 2nd International Conference on Advanced Research in Computing (ICARC); 2022 Feb 23–24; Belihuloya, Sri Lanka. IEEE; 2022. p. 367–372. DOI: 10.1109/ICARC54489.2022.9754191

15. Rajagopal A., Vedamanickam N. New approach to human AI interaction to address digital divide& AI divide: Creating an interactive AIplatform to connect teachers & students. In: Proceedings of the IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT); 2019 Feb 20–22; Coimbatore, India. IEEE; 2019. p. 1–6. DOI: 10.1109/ICECCT.2019.8869174

16. González-Castro N., Muñoz-Merino P. J., Alario-Hoyos C. Adaptive learning module for a conversational agent to support MOOC learners. Australasian Journal of Educational Technology. 2021; 37 (2): 24–44. DOI: 10.14742/ajet.6646

17. Hamal O., Faddouli N. E. Intelligent system using deep learning for answering learner questions in a MOOC. International Journal of Emerging Technologies in Learning (iJET). 2022; 17 (2): 32–42. DOI: 10.3991/ijet.v17i02.26605

18. Jung S., Son M., Kim C. I., Rew J., Hwang E. Video-based learning assistant scheme for sustainable education. New Review of Hypermedia and Multimedia. 2019; 25 (3): 161–181. DOI: 10.1080/13614568.2019.1678682

19. V. Shmyrev N. V. Vosk speech recognition toolkit: Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node [Internet]. 2023 [cited 2023 Aug 03]. Available from: https://github.com/alphacep/vosk-api

20. Bird S., Klein E., Loper E. Natural language processing with Python: Analyzing Text with the natural language toolkit [Internet]. Sebastopol, CA: O’Reilly Media; 2009 [cited 2023 Aug 03]. 504 p. Available from: https://mathewscustoms.com/pdf-files/Python/Natural%20Language%20Processing%20with%20Python.pdf

21. Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Duchesnay E. Scikitlearn: Machine learning in Python. Journal of Machine Learning Research [Internet]. 2011 [cited 2023 Aug 03]; 12: 2825–2830. Available from: https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf

22. Mouromtsev D. I. Models and methods of e-learning individualization in the context of ontological approach. Ontologiya proektirovaniya = Ontology of Designing. 2020; 1 (35): 34–49. DOI: 10.18287/2223-9537-2020-10-1-34-49 (In Russ.)

23. Solovov A. V., Menshikova A. A. Designing an ontology of the e-learning course content. Ontologiya proektirovaniya = Ontology of Designing. 2023; 1 (47): 99–112. DOI: 10.18287/2223-9537-2023-13-1-99-112

24. Stancin K., Poscic P., Jaksic D. Ontologies in education–state of the art. Education and Information Technologies. 2020; 25 (6): 5301–5320. DOI: 10.1007/s10639-020-10226-z

25. Rahayu N. W., Ferdiana R., Kusumawardani S. S. A systematic review of ontology use in E-Learning recommender system. Computers and Education: Artificial Intelligence [Internet]. 2022 [cited 2023 Aug 03]; 3. Available from: https://www.sciencedirect.com/science/article/pii/S2666920X22000029

26. Sikos L. F. Cybersecurity knowledge graphs. Knowledge and Information Systems [Internet]. 2023; 65: 3511–3531. DOI: 10.1007/s10115-023-01860-3

27. Zakharov A. A., Shabalin A. M., Khanbekov Sh. I., Dzhalilzoda D. B., Pоnomarev K. Y. Using a voice assistant as a virtual consultant for administration of the security of the local computer network infrastructure. Vestnik UrFO. Bezopasnost’ v informacionnoj sfere = Bulletin of the Ural Federal District. Security in the Information Sphere. 2022; 4 (46): 68–75. DOI: 10.14529/secur220408 (In Russ.)

28. Soares M. A. C., Parreiras F. S. A literature review on question answering techniques, paradigms and systems. Journal of King Saud University-Computer and Information Sciences. 2020; 32 (6): 635–646. DOI: 10.1016/j.ksuci.2018.08.005

29. Soares T. G., Azhari A., Rokhman N., Wonarko E. Education question answering systems: a survey. In: Proceedings of the International MultiConference of Engineers and Computer Scientists (IMECS 2021) [Internet]; 2021 Oct 20–22; Hong Kong. IAENG; 2021 [cited 2023 Aug 03]; p. 24–34. Available from: https://www.iaeng.org/publication/IMECS2021/IMECS2021_pp24-34.pdf

30. Caballero M. A brief survey of question answering systems. International Journal of Artificial Intelligence & Applications (IJAIA) [Internet]. 2021 [cited 2023 Aug 03]; 12 (5). Available from: https://aircconline.com/ijaia/V12N5/12521ijaia01.pdf

31. Kurdi G., Leo J., Parsia B., Sattler U., Al-Emari S. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education. 2020; 30: 121–204. DOI: 10.1007/s40593-019-00186-y

32. Holmes W., Tuomi, I. State of the art and practice in AI in education. European Journal of Education. 2022; 57: 542–570. DOI: 10.1111/ejed.12533

33. Premarathne H. H. N. An overview on accessibility to the graphical study materials for visually impaired or blind students. International Journal of Computer Trends and Technology. 2019; 67 (4): 134–138. DOI: 10.14445/22312803/IJCTT-V67I4P127

34. Terzopoulos G., Satratzemi M. Voice assistants and smart speakers in everyday life and in education. Informatics in Education. 2020; 19 (3): 473–490. DOI: 10.15388/infedu.2020.21

35. Cotton D. R., Cotton P. A., Shipway J. R. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. 2023. DOI: 10.1080/14703297.2023.2190148

36. Atlas S. ChatGPT for higher education and professional development: A guide to conversational AI. University of Rode Island Publications [Internet]. 2023 [cited 2023 Aug 03]. Available from: https://digitalcommons.uri.edu/cba_facpubs/548

37. Rudolph J., Tan Sh., Tan S. ChatGPT: “Bullshit” spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching. 2023; 6 (1): 242–263. DOI: 10.37074/jalt.2023.6.1.9