A bibliometric analysis of publications on ChatGPT in education: Research patterns and topics

Marina R. Zheltukhina 1, Olga V. Sergeeva 2, Alfiya R. Masalimova 3 * , Roza L. Budkevich 4, Nikolay N. Kosarenko 5, Georgy V. Nesterov 6
More Detail
1 Pyatigorsk State University, Pyatigorsk, RUSSIA
2 Kuban State University, Krasnodar, RUSSIA
3 Kazan Federal University, Kazan, RUSSIA
4 Almetyevsk State Oil Institute, Almetyevsk, RUSSIA
5 Plekhanov Russian University of Economics, Moscow, RUSSIA
6 I.M. Sechenov First Moscow State Medical University (Sechenov University), Moscow, RUSSIA
* Corresponding Author
Online Journal of Communication and Media Technologies, Volume 14, Issue 1, Article No: e202405. https://doi.org/10.30935/ojcmt/14103
OPEN ACCESS   1846 Views   1851 Downloads   Published online: 05 Jan 2024
Download Full Text (PDF)

ABSTRACT

This paper aims to conduct a bibliometric analysis and a comprehensive overview of publications on ChatGPT in educational research. This research also aimed to present the bibliometric results to interpret the research patterns and themes of the application of ChatGPT in educational research. The researchers used the VOSviewer program to conduct a bibliometric analysis and identify research patterns and topics in publications indexed in the Scopus database. For this purpose, the researchers used the Scopus database to find related publications. After applying inclusion and exclusion criteria, they found 82 publications and analyzed them using the bibliometric method. This study showed that researchers from 42 countries examined various topics, including academic writing, artificial intelligence’s (AI) potential, and benefits, using ChatGPT in research, exploring best practices, and reviewing AI. The keyword analysis results showed that five clusters emerged from the current studies on ChatGPT in education research. These results showed that researchers focused on understanding the use of ChatGPT in medical and nursing education, generative AI’s ethical dimensions, the effects of ChatGPT on educational outcomes, large language models and medical education, and ChatGPT and AI. In general, the use of ChatGPT in educational contexts and research is frequently discussed in the publications analyzed in this study. In addition, medical and nursing education was the most studied of the many research studies. Based on the obtained results, recommendations for further studies are drawn.
Keywords: artificial intelligence, ChatGPT, bibliometric, ethics

CITATION

Zheltukhina, M. R., Sergeeva, O. V., Masalimova, A. R., Budkevich, R. L., Kosarenko, N. N., & Nesterov, G. V. (2024). A bibliometric analysis of publications on ChatGPT in education: Research patterns and topics. Online Journal of Communication and Media Technologies, 14(1), e202405. https://doi.org/10.30935/ojcmt/14103

REFERENCES

  • Alneyadi, S., & Wardat, Y. (2023a). ChatGPT: Revolutionizing student achievement in the electronic magnetism unit for eleventh-grade students in Emirates schools. Contemporary Educational Technology, 15(4), ep448. https://doi.org/10.30935/cedtech/13417
  • Alneyadi, S., Wardat, Y., Alshannag, Q., & Abu-Al-Aish, A. (2023b). The effect of using smart e-learning app on the academic achievement of eighth-grade students. EURASIA Journal of Mathematics, Science and Technology Education, 19(4), em2248. https://doi.org/10.29333/ejmste/13067
  • Atlas, S. (2023). ChatGPT for higher education and professional development: A guide to conversational AI. University of Rhode Island. https://digitalcommons.uri.edu/cba_facpubs/548
  • Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. SSRN. https://doi.org/10.2139/ssrn.4337484
  • Bauer, E., Greisel, M., Kuznetsov, I., Berndt, M., Kollar, I., Dresel, M., Fischer, M. R., & Fischer, F. (2023). Using natural language processing to support peer-feedback in the age of artificial intelligence: A cross-disciplinary framework and a research agenda. British Journal of Educational Technology, 54(5), 1222-1245. https://doi.org/10.1111/bjet.13336
  • Bearman, M., & Ajjawi, R. (2023). Learning to work with the black box: Pedagogy for a world with artificial intelligence. British Journal of Educational Technology, 54(5), 1160-1173. https://doi.org/10.1111/bjet.13337
  • Chaudhry, I. S., Sarwary, S. A. M., El Refae, G. A., & Chabchoub, H. (2023). Time to revisit existing student’s performance evaluation approach in higher education sector in a new era of ChatGPT–A case study. Cogent Education, 10(1), 2210461. https://doi.org/10.1080/2331186x.2023.2210461
  • Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32, 444-452. https://doi.org/10.1007/s10956-023-10039-y
  • Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2190148
  • Crawford, J., Cowling, M., & Allen, K. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice, 20(3). https://doi.org/10.53761/1.20.3.02
  • Dawson, P., & Sutherland-Smith, W. (2018). Can markers detect contract cheating? Results from a pilot study. Assessment & Evaluation in Higher Education, 43(2), 286-293. https://doi.org/10.1080/02602938.2017.1336746
  • Dawson, P., Sutherland-Smith, W., & Ricksen, M. (2020). Can software improve marker accuracy at detecting contract cheating? A pilot study of the Turnitin authorship investigate alpha. Assessment & Evaluation in Higher Education, 45(4), 473-482. https://doi.org/10.1080/02602938.2019.1662884
  • Day, T. (2023). A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. The Professional Geographer. https://doi.org/10.1080/00330124.2023.2190373
  • Deng, J., & Lin, Y. (2023). The benefits and challenges of ChatGPT: An overview. Frontiers in Computing and Intelligent Systems, 2(2), 81-83. https://doi.org/10.54097/fcis.v2i2.4465
  • Dowling, M., & Lucey, B. (2023). ChatGPT for (finance) research: The Bananarama conjecture. Finance Research Letters, 53, 103662. https://doi.org/10.1016/j.frl.2023.103662
  • Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., . . ., & Wright, R. (2023). Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
  • Emenike, M. E., & Emenike, B. U. (2023). Was this title generated by ChatGPT? Considerations for artificial intelligence text-generation software programs for chemists and chemistry educators. Journal of Chemical Education, 100(4), 1413-1418. https://doi.org/10.1021/acs.jchemed.3c00063
  • Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education, 9, e46885. https://doi.org/10.2196/46885
  • Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2195846
  • Fernandez, P. (2023). “Through the looking glass: Envisioning new library technologies” AI-text generators as explained by ChatGPT. Library Hi Tech News, 40(3), 11-14. https://doi.org/10.1108/lhtn-02-2023-0017
  • Ferrucci, D., Levas, A., Bagchi, S., Gondek, D., & Mueller, E. T. (2013). Watson: Beyond jeopardy! Artificial Intelligence, 199-200, 93-105. https://doi.org/10.1016/j.artint.2012.06.009
  • Floridi, L., & Chiriatti, M. (2020). GPT-3: Its nature, scope, limits, and consequences. Minds and Machines, 30(4), 681-694. https://doi.org/10.1007/s11023-020-09548-1
  • Frederick, D. E. (2023). ChatGPT: A viral data-driven disruption in the information environment. Library Hi Tech News, 40(3), 4-10. https://doi.org/10.1108/lhtn-04-2023-0063
  • Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digital Medicine, 6, 75. https://doi.org/10.1038/s41746-023-00819-6
  • Gardner, D. E., & Giordano, A. N. (2023). The challenges and value of undergraduate oral exams in the physical chemistry classroom: A useful tool in the assessment toolbox. Journal of Chemical Education, 100(5), 1705-1709. https://doi.org/10.1021/acs.jchemed.3c00011
  • Giannos, P., & Delardas, O. (2023). Performance of ChatGPT on UK standardized admission tests: Insights from the BMAT, TMUA, LNAT, and TSA examinations. JMIR Medical Education, 9, e47737. https://doi.org/10.2196/47737
  • Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9, e45312. https://doi.org/10.2196/45312
  • González-Pérez, L. I., & Ramírez-Montoya, M. S. (2022). Components of education 4.0 in 21st century skills frameworks: Systematic review. Sustainability, 14(3), 1493. https://doi.org/10.3390/su14031493
  • Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), ep421. https://doi.org/10.30935/cedtech/13036
  • Hamzaçebi, C., Akay, D., & Kutay, F. (2009). Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting. Expert Systems With Applications, 36(2), 3839-3844. https://doi.org/10.1016/j.eswa.2008.02.042
  • Hassabis, D. (2017). Artificial intelligence: Chess match of the century. Nature, 544(7651), 413-414. https://doi.org/10.1038/544413a
  • Humphry, T., & Fuller, A. L. (2023). Potential ChatGPT use in undergraduate chemistry laboratories. Journal of Chemical Education, 100(4), 1434-1436. https://doi.org/10.1021/acs.jchemed.3c00006
  • Ivanov, S., & Soliman, M. (2023). Game of algorithms: ChatGPT implications for the future of tourism education and research. Journal of Tourism Futures, 9(2), 214-221. https://doi.org/10.1108/jtf-02-2023-0038
  • Jarrah, A. M., Wardat, Y., & Fidalgo, P. (2023). Using ChatGPT in academic writing is (not) a form of plagiarism: What does the literature say? Online Journal of Communication and Media Technologies, 13(4), e202346. https://doi.org/10.30935/ojcmt/13572
  • Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28, 15873-15892. https://doi.org/10.1007/s10639-023-11834-1
  • Karaali, G. (2023). Artificial intelligence, basic skills, and quantitative literacy. Numeracy, 16(1), 9. https://doi.org/10.5038/1936-4660.16.1.1438
  • Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., . . ., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
  • Kooli, C. (2023). Chatbots in education and research: A critical examination of ethical implications and solutions. Sustainability, 15(7), 5614. https://doi.org/10.3390/su15075614
  • Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790
  • Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410
  • Masters, K. (2023a). Ethical use of artificial intelligence in health professions education: AMEE guide no. 158. Medical Teacher, 45(6), 574-584. https://doi.org/10.1080/0142159x.2023.2186203
  • Masters, K. (2023b). Medical teacher’s first ChatGPT’s referencing hallucinations: Lessons for editors, reviewers, and teachers. Medical Teacher, 45(7), 673-675. https://doi.org/10.1080/0142159x.2023.2208731
  • McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the Dartmouth summer research project on artificial intelligence, august 31, 1955. AI Magazine, 27(4), 12.
  • Mhlanga, D. (2023). Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. SSRN. https://doi.org/10.2139/ssrn.4354422
  • OpenAI. (2022). ChatGPT. https://openai.com/blog/ChatGPT/
  • Perkins, M. (2023). Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching and Learning Practice, 20(2). https://doi.org/10.53761/1.20.02.07
  • Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1), 342-363. https://doi.org/10.37074/jalt.2023.6.1.9
  • Skavronskaya, L., Hadinejad, A. H., & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion of ChatGPT in tourism education. Journal of Teaching in Travel & Tourism, 23(2), 253-258. https://doi.org/10.1080/15313220.2023.2196658
  • Sok, S., & Heng, K. (2023). ChatGPT for education and research: A review of benefits and risks. SSRN. https://doi.org/10.2139/ssrn.4378735
  • Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments. https://doi.org/10.1080/10494820.2023.2209881
  • Subramani, M., Jaleel, I., & Krishna Mohan, S. (2023). Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Advances in Physiology Education, 47(2), 270-271. https://doi.org/10.1152/advan.00036.2023
  • Sun, G. H., & Hoelscher, S. H. (2023). The ChatGPT storm and what faculty can do. Nurse Educator, 48(3), 119-124. https://doi.org/10.1097/nne.0000000000001390
  • Teixeira da Silva, J. A. (2023). Is ChatGPT a valid author? Nurse Education in Practice, 68, 103600. https://doi.org/10.1016/j.nepr.2023.103600
  • Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science, 379(6630), 313. https://doi.org/10.1126/science.adg7879
  • Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, 15. https://doi.org/10.1186/s40561-023-00237-x
  • Uddin, S. M. J., Albert, A., Ovid, A., & Alsharef, A. (2023). Leveraging ChatGPT to aid construction hazard recognition and support safety education and training. Sustainability, 15(9), 7121. https://doi.org/10.3390/su15097121
  • van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: Five priorities for research. Nature, 614(7947), 224-226. https://doi.org/10.1038/d41586-023-00288-7
  • van Eck, N. J., & Waltman, L. (2020). VOSviewer manual. https://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.8.pdf
  • Wardat, Y., Tashtoush, M. A., AlAli, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. EURASIA Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272
  • Wu, R., & Yu, Z. (2023). Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13334
  • Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28, 13943-13967. https://doi.org/10.1007/s10639-023-11742-4
  • Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–Where are the educators? International Journal of Educational Technology in Higher Education, 16, 39. https://doi.org/10.1186/s41239-019-0171-0