Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examination
Article
Khorshidi, Hamid, Mohammadi, Afshin, Yousem, David M., Abolghasemi, Jamileh, Ansari, Golnoosh, Mirza-Aghazadeh-Attari, Mohammad, Acharya, U Rajendra and Ardakani, Ali Abbasian. 2023. "Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examination." Informatics in Medicine Unlocked. 41. https://doi.org/10.1016/j.imu.2023.101314
Article Title | Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examination |
---|---|
ERA Journal ID | 212809 |
Article Category | Article |
Authors | Khorshidi, Hamid, Mohammadi, Afshin, Yousem, David M., Abolghasemi, Jamileh, Ansari, Golnoosh, Mirza-Aghazadeh-Attari, Mohammad, Acharya, U Rajendra and Ardakani, Ali Abbasian |
Journal Title | Informatics in Medicine Unlocked |
Journal Citation | 41 |
Article Number | 101314 |
Number of Pages | 6 |
Year | 2023 |
Publisher | Elsevier |
Place of Publication | United Kingdom |
ISSN | 2352-9148 |
Digital Object Identifier (DOI) | https://doi.org/10.1016/j.imu.2023.101314 |
Web Address (URL) | https://www.sciencedirect.com/science/article/pii/S2352914823001600 |
Abstract | Background: ChatGPT is a large language model (LLM) artificial intelligence instrument trained on massive amounts of text data extracted from the internet and/or user input. In the present article, we aim to apply the latest version of ChatGPT to the Iranian Medical Residency Examination. Methods: The Iranian Medical Residency Examination is composed of 200 multichoice questions covering all domains of medicine. We used ChatGPT to translate questions into English, French, and Spanish. We fed the questions as multiple-choice questions and allowed ChatGPT to provide comprehensive answers and justifications for its choices. Results: ChatGPT was able to answer 161 (81.3% = 161/198) questions correctly when the Persian language was used. When the questions were translated into English, French, and Spanish, ChatGPT answered six, one, and five additional questions correctly, respectively. When comparing the different languages, there was no significant difference in the functioning of ChatGPT in different languages using either the McNemar test or the Binomial test. Conclusion: ChatGPT can deliver above-average performance in the Iranian Medical Residency Examination, demonstrating its potential for using language models in medicine. © 2023 The Authors |
Keywords | ChatGPT; USMLE; Medical education ; Language model |
ANZSRC Field of Research 2020 | 400306. Computational physiology |
Byline Affiliations | University of Padova, Italy |
Urmia University of Medical Science, Iran | |
Johns Hopkins University, United States | |
Iran University of Medical Sciences, Iran | |
Northwestern University, United States | |
School of Mathematics, Physics and Computing | |
Shahid Beheshti University of Medical Sciences, Iran |
Permalink -
https://research.usq.edu.au/item/z2615/application-of-chatgpt-in-multilingual-medical-education-how-does-chatgpt-fare-in-2023-s-iranian-residency-entrance-examination
Download files
89
total views37
total downloads3
views this month1
downloads this month