AHMET CELİK
Journal of Clinical Trials and Experimental Investigations - 2024;3(4):156-162
Objective:This study aims to evaluate the accuracy of ChatGPT and Gemini AI in the field of otorhinolaryngology.Materials and methods: This study evaluated the performance of ChatGPT 4.0 and Gemini AI in answering 150 multiple-choice questions evenly distributed across otorhinolaryngology domains: ear, nose, and throat. Both models were tested under standardized conditions, with their responses compared to an answer key. The true and false answers were evaluated.Results: For ear-related questions, ChatGPT correctly answered 34 (68%), while Gemini AI correctly answered 33 (66%) (p=0.832). For nose-related questions, both models achieved identical results: 34 correct answers (68%) and 16 incorrect answers (32%) (p=1.000). For throat-related questions, ChatGPT provided 34 correct answers (68%) compared to Gemini AI’s 38 correct answers (76%) (p=0.373). Overall, ChatGPT achieved 102 correct answers (68%) and Gemini AI achieved 105 (70%), with no statistically significant difference between the models (p=0.708). The total correct answers across all topics were 207 (69%), and incorrect answers were 91 (31%). Binary logistic regression showed no significant differences in performance between the AI models or topics, confirming their comparable accuracy in otorhinolaryngology question sets.Conclusion: ChatGPT 4.0 and Gemini AI demonstrated comparable performance in answering otorhinolaryngology questions, with no statistically significant differences observed across ear, nose, and throat topics. Both models achieved high accuracy rates (ChatGPT: 68%, Gemini AI: 70%), suggesting their potential applicability in clinical decision-making and supporting otorhinolaryngology-related diagnostics.
