Huriye Aybüke KOÇ, Elif ÖZENİR, Cansu Altınöz GÜNEY
Turkish Journal of Dermatology - 2026;20(2):58-63
Aim: This study aimed to evaluate and compare the readability and quality of information in responses generated by artificial intelligence (AI) models to patients' frequently asked questions about systemic isotretinoin, a medication commonly prescribed in dermatology. Materials and Methods: Thirty-four frequently asked questions from patients using isotretinoin were prepared by a team of dermatology specialists. These questions were posed to three AI-based text-generation tools (ChatGPT, Gemini 2.0, and Copilot), and the responses were analyzed. The resulting texts were compared in terms of readability levels [Flesch Reading Ease score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), Gunning Fog index (GFOG), Coleman-Liau index (CLI), and automated readability index (ARI)], sentence lengths, and content quality, which was evaluated by dermatologists. Results: None of the AI models achieved the optimal readability threshold (FRES >= 60). Readability metrics differed significantly among models. Gemini produced responses that were significantly less readable and more complex than those produced by ChatGPT and Copilot across all readability indices, including FRES, FKGL, SMOG, GFOG, CLI, and ARI; post-hoc analyses confirmed differences between Gemini and the other models. Sentence counts also differed significantly, with Gemini generating longer responses than Copilot. In contrast, Likert-based quality scores and response appropriateness were comparable across models, with no statistically significant differences observed. Conclusion: This study demonstrates that AI models produce academic responses that are difficult for those unfamiliar with medical terminology to understand, and can generate outputs with variable readability in health-related content. These findings highlight the need for careful evaluation of AI-based content for use in healthcare.
