READABILITY, RELIABILITY, AND QUALITY OF AI CHATBOT RESPONSES ON HOME MECHANICAL VENTILATION

Ferhan Demirer AYDEMİR, Volkan HANCI

Journal of Medicine and Palliative Care - 2026;7(3):553-561

Division of Intensive Care Medicine, Department of Internal Medicine, Faculty of Medicine, Çanakkale Onsekiz Mart University, Çanakkale, Turkiye

 

Aims: Home mechanical ventilation (HMV) is an essential component of long-term respiratory care. Patients and caregivers increasingly use online resources, including Artificial Intelligence (AI)-based chatbots, to obtain information regarding HMV. However, the readability, reliability, and quality of AI-generated information on HMV remain unclear. This study aimed to evaluate the readability, reliability, and quality of responses generated by ChatGPT, Gemini, and Perplexity for frequently asked questions about HMV. Methods: A cross-sectional content analysis was conducted in December 2025 using 47 high-interest HMV-related questions identified via Google Trends. Each question was entered verbatim into ChatGPT, Gemini, and Perplexity under standardized conditions. Readability was assessed using seven validated indices. Reliability was evaluated using the JAMA benchmark criteria and the modified DISCERN instrument, while overall educational quality was assessed using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) tool. Non-parametric statistical tests were applied for inter-model comparisons. Results: All AI-generated responses significantly exceeded the recommended sixth-grade readability threshold across all indices (p<0.001). ChatGPT produced comparatively lower grade-level readability scores, whereas Perplexity demonstrated significantly higher reliability and quality scores. Gemini showed intermediate performance across most metrics. Significant inter-model differences were observed for multiple readability, reliability, and quality measures. A consistent association was identified between higher informational quality scores and increased textual complexity. Conclusion: AI-generated information on HMV is generally more complex than recommended for patients. Differences between platforms suggest that AI-generated medical content is not interchangeable and should be used only as a supplement to professional medical guidance.