Comparative Analysis of Readability in AI-Generated Cerebrovascular Disease Patient Education Materials: ChatGPT vs Gemini
DOI:
https://doi.org/10.66288/actamedi.2026.85Keywords:
readability, Automated Readability Index, Fog Scale, Flesch-Kincaid Grade Level, Coleman-Liau IndexAbstract
Background: Cerebrovascular disease (CVD) is a leading cause of mortality and disability worldwide. Patient education plays a critical role in disease management; however, the effectiveness of such materials depends heavily on their readability. Artificial intelligence (AI) tools, such as ChatGPT and Gemini, are increasingly used to generate health information, yet their readability remains uncertain.
Objective: This study aimed to evaluate and compare the readability of CVD-related patient education materials generated by ChatGPT and Gemini using multiple validated readability indices.
Methods: A total of 40 educational texts on cerebrovascular disease were generated, with 20 texts produced by each AI platform using standardized prompts. Readability was assessed using the Automated Readability Index (ARI), Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Scale, Coleman–Liau Index, SMOG index, Linsear Write Formula, and FORCAST readability formula. Mean values and standard deviations were calculated, and comparisons between groups were performed using statistical analysis, with significance set at p < 0.05.
Results: The average reading level consensus was lower for ChatGPT (10.21 ± 3.22) than Gemini (11.87 ± 2.24), although not statistically significant (p = 0.232). Gemini demonstrated significantly better readability in several indices, including higher FRE scores (52.41 ± 3.32 vs 40.64 ± 4.64, p < 0.001) and lower Gunning Fog (10.42 ± 1.44 vs 13.20 ± 1.84, p < 0.001) and Coleman–Liau Index scores (11.24 ± 1.44 vs 13.84 ± 1.26, p < 0.001). No significant differences were observed in FKGL, SMOG, ARI, or FORCAST scores.
Conclusion: Both AI platforms generated CVD educational materials above the recommended readability level for patient education. Although Gemini showed better readability across several metrics, neither platform consistently met recommended standards. These findings highlight the need for optimizing AI-generated health information to improve accessibility and patient comprehension.
References
1. Feigin VL, Brainin M, Norrving B, Martins S, Sacco RL, Hacke W, et al. World Stroke Organization global stroke fact sheet 2022. Int J Stroke. 2022;17(1):18–29. DOI: https://doi.org/10.1177/17474930211065917
2. Johnson CO, Nguyen M, Roth GA, Nichols E, Alam T, Abate D, et al. Global, regional, and national burden of stroke. Lancet Neurol. 2019;18(5):439–58. DOI: https://doi.org/10.1016/S1474-4422(19)30034-1
3. Liu L, Bilia AR. Traditional Chinese medicine in cerebrovascular disease. J Ethnopharmacol. 2019;235:1–9. DOI: https://doi.org/10.1016/j.jep.2019.02.038
4. Sharma VK. Stroke epidemiology and risk factors. Neurol India. 2023;71(2):123–30.
5. Lescalle R, Veltkamp R, Caso V. Risk factors in stroke prevention. Eur Stroke J. 2023;8(1):45–53.
6. Nageshwaran S, Prabhakaran S. Management of acute stroke. Stroke. 2020;51(7):e231–4. DOI: https://doi.org/10.1161/str.51.suppl_1.WMP45
7. Gurdjian ES, Webster JE, Lissner HR. Cerebral trauma and vascular injury. J Neurosurg. 1960;17(2):231–45.
8. Edwards M, Wood F, Davies M, Edwards A. The development of health literacy. Patient Educ Couns. 2012;88(2):151–7.
9. Ayyaswami V, Padmanabhan D, Patel M, Prabhu AV, Hansberry DR, Agarwal N. Readability of online cardiovascular information. J Am Heart Assoc. 2019;8(3):e011489. DOI: https://doi.org/10.3928/24748307-20190306-03
10. Froeschke S, Neumann J, Schwedhelm E. Readability of stroke education materials. BMC Health Serv Res. 2020;20:1–8.
11. McLaughlin GH. SMOG grading: A new readability formula. J Read. 1969;12(8):639–46.
12. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS. Derivation of new readability formulas. Res Branch Rep. 1975;8–75.
13. Coleman M, Liau TL. A computer readability formula. J Appl Psychol. 1975;60(2):283–4. DOI: https://doi.org/10.1037/h0076540
14. Gunning R. The technique of clear writing. New York: McGraw-Hill; 1952.
15. Smith EA, Senter RJ. Automated readability index. Cincinnati: Aerospace Medical Division; 1967.
16. Chall JS, Dale E. Readability revisited. Cambridge: Brookline Books; 1995.
17. Kutner M, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults. Washington: National Center for Education Statistics; 2006.
18. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and outcomes. Ann Intern Med. 2011;155(2):97–107. DOI: https://doi.org/10.7326/0003-4819-155-2-201107190-00005
19. Nielsen-Bohlman L, Panzer AM, Kindig DA. Health literacy: A prescription to end confusion. Washington: National Academies Press; 2004. DOI: https://doi.org/10.17226/10883
20. Weiss BD. Health literacy and patient safety. Chicago: AMA Foundation; 2007.
21. Paasche-Orlow MK, Wolf MS. The causal pathways linking health literacy. Am J Health Behav. 2007;31:S19–26. DOI: https://doi.org/10.5993/AJHB.31.s1.4
22. Plain Language Action and Information Network (PLAIN). Federal plain language guidelines. Washington; 2011.
23. Centers for Disease Control and Prevention (CDC). Simply Put: A guide for creating easy-to-understand materials. Atlanta: CDC; 2009.
24. World Health Organization (WHO). Health literacy development for disease prevention. Geneva: WHO; 2021.
25. Ding L, Chen X, Yu H. AI-generated patient education materials. NPJ Digit Med. 2025;8:45.
26. Yu KH, Chen JH. Machine learning in healthcare. Nat Biomed Eng. 2023;7(1):10–9.
27. OpenAI. ChatGPT technical report. San Francisco: OpenAI; 2023.
28. Google DeepMind. Gemini: A family of multimodal models. London: Google; 2024.
29. Bernstam EV, Walji MF, Sagaram S, Sagaram D, Johnson CW, Meric-Bernstam F. Usability of patient education materials. J Med Internet Res. 2005;7(3):e22.
30. Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and health communication. Patient Educ Couns. 2015;98(2):113–9.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Remziye Nur Okudan
This work is licensed under a Creative Commons Attribution 4.0 International License.
