Author(s)
Nathaniel Hunter, BS
Ella Hawes, BS
Amy Leming, MD
Ahmad Abdelhadi, BS
Shaina Gong, MD
David Allen, MD
W. Katherine Kao, MD
Affiliation(s)
The University of Texas Health Science Center at Houston, Department of Otorhinolaryngology;
Abstract:
Background: Publicly available artificial intelligence (AI) software has revolutionized health information delivery. Ensuring that AI-based tools, like ChatGPT, generate patient-friendly content is critical for effective patient education.
Learning Objectives: Identify the potential of ChatGPT in generating educational content related to facial feminization surgery (FFS) in the clinical setting.
Study Objective: Compare the readability, understandability, and actionability of ChatGPT-generated materials on FFS with those currently available on Google.
Design Type: Observational
Methods: Search terms related to "facial feminization surgery" were identified. Top-ranking websites were evaluated for readability using the Flesch Reading Ease (FRE) score and Flesch-Kincaid Grade Level (FKGL). The Patient Education Materials Assessment Tool (PEMAT) was used to evaluate understandability and actionability. Frequently asked questions were entered into ChatGPT, generating two types of responses: a general response and one adapted for an eighth-grade reading level. Both sets of responses were assessed using identical criteria.
Results: Only 2% of Google-sourced websites met recommended readability guidelines. ChatGPT met readability standards 0% of the time for general responses, but achieved the appropriate standards in 28% of cases when tailored to an eighth-grade reading level. Additionally, ChatGPT's responses scored higher on PEMAT for understandability and actionability compared to existing online resources.
Conclusion: When prompted for the appropriate audience, ChatGPT produced patient education materials on FFS with better readability, understandability, and actionability than those available through Google. As interest in FFS increases, ChatGPT may serve as a valuable resource for patient education, provided the content is tailored to the target audience and its accuracy verified.