JMIR Publications

ABSTRACT

Background:

AI-driven generative language models (GLM) have enormous potential in medicine. A quickly implementable and impactful application of GLMs is to address low health literacy (LHL).

Objective:

The goal of this study is to evaluate GLM’s potential to tailor the complexity of medical information to patient-specific input education level, which is crucial if it is to serve as a tool in addressing LHL.

Methods:

Input templates related to three prevalent chronic diseases–type II diabetes, chronic obstructive pulmonary disease (COPD), and hypertension were designed. Each clinical vignette was adjusted for hypothetical patient education levels to evaluate output personalization. To assess the success of a GLM in tailoring output writing, the readability of pre- and post-transformation outputs were quantified using the Flesch Reading Ease Score (FKRE) and the Flesch-Kincaid Grade Level (FKGL).

Results:

Responses (n=90) were generated using the GLM program across three clinical vignettes. Mean and standard deviations (SD) were FKRE=62.6 (SD=3.26), 54.6 (3.73), 46.2 (7.19), 42.1 (5.02), 38.0 (7.93), 23.9 (3.53) when aggregated for education level (5th grade, 8th grade, high school, associate’s, bachelor’s, and doctorate, respectively). For the three lowest education levels, the output FKRE means did not show concordance with pre-specified input education level. However, output FKRE means did show concordance for the three highest education levels. The respective FKGL means=8.32 (SD=0.642), 9.70 (0.727), 11.2 (1.17), 11.6 (0.841), 12.4 (1.19), 14.6 (0.550). The GLM produced outputs with statistically significant differences between mean FKRE and FKGL across input education levels.

Conclusions:

GLMs can change the structure and readability of medical text outputs according to specified education levels but are more effective for higher than lower education level inputs. GLMs are also able to categorize input education designation into three broad tiers of output readability: Easy (5th and 8th grade), Medium (high school, associate’s, and bachelor’s degree), and Difficult (doctorate). Future research must establish how GLMs can reliably personalize medical texts for patients with lower education levels to improve healthcare literacy.