Original ResearchQuantitative Patient-Reported Experience Measures Derived From Natural Language Processing Have a Normal Distribution and No Ceiling EffectRajagopalan, Dayal BS; Thomas, Jacob MA; Ring, David MD, PhD; Fatehi, Amirreza MD Author Information Department of Surgery and Perioperative Care, Dell Medical School, The University of Texas at Austin. Correspondence: David Ring, MD, PhD, Department of Surgery and Perioperative Care, Dell Medical School, The University of Texas at Austin, 1501 Red River St, Austin, TX 78712 ([email protected]). Royalties: Wright Medical, Skeletal Dynamics, Up-to-Date. Honoraria: Multiple universities and CME providers. Editor: Deputy Editor for Hand and Wrist, Journal of Orthopaedic Trauma; Deputy Editor for Hand and Wrist, Clinical Orthopedics and Related Research. Consulting: Department of Justice—Vaccine Injury Compensation Program, Pfizer, lawyers for expert testimony. The authors declare no conflicts of interest. Quality Management in Health Care: October/December 2022 - Volume 31 - Issue 4 - p 210-218 doi: 10.1097/QMH.0000000000000355 Buy Metrics Abstract Background and Objectives: Patient-reported experience measures have the potential to guide improvement in health care delivery. Many patient-reported experience measures are limited by the presence of strong ceiling effects that limit their analytical utility. Methods: We used natural language processing to develop 2 new methods of evaluating patient experience using text comments and associated ordinal and categorical ratings of willingness to recommend from 1390 patients receiving specialty or nonspecialty care at our offices. One method used multivariable analysis based on linguistic factors to derive a formula to estimate the ordinal likelihood to recommend. The other method used the meaning extraction method of thematic analysis to identify words associated with categorical ratings of likelihood to recommend with which we created an equation to compute an experience score. We measured normality of the 2 score distributions and ceiling effects. Results: Spearman rank-order correlation analysis identified 36 emotional and linguistic constructs associated with ordinal rating of likelihood to recommend, 9 of which were independently associated in multivariable analysis. The calculation derived from this model corresponded with the original ordinal rating with an accuracy within 0.06 units on a 0 to 10 scale. This score and the score developed from thematic analysis both had a relatively normal distribution and limited or no ceiling effect. Conclusions: Quantitative ratings of patient experience developed using natural language processing of text comments can have relatively normal distributions and no ceiling effect. © 2022 Wolters Kluwer Health, Inc. All rights reserved.