Remembering Paul E. Meehl: Historical Contributions to Predictive Modeling in Human Behavior

Abstract

This article briefly examines the life and work of the late clinical psychologist and philosopher of science Paul E. Meehl. His thesis in Clinical versus Statistical Prediction (1954) that the data combination performed by mechanical operations, as compared to clinicians, achieves higher accuracy in predicting human behavior is one of the earliest theoretical works that laid the groundwork for utilizing statistics and computational modeling in research in psychiatry and clinical psychology. For today’s psychiatric researchers and clinicians grappling with the challenges of translating the ever-increasing data of the human mind into practice tools, Meehl’s advocacy for both accurate modeling of the data and their clinically relevant use is timely.

From making a diagnosis, to creating a treatment plan, to deciding on a disposition for the patient who reports suicidal ideation in the hospital, psychiatrists routinely make predictions for an individual patient using clinical judgment about the presenting case and the data that support it. Psychiatry is approaching an era of an unprecedented knowledge base for conceptualizing and treating mental illness, due in part to rapid development in neuro- and cognitive sciences, and technological advances that can translate discoveries to real-world evidence to guide clinical decision support. However, clinicians have long known that beyond the evidence pooled from populations, there is unique information about the individual patient that can influence a clinician’s prediction at a given moment. A complex model has always been necessary to understand the mechanics of the human mind.

Paul E. Meehl was an American clinical psychologist, philosopher of science, and one of the earliest thinkers to contribute to the theoretical grounds for predicting clinical outcomes using quantitative methods.¹ Specifically, his 1954 book Clinical versus Statistical Prediction famously examines the limitations and unique leverage of clinician judgment for predicting human behaviors. This column will trace the origins of his seminal argument that the application of a data-driven approach should function together with clinician insight in selecting models of behavioral prediction. His legacy is particularly timely for today’s psychiatric researchers and clinicians, who are increasingly faced with the challenging task of translating rapidly expanding data about the human brain and behavior into sound clinical practice guidelines.

Meehl was born on 3 January 1920 in Minneapolis, Minnesota in a liberal Methodist home. When he was 16, his mother died after her internist misdiagnosed a spinal tumor as Meniere’s disease. In his autobiography, Meehl wrote that “This episode of gross medical bungling permanently immunized me from the childlike faith in physicians’ omniscience . . . and helped me to avoid dogmatism about my own diagnostic inferences.”² In 1945, he began his doctoral studies in psychology at the University of Minnesota under Starke R. Hathaway, who was influential in the development of the Minnesota Multiphasic Personality Inventory (MMPI), a standardized psychometric test of adult personality and psychopathology. Meehl’s subsequent postdoctoral research investigated the suppressor variable called K factor, a quantitative proxy of one’s test-taking attitude and its statistical correlations with psychological traits of the subject taking the MMPI,³ such as defensiveness. This marked his early interest in using an objective index to optimize the reliability of behavioral prediction enabled by self-reports.

CLINICAL VERSUS STATISTICAL PREDICTIONS

Meehl’s legacy in delineating the boundary between hypothetical constructs made by humans and structured, quantitative data generated by statistical methods was best exemplified in Clinical versus Statistical Predictions. In the controversial manuscript, he argued that “actuarial methods,” i.e., mathematical combination of data, outperform “clinical methods” when predicting behavior. He opened his thesis by disagreeing with one of his prominent contemporaries, T. R. Sarbin, who wrote that the clinician always, albeit implicitly and inefficiently, operates as an empiricist by gathering impressions of past behavior that directly allows prediction of future behavior.⁴ To this, Meehl stated that one should “distinguish carefully between how you get there and how you check the trustworthiness of your judgment.”⁵ Writing that the problem of bias arises when clinicians “assign a confidence, weight, or probability to the sentence which speaks of [the person’s behavior],” Meehl argued that the empirical lens through which clinicians interpret human behavior is inherently limited and does not achieve the precision essential to the quality of prediction.

For example, he refuted a conclusion in the 1941 study that the psychometric scale is inferior to qualitative case histories as a predictor of life choices made by Harvard College graduates.⁶ The scale’s prediction rates of only 36.9%, which was the second lowest number among the compared modalities of prediction, and hence the evidence for its inferiority, were not the issue. The problem was that the methodology did not involve any empirical comparison between actuarial and nonactuarial methods, but instead between different kinds of data from “the [same] judges [who] combined the information in whatever manner seemed subjectively most appropriate, in the absence of any exact knowledge concerning the statistical relationships between [the judgment] and the to-be-predicted behavior.” “The ideal design,” he wrote, “is one in which the same basic set of facts is subjected on the one hand to the skilled analysis of a trained clinician, and on the other hand to mechanical operations for comparison” where the bias of subjectivity in combining data is minimized.⁵ For example, entering the frequencies of a qualitative observation across cells in a table, followed by mathematically manipulating those numbers to arrive at the prediction of the ground truth will be equivalent to the actuarial method. By having the life choices predicted always by judges in the original paper, the result was that “actually, all the predictions were made clinically.” Reviewing twenty studies that used the appropriate comparative analysis, he concluded that the actuarial method had indeed higher predictability. A 2000 meta-analysis that compared methods used in 136 studies concerning the prediction of human health and behavior replicated this conclusion.⁷

MEEHL’S VIEW ON THE CLINICIAN’S “SPECIAL POWER”

While the most essential contribution of his thesis was “to draw a sharp distinction between data gathering (e.g., interview, psychometrics, behavioral observation) from data combination,” Meehl’s friend and frequent collaborator William Grove wrote that Clinical versus Statistical Predictions actually originated in the late psychologist’s efforts to examine “the nature and proper role of clinical inference.”⁸ In fact, Meehl admits that he spent more pages “defending the unique inferential activity” of a clinician than “criticizing his [or her] predictive deficiencies.”² According to him, “the behavior which is important to clinicians always involves, at least indirectly, interaction with other human organisms” and “the problem of specifying [behavioral] response classes [is] a fantastically complicated one.” “What the clinician does,” he wrote, “is to utilize the given facts … to invent a hypothesis concerning the state of certain intervening variables or hypothetical constructs in his patient.”⁵ This also suggested that, despite its predictability, data combination using actuarial methods may not always be relevant to the problem the clinician wants to predict.

Meehl was aware of the implication of his thesis for the clinical community. To the critics who said that his work pitted the value of clinical evaluation against mechanical methods, he argued that the concocted rivalry between the two is “a ridiculous position when the context is pragmatic decision making.”⁹ It is important to note that Meehl was a psychodynamically trained clinician, who continuously practiced psychotherapy until several years after his retirement from the University of Minnesota. He was less interested in choosing clinical versus statistical prediction than in re-examining the widely influential conjectures made by his contemporaries in the psychoanalytic field using rigorous methods. He recalls “it was easy for me to be relatively fair-minded about this charged topic, as I had strong identifications on both sides.”²

His comfort in navigating different disciplines distinguishes his professional highlights in general. In “What, Then, Is Man?”(1958), Meehl collaborated with a group of Lutheran theologians and psychologists to explore how Christians could responsibly practice faith and science without betraying orthodoxy in either side.¹⁰ In 1962, he was elected president of American Psychological Association at age 42 and delivered “a radical doctrine” that phenotypic heterogeneity of schizophrenia demands “an adequate theoretical account [of] explanatory constructs at the neurophysiological level,”^2,11 long before the complex, polygenic model of schizophrenia was discovered through advanced research methods in molecular genetics and genomics.¹² After a long career training the next generation of mental health clinicians in diagnosis and psychotherapy, he died of chronic myelomonocytic leukemia on 14 February 2003 at his home in Minneapolis.

MEEHL’S LEGACY FOR SCIENTISTS AND CLINICIANS STUDYING HUMAN BEHAVIOR

Almost seven decades after the publication of Clinical versus Statistical Predictions, Meehl’s contribution to the practice of psychological sciences transcends academic boundaries. In the field of psychometrics, MMPI, where he developed the K factor and now in its third iteration after undergoing standardizations on more racially and educationally representative adult samples, is a hallmark of psychological testing with validity scales that provide information about a subject's response bias such as overreporting and social desirability. The settings of its administration have since been extended to forensic assessment¹³ and pre-surgical screening evaluations¹⁴ to assist legal and medical professionals to understand the psychological profiles of a client or patient as objectively as possible in making high-stakes decisions. On the data science side, especially in the field of machine learning and artificial intelligence, the predictive principles and structure of neural and behavioral data are surfacing at faster rates with few human formal assumptions.¹⁵ This computational approach, which Meehl would likely classify under actuarial methods, now provides experimental capacity for researchers to analyze natural language, vocal acoustics, facial expressions, neural signals, behavioral task, and smartphone data for diagnostic purposes.^16-21 The launching of the Research Domain Criteria (RDoC) program in 2009 by the National Institute of Mental Health to generate a transdiagnostic framework for classifying mental illness represents the field’s turn toward quantitatively interpreting the large amount of newly available data.²²

In parallel, scientists argue that data require plausible models if they are to be understood and explained. Increasingly nuanced discussions address how data collected in laboratory and real-world environments must model time and context to explain complex behaviors in mental health populations,^23,24 and that generalizability and interpretability in machine learning algorithms are priorities in developing sound clinical prediction models.^25,26 Humans have always been the active agents who find competing models for explaining data and rely on quantitative information to select the best among them.²⁷ More than ever, it is the responsibility of researchers to actively examine bias in datasets and algorithms if their research is to be clinically useful and safe. For clinicians, knowing when to apply data-based insights in an individual context, checking their own cognitive biases and racial assumptions during clinical work,^28,29 and using intuition to frame important research questions from the bedside continue to be uniquely a human work. This strict evaluation of new research process and delivery is “a clinician’s special power,” the crucial step in solving a prediction problem to benefit patients.

Currently, RDoC is a framework for organizing psychiatric research, not a diagnostic system. There is also a promising, but long way to go toward the development of precision psychiatry as a paradigm of every day clinical practice. However, already in the growing field of computational psychiatry, a move toward collecting human behavior data in naturalistic settings, such as gamified smartphone apps, reflects the scientific community’s interest in generating quantitatively robust, yet clinically usable data that fit with human behaviors in the wild.³⁰ From modelling to deployment stages of artificial intelligence algorithms, implementation scientists are increasingly paying attention to approaches for debiasing training data to account for racial inequities in mental health, and to the clinician-machine interface to minimize potential conflicts between human experts and mathematical models.³¹ In the changing landscape of contemporary psychiatric research and practice, Meehl’s balanced advocacy for both matter-of-fact and intuitive approaches to quantify human behavior reminds us of the limitations and opportunities of being human.