Secondary Logo

Journal Logo

Invited Commentaries: Philosophy of Science

Critical Realism and Realist Inquiry in Medical Education

Ellaway, Rachel H. PhD; Kehoe, Amelia PhD; Illing, Jan PhD

Author Information
doi: 10.1097/ACM.0000000000003232
  • Free

Abstract

Given that medical education and training are delivered in multiple contexts and settings around the world, it is a recurring challenge that educational interventions developed in one context may not translate to other contexts.1 Understanding complex interventions, such as those in medical education, requires a philosophy of science that can explain how and why things work, or fail to work, in different contexts. Critical realism and its operationalization in the form of realist inquiry can provide this much-needed explanatory power.2 This Invited Commentary focuses on the main features of critical realism and their realization in realist inquiry (see Table 1 for definitions of key concepts).

Table 1
Table 1:
Key Concepts in Critical Realism and Realist Inquiry

Critical realism has its basis in the work of British philosopher Roy Bhaskar,3,4 who raised the paradox that our knowledge of the natural world is inescapably socially constructed. In doing so, he differentiated between the real (what exists and how it might behave), the actual (what actually happens), and the empirical (our knowledge and experiences of what happens). Bhaskar called this transcendental realism. Central to this worldview is the idea of mechanisms: the causal factors that shape reality. The companion concept of critical naturalism reflects that, while social realities are also driven by mechanisms, how we explore them requires different approaches to those used in exploring the natural (nonhuman) world. Critical realism is a stance based on combining transcendental realism and critical naturalism.

Philosophical Foundations of Critical Realism

Ontology: The nature of reality

From an ontological perspective, critical realism reflects a postpositivist perspective that the social world is real and independent of our knowledge of it and that it is driven by mechanisms. However, while these mechanisms exist, they may not always be active, they may not be directly observable, and some mechanisms may be obscured or inhibited by other mechanisms. Moreover, while a critical realist accepts a postpositivist perspective of ontology, they embrace the idea of multiple legitimate perspectives on reality.5

Epistemology: The nature of knowledge

Epistemologically, critical realism is focused on exploring and understanding the mechanisms that drive social reality and the way these mechanisms are mediated by the contexts within which they work. Critical realists “do not deny the reality of events and discourses; on the contrary, they insist upon them. But they hold that we will only be able to understand—and so change—the social world if we identify the structures at work that generate those events or discourses.”4(p55) In doing so, critical realists draw upon the repertoire of social science methodologies and methods and their associated epistemologies to explore the mechanisms that underpin social phenomena. By focusing on mechanisms, the knowledge that critical realism generates goes beyond description to seek out explanations of how phenomena work and, from that, how they might be manipulated. Critical realism therefore combines a realist ontology with a constructivist epistemology.5

Axiology: The study of values and how they influence the research process

Axiologically, critical realism emphasizes the mechanistic origins and impacts of values and esthetics: what are the mechanisms that drive values and the ways in which values work in social systems? A critical realist stance also allows us to consider the role of values as mechanisms or as contexts in social phenomena. For instance, we might ask, “How do values shape practices (such as admissions, assessments, or curricula) in medical education?” Or we might ask, “How do these practices behave differently in the context of different value systems?”

Methodology: How to conduct scientific research

The methodology most commonly used in the critical realist paradigm is realist inquiry, a theory-driven research approach that focuses on the triadic relationships between context, mechanisms, and outcomes.6 Realist inquiry is particularly suitable for explaining complex programs and interventions, such as those so often found in medical education. At its core, realist inquiry is concerned with “What works for whom, under what circumstances, how, and why?”7 This approach goes further than the gold standard (i.e., a randomized controlled trail), as it focuses not just on outcomes but seeks to explain how outcomes are (or are not) achieved in different contexts.

We know that interventions are often applied in multiple contexts. Rather than controlling for contextual variations, as many other paradigms would seek to do, realist inquiry seeks to explain how mechanisms drive outcomes in different contexts. Context refers to the setting for human actions and includes both fixed characteristics, such as geography and organization, and variable human characteristics, such as culture and leadership. These contextual factors are a focus of realist inquiry rather than a source of noise to be controlled or normalized. Realist inquiry looks for shared social structures across different contexts and the common mechanisms that drive them. In doing so, it diverges from a constructivist stance that is unwilling to generalize or extrapolate findings beyond what are typically small study samples and contexts.7 Mechanisms are therefore a key construct in the ability for realist inquiry to generalize beyond the specific.

Mechanisms tend to operate at a psychological level, are normally invisible, and are sensitive to context.8 In realist inquiry, mechanisms are separately identified to understand the relationships between the context and study outcome; mechanisms explain how and why individuals may respond to an intervention in a specific context. The subject of inquiry (typically an intervention of some kind) is introduced into the context and changes the context by enabling different choices to be made, but it is the mechanism that leads to the outcome rather than the intervention.9Outcomes are the changes or impacts of interest that follow from the intervention, both intended and unintended. How a context shapes how a mechanism is triggered to produce or contribute to a particular outcome is called a context–mechanism–outcome (CMO) configuration.6 It is by identifying the relationships between the study context(s) and the mechanism(s) that produces the outcome(s) that realist inquiry seeks to explain how and why interventions work or fail to work.

Program Theory

The aim of realist inquiry is to develop a program theory. A program theory is a theoretical explanation of how an intervention works or is expected to work in different contexts and with different individuals. Realist inquiry begins by identifying or developing a provisional program theory that sets out at a theoretical level what factors could help to explain how an intervention should lead to different outcomes. A program theory is typically made up of multiple elements that may draw from practical experience, the literature, or other theories. Taking the case of Lee (see Box 1), a program theory here may draw on theories of resident and preceptor behaviors in stressful situations, theories of medical errors and their prevention, and human factors theories. It may also draw on reports of similar situations from the literature, on the investigators’ personal experiences of similar situations, and on relevant institutional policies and procedures. Middle-range theories are then developed from the data and are combined within program theory as a way of empirically testing the program theory’s suggested connections and causal relationships. For instance, a middle-range theory might suggest why Lee made the error and why Lee and the attending responded to the incident in the ways they did. Middle-range theories in this regard are hypotheses to be tested using the data from a particular case or number of cases.10

Box 1

Sample Casea

Lee was a resident assigned to monitor a postop patient. The patient had a periodically low respiratory rate and lower-than-normal pulse and blood pressure. Narcan was ordered on an “as needed” basis to be given in doses of 0.2 mg intravenously. In checking the patient’s vitals, Lee decided it was time to administer an intravenous (IV) dose of Narcan.

Once Lee injected the vial of Narcan into the IV port, Lee noticed it was labeled “2 milligrams per 1 milliliter (ml)”—the entire vial should not have been injected. Feeling panicky, Lee reported the mistake to an attending and rushed back to the patient’s side to monitor the vital signs. Lee was surprised to find that the patient’s vitals had come up to normal rates, and the patient was actually much more alert. When Lee reported this change to the attending surgeon and anesthesiologist, they told Lee to continue to monitor the patient closely, remarking that it may have been just what the patient needed.

Lee felt hugely relieved, but was still overwhelmed and very upset. In most cases, giving 10 times a normal dose of any medication could have led to extremely serious consequences and even death. Still, Lee managed to remain outwardly composed, and took the time to complete an incident report. At the end of the day, when Lee finally sat down to rest, the incident played over and over again. Lee did not sleep.

aThis sample case is used throughout the Philosophy of Science Invited Commentaries to illustrate each research paradigm.

Data and Methods

Realist inquiry requires no single definitive methodological stance; it is both pragmatic (adapting and changing the model of inquiry to respond to emerging issues) and eclectic (drawing upon the best methods available to help address its questions and problems).5,11 It is common therefore for realist inquiry to engage a range of social science research techniques, both qualitative and quantitative, and data may be drawn from the literature or from empirical sources (or both) to test the program theory and its middle-range theories.

For example, literature reviews (realist syntheses) can be used to answer certain questions. However, from a realist synthesis stance, papers are not selected by design or graded for quality; instead, they are selected based on rigor and relevance. Rigor is loosely defined as “whether the method used to generate that particular piece of data is credible and trustworthy,”12 while relevance is the extent to which “it can contribute to theory building and/or testing.”12 This means that a study low on quality could still be included if it is high on relevance, for example, if it clearly illustrates how the theory could explain the outcome. Indeed, realist syntheses usually draw from a wider range of research design than an equivalent systematic review, including gray literature (such as policy documents). Although there are no set procedural rules, the RAMESES reporting standards provide a quality benchmark in reporting realist syntheses.12,13

Realist inquiry can also involve collecting empirical data to develop or test a program theory and its associated middle-range theories. For example, in Lee’s case, we might interview all of the participants to explore their perceptions of what happened, why it happened, and what might be done to stop or reduce the likelihood of these kinds of issues happening again. We might also undertake a human factors analysis of the environment and the contextual factors that contributed to the incident and the ways in which those involved responded leading to a theory to explain what happened.

Analysis

Analysis in realist inquiry runs in parallel with data collection; as new data are collected, they are used to test and refine the program theory. At a practical level, the analysis stage for all types of realist inquiry involves first looking for CMO configurations within the evidence to identify the existence of mechanisms and how they could work in different contexts. The next step is to look for reoccurring patterns across the range of CMO configurations identified from the data. These may share contexts, mechanisms, or outcomes, or various combinations of them. These patterns are referred to as demiregularities. There are no set rules about how this stage should be conducted; however, as with any interpretive and postpositivist research process in analyzing data, it should be cautious and thorough, open to considerations of alternative explanations and confounding factors, and details of the process used should be thoroughly documented. Finally, the original program theory is then amended to incorporate or better reflect the findings. This cycle (program theory→data→analysis→amend program theory) may run once or many times according to available time and resources and to the need to further detail or expand the program theory.

Practical Example

We can illustrate a critical realist approach using our example case of Lee (see Box 1). A critical realist reading of this case would focus on identifying the mechanisms at play that can explain the participants’ behaviors and the outcomes of their behaviors. A program theory (noting that a single case such as this reflects a broader program of residency training in clinical workplaces) may include theories of resident behavior in stressful situations, causes of and responses to medical errors, and human factor theories that refer to the physical environment and organizational culture. It may also draw on reports of similar situations from the literature, on the investigators’ personal experiences of similar situations, and on relevant institutional policies and procedures. Data could be collected in the form of interviews gathering participant and onlooker experiences of what happened and their thoughts as to why it happened. Additional data on contextual factors (e.g., the nature of the physical environment, attitudes and previous experiences of medical errors) and descriptions of similar cases in other settings could also be collected. This could be used to identify the underlying mechanisms (e.g., Lee’s inattention or fear of making mistakes, why the patient did not suffer from the inappropriate dosage), along with the contextual factors that shape how those mechanisms work (e.g., participants’ lack of training, Lee’s lack of familiarity with the equipment used or the policies that apply). The aim would be to develop a program theory that explains how these kinds of errors occur, what their consequences are, and how to avoid or mitigate situations like this in future.

Discussion

While we have focused on the operationalization of critical realism in the form of realist inquiry, critical realism can be applied in other methodological configurations. For instance, a critical realist phenomenological perspective may seek to identify and explain the mechanisms that shape individual perceptions of a particular phenomenon (such as feedback or professionalism), a critical realist ethnography might seek to explain how and why practices differ between contexts (such as admissions or mentorship), or a critical realist take on discourse analysis might focus on the mechanisms that shape different discursive positions (such as in advancing programmatic assessment or equity of access to the professions). Differences in approaches to inquiry guided by realism are well illustrated by Fletcher in her descriptions of abductive (where “empirical data are redescribed using theoretical concepts”) and retroductive (drawing out causal mechanisms and the necessary conditions for those mechanisms to work) stages in data analysis.14 Given their focus on explaining complex interventions, principles of critical realism and their application in realist inquiry align well with the needs and dynamics of medical education. The strength of critical realism is its potential to explain a complex intervention and how it works (or not) in differing contexts and individuals. In doing so, it has the potential to offer greater generalizability than many other paradigms.

Despite these strengths, there are also challenges in taking a critical realist perspective, not least of which is its broad philosophical basis and its focus on pragmatism and eclecticism. The absence of a definitive approach to realist inquiry has been addressed to some degree by Pawson and his collaborators in establishing procedures for realist synthesis and realist evaluation.6,11 While realist inquiry is not the only way to operationalize critical realist principles, its techniques are well developed and have proved useful and generative in medical education scholarship.15,16 A second challenge is that the operationalization of critical realism in the form of realist inquiry is still being explored and developed and its tenets tested and refined. While this lends a degree of interest and even excitement to working in this area, it also means that it is a somewhat contested conceptual space.11,17 At a practical level, it can also be challenging to identify the relevant contexts and mechanisms in and around a particular phenomenon; identifying CMO configurations for each case can be time consuming; and in the early stages of a study, it may not be clear which CMO configurations will prove to be the key drivers or inhibitors in a system.18 Mechanisms may be inferred rather than directly observable or measurable, and the important elements of a context might only become apparent once patterns start to emerge following analysis of large datasets. In addition, there may be complicated chains of causality with one mechanism acting on another and thereby creating the context for a third and so on.11,17 We cannot say, therefore, that something is intrinsically a context, a mechanism, or an outcome; they are defined as such in relation to other dynamic factors within a phenomenon. To that end, a critical realist stance requires a degree of fluidity and systems-level thinking that may be challenging to scientists trained to other paradigmatic traditions.

Conclusion

Critical realism, like other philosophies of science, sets out a particular worldview, in this case that the world is real and is driven by mechanisms that may function differently according to context. Realist science focuses on exploring these mechanisms and the way they work to develop explanatory theories of the phenomena under consideration. Realist inquiry has arisen out of critical realism, as a pragmatic scientific approach that can confront the shortcomings of mainstream science in addressing issues of complexity. By generating theoretical models based on empirical data, realist inquiry affords greater explanatory power than other scientific approaches.5 Although, compared with other approaches, realist inquiry is relatively new in medical education, the value of realist inquiry becomes particularly apparent when it is used to model how interventions work across multiple contexts and to explain what works and how it works, for whom, and in what contexts (see Box 2 for key readings).

Box 2

Key Readings in Critical Realism and Realist Inquiry

While the work of Bhaskar3,4 is central to critical realism, it is not the most immediately accessible. We would therefore recommend the following as texts to learn more about critical realism and realist inquiry:

  • Ray Pawson has written a series of seminal books on realist inquiry. His 2013 realist manifesto reprises his earlier work and draws in some of the key issues in this emerging paradigm: Pawson, R. The Science of Evaluation: A Realist Manifesto. London, UK: SAGE; 2013.
  • Critical realism can shape research in different ways: Fletcher AJ. Applying critical realism in qualitative research: Methodology meets method. Int J Soc Res Methodol. 2017; 20;181–194.
  • In the context of medical education, the 2013 introductory paper by Wong et al on realist methods provides a sound foundation: Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: What are they and what can they contribute? Med Educ. 2012;46:89–96.
  • Realist inquiry is still being developed and expanded upon: Emmel N, Greenhalgh J, Manzano A, Monaghan M, Dalkin S. Doing Realist Research. London, UK: SAGE; 2018.

References

1. Ellaway RH, Pusic M, Yavner S, Kalet AL. Context matters: Emergent variability in an effectiveness trial of online teaching modules. Med Educ. 2014;48:386–396.
2. Collier A. Critical Realism: An Introduction to Roy Bhaskar’s Philosophy. 1994.London, UK: Verso;
3. Bhaskar RA. A Realist Theory of Science. 1975.Leeds, UK: Leeds Books;
4. Bhaskar RA. Reclaiming Reality: A Critical Introduction to Contemporary Philosophy, 1989.London, UK: Verso;
5. Maxwell JA. A Realist Approach for Qualitative Research. 2012.Thousand Oaks, CA: SAGE;
6. Pawson R. Evidence-Based Policy: A Realist Perspective. 2006.London, UK: SAGE;
7. Pawson R, Tilley N. Realistic Evaluation. 1997.London, UK: SAGE;
8. Astbury B, Leeuw FL. Unpacking black boxes: Mechanisms and theory building in evaluation. Am J Eval. 2010;31:363–381.
9. Dalkin SM, Greenhalgh J, Jones D, Cunningham B, Lhussier M. What’s in a mechanism? Development of a key concept in realist evaluation. Implement Sci. 2015;10:49.
10. Merton R. Social Theory and Social Structure. 1968.New York, NY: Free Press;
11. Pawson R. The Science of Evaluation: A Realist Man ifesto. 2013.London, UK: SAGE;
12. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: Realist syntheses. BMC Med. 2013;11:21.
13. Wong G, Westhorp G, Manzano A, Greenhalgh J, Jagosh J, Greenhalgh T. RAMESES II reporting standards for realist evaluations. BMC Med. 2016;14:96.
14. Fletcher AJ. Applying critical realism in qualitative research: Methodology meets method. Int J Soc Res Methodol. 2017; 20;181–194.
15. Illing JC, Carter M, Thompson NJ, et al. Evidence synthesis on the occurrence, causes, consequences, prevention and management of bullying and harassing behaviours to inform decision making in the NHS. Final report. NIHR Service Delivery and Organisation programme. http://www.netscc.ac.uk/hsdr/files/project/SDO_ES_10-1012-01_V02.pdf. Published 2013. Accessed April 30, 2020.
16. Ellaway RH, O’Gorman L, Strasser R, et al. A critical hybrid realist-outcomes systematic review of relationships between medical education programmes and communities: BEME guide no. 35. Med Teach. 2016;38:229–245.
17. Emmel N, Greenhalgh J, Manzano A, Monaghan M, Dalkin S. Doing Realist Research. 2018.London, UK: SAGE;
18. Illing J, Corbett S, Kehoe A, et al. How Does the Education and Training of Health and Social Care Staff Transfer to Practice and Benefit Patients? A Realist Approach. 2018. University of Newcastle, UK: Final Report for Department of Health; https://eprint.ncl.ac.uk/250597. Accessed February 14, 2020.
Copyright © 2020 by the Association of American Medical Colleges