Comment on the Point of View “Ecological Validity, External Validity and Mundane Realism in Hearing Science”

To the Editor:

Ear and Hearing recently published a point of view paper on “Ecological validity, external validity and mundane realism in hearing science” (^{Beechey 2022}) that references the 2020 Eriksholm workshop consensus paper on “The quest for ecological validity in hearing science: what it is, why it matters, and how to advance it” (^{Keidser et al. 2020}). The consensus paper proposes an operational definition of ecological validity for use in hearing science and is the outcome of a three-day workshop on how to advance ecologically valid assessments of hearing and hearing devices. We thank the author of the point of view paper for his contribution to the discussion about the meaning of the term “ecological validity”. As his article suggests, there is an early original as well as a later and more recently popular usage of this term. The early definition proposed by ^{Brunswik (1952,1955}) refers to the correlation between cues received at the peripheral nervous system and the identity of distant objects or events in the environment, whereas the more popular definition, which has evolved from Brunswik’s original observations, construes ecological validity as a concept that examines to what extent the results of a research study are related to outcomes in situations occurring in everyday life [e.g., ^{Schmuckler (2001}); ^{Neuhoff (2004}); American Psychological Association, n.d.]. While the author makes an excellent case in favor of the first definition of ecological validity, the 2020 Eriksholm Workshop consensus paper builds on the more popular definition of the term, which is broadly considered a type of external validity.

As a first point, we would like to take this opportunity to clarify that the working group behind the consensus paper was aware of Brunswik’s proposed definition and probably should have acknowledged its historical usage in the background review of the consensus paper. Our choice of following the more recently popular understanding of the term and proposing a definition that exists as a sub-category of external validity was twofold: (1) it captures a direction in which many hearing researchers have been heading for more than a decade, albeit without a comprehensive theoretical motivation and justification, and (2) it lends itself to an operational framework (like the one presented in the consensus paper) that we believe will assist hearing research in making long-awaited advancements in the field. While the two definitions of ecological validity focus on different aspects of experimental conditions, the concepts they stand for are not mutually exclusive, and although our adaptation of ecological validity is more consistent with current usage of the term, as long as authors clearly state in their articles the intended operationalized definition of ecological validity, either definition could be used.

We should further like to take this opportunity to stress that the definition of ecological validity proposed by the working group refers to a type of external validity that examines how representative research outcomes are of the everyday listening situation(s) under investigation. In hearing science, such an investigation could address real-life hearing-related function, activity, or participation. This type of validity specifically targets defined situations occurring in everyday life, for example, a conversation between three people seated in a busy cabin on a moving train, while external validity more broadly examines whether research outcomes can be generalized to other contexts (^{Juni et al. 2001}), for example, whether speech performance in static noise obtained with a group of university students translates to a broader population and to other background noises. In other words, our definition of ecological validity should not, as suggested in the point of view paper, be construed as implying that realism equates to any theoretical form of generalizability. The “external” validity we were concerned with is purely in the translation from an experimental condition to the (possibly singular) real-life condition it is intended to represent.

Meanwhile, although the term “psychological realism” (as also discussed in the point of view paper) was not used in the consensus paper, it should be apparent from the consensus paper’s emphasis on variables beyond the perceptual, that this is the form of realism, which we sought to promote. While there is some emerging evidence that hearing research needs to implement more (psychologically) realistic listening environments, stimuli, and tasks [e.g., Carlile and ^{Keidser (2020}); ^{Grimm et al. (2020}); ^{Hohmann et al. (2020}); ^{Orenois & Buchholz (2016})], we agree with the author of the point of view paper that realism itself is merely a potential means to an end, and that “ecological validity” is not to be equated with mundane realism.

Finally, we welcome the author’s outline of the value of “Representative Design” in hearing research, and of how Ecological Momentary Assessment (EMA) is a step toward this approach. One of the biggest challenges for hearing researchers wishing to meaningfully emulate aspects of real-life listening in the laboratory is to gather knowledge of the physical and psychological variety, which has to be encompassed in a study design to claim a high level of ecological validity of the measured outcomes. Intuitively, such information must be informed by real-life behaviors and experiences that EMA studies may capture in an organized way. Although to achieve fully representative information from EMA studies there remain numerous challenges to overcome, such as preserving privacy; ensuring manageability of equipment; and reducing asynchrony between objective and subjective measures during data collection, as discussed in ^{Holube et al. (2020}).

In summary, this discussion highlights the importance of thoughtful consideration to the research protocol and clear statements by researchers of the operational definitions of specialized terms such as ecological validity.

ACKNOWLEDGMENT

The views expressed in this letter are those of the authors and do not reflect the official policy of the Department of Defense, or U.S. Government (D.B.).

Authors were the participants of the 6th Eriksholm Workshop on Ecologically Valid Assessments of Hearing and Hearing Devices.

The first two authors composed this Letter. The remaining authors reviewed and provided critical feedback to preliminary versions of the Letter.

The work referred to in our Letter to Editor is the product of the 6th Eriksholm Workshop that was funded by the William Demant Foundation, including travel and accommodation for all authors.