In July we started a series of editorials that address the idiosyncrasies of observational studies for generating knowledge (universals) for use in practice (interpreting particulars).1 That editorial addressed the conundrum of particulars and universals, and put forth the claim that while we are trying to understand what we observe, we also tend to observe what we understand.
There is a subtly to the message that we tend to observe what we understand. When we narrow our observations to what we understand we run a risk of bias. Of course, we are all biased and the irony of making that statement and the sand trap it lands me in is not lost. That is what makes research unique, it is a process that structures observation to reduce bias in the process of generating understanding. It is important that we take this seriously. We must explicitly check our thinking, examine our steps and consider our observations. Recall the wisdom of Einstein, “the world we have created today as a result of our thinking thus far has problems which cannot be solved by thinking the way we thought when we created them.”2 Research and reflective practice go hand in hand, they call the profession to consider how our prior thinking has impacted our observations and created our current problems. And, importantly, how should we be thinking if we are to solve these problems?
At CPTJ we require submission of the STROBE checklist, listed at the EQUATOR Network,3 to be sure papers reporting the results of observational designs have considered how the approach taken to making observations may lead to bias. STROBE stands for Strengthening The Reporting of OBservational studies in Epidemiology. Just the name should highlight a feature of observational studies. These studies fall under the purview of the methodological developments in epidemiology. If you are considering an observational study it is highly recommended that you consult with an epidemiologist on your design and analysis plans. This is not always possible. The next best thing would be to consult with someone that has some epidemiological training (a course or perhaps courses as part of a degree in public health). If that remains a challenge, then reading about epidemiological methods would be a good place to start. If you have plans to pursue ongoing observational studies in your professional goals, consider coursework or a degree in epidemiology or public health.
The reason this is important is that all aspects of an observational study are possible sources of bias. In the STROBE checklist the word bias shows up in the section on limitations (Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias); the root of bias and its potential sources, direction and magnitude is related to all other sections of the STROBE checklist, and hence, your future (or completed) observational research study. And I must tell you, “NA” is not an appropriate answer to the bias item on the STROBE checklist. In my time as Editor I have had to send back more than one paper because we will not review a paper about an any study that attempts to claim “NA” for limitations and discussion about potential sources of bias.
We will spend the next several editorials on various sections and parts of the STROBE checklist as we continue the series on the idiosyncrasies of observational studies. This Month we will discuss the very basic consideration of where you are standing. Where you are standing refers to the approach being taken to make observations. It is most obvious in 2 inter -related areas of the study—the design and subject recruitment. Design of observational studies tends to fall into either cohort, case-control or cross-sectional. Both cohort and case-control can be further broken down into prospective and retrospective. Cohort simply refers to the method observing a group of subjects (a cohort). When followed from a beginning time and into the yet determined future it is a prospective cohort study. When following from the end and looking back in time it is a retrospective cohort. A case-control study intentionally matches cases with control subjects and the contrast between the groups is hoped to be only related to what defines the case group. If the groups are defined and followed forward in time it is a prospective case-control (not common). If the groups are defined and then observation goes back in time it is a retrospective case-control (more common). Cross-sectional simply looks at one point in time without any attempt to reconstruct or follow events temporally. This is a very broad look at these designs. We will go deeper into the idiosyncrasies and potential biases of each design in future editorials.
My focus with this editorial is to consider all designs from the perspective of where and how you are recruiting and selecting subjects. Selection bias and recruitment bias are possible in all forms of research, particularly observational studies. Selection bias is introduced when the selection of individuals, groups or data results in a sample that is not representative of the intended population. Recruitment bias is when the subjects that choose to perform the study (are in the study) are somehow different than those that do not choose to the perform the study resulting in a sample that is not representative of the intended population. The only situation that can claim to have no selection or recruitment bias is with complete randomization of the intended population being offered the opportunity to participate in the study, and then 100% participation of subjects that are offered the opportunity participate in the study. This is a highly idealized (near impossible) situation. From that ideal, the researchers must consider the possible impact of these biases and minimize them through analysis and consider their magnitude and direction in their interpretation and presentation of findings.
Simply put, consider where you are standing and who you are observing. For example, if we sit at the top of Mount Katahdin (Maine) and keep track of all the hikers reaching the summit, asking them where they have come from and noting how many started in Georgia and completed the entire 2180 miles of the Appalachian Trail (AT) we will get the impression that hiking the AT from Georgia is not a big deal (retrospective cohort study on completing the AT). Let's say in a season we note 1000 people complete the full south to north AT thru hike. We will have the impression that everyone that starts finishes because all we are observing are the finishers. Our conclusion is clearly biased via selection bias. We have only selected to observe the finishers. Being bounded by our place we have not observed all the people that started in Georgia and stopped along the way.
Consider a less silly and obvious example. If you are a PT working in an acute care hospital and only see people that survive otherwise life threatening illnesses and are in the intensive care unit and you do not consider the inherent biases associated with such observations you run the risk of drawing incorrect (biased) conclusions. I will never forget the PT that worked in a cardiac care unit that confronted me after a presentation on electrocardiograph data that was convinced that complete AV heart block was not life threatening because they see people that survived complete AV heart block all the time. What they do not see are those that did not survive…….and for many that do not survive we do not even know that complete AV heart block was the cause of “sudden cardiac death” as a rather vague mechanistic explanation.
Here are some considerations for readers and authors planning future observational studies. What did the potential subjects have to go through to be in a location that you can observe them? How do those experiences influence, select, modify the potential sample? Is your potential sample based on potential subjects choosing to do something that makes them different from others in the intended population? For example, all cardiac and pulmonary rehabilitation facilities doing an observational study start with a sample that is different from all patients with cardiopulmonary disease. The subjects are ambulatory enough to leave the house and get to the facility, they are willing to participate, they have a physician provider that referred them to rehabilitation which may be associated with other practice pattern behaviors of the physician provider (we will break down such inter-causal biasing characteristics in future editorials).
In This Issue
In this issue of CPTJ we are pleased to bring the annual CSM Linda Crane Lecture, 3 original research reports. Gurovich et al4 report on their original research to determine whether clinical markers of exercise intensity, such as heart rate and rate of perceived exertion, reflect physiological demands, measured via blood lactate levels, during a graded exercise test in cardiac patients. Their findings are very interesting and raise interesting questions about clinical practice. To help us consider these questions the issue also includes a commentary by CPTJ Associate Editor, Dr. Michael Shoemaker.5 Larsen et al6 present a systematic review and meta-analysis that compares physical fitness and cardiopulmonary exercise test performance using arm versus leg cycling in patients with cardiovascular or pulmonary disease. Arm cycling seems to offer an important alternative form of exercise testing for patients however, with this mode of testing clinicians must consider that VO2 max values obtained from arm cycling are lower than those achieved with leg cycling. A great clinical feature of this meta-analysis is the presentation of an evidence based estimation for the proportional differences between arm and leg VO2 max specifically for people with cardiac or pulmonary conditions. Finally, Brown et al7 report on how body position affects ultrasonographic measurement of diaphragm contractility in healthy adults as an important step toward possible widespread clinical use of this imaging modality by physical therapists in patients with suspect impairments in diaphragm contractility.
1. Collins SM. Particulars and universals: Understanding what we observe; observing what we understand. Cardiopulm Phys Ther J. 2018;29(3):97–98.
2. Bhaskar R. Enlightened Common Sense: The Philosophy of Critical Realism. Routledge; 2016.
4. Gurovich AN, Heiser B, Hayes C, Marshall E, Roath S, Kabous N. Clinical markers of exercise intensity as a surrogate for blood lactate levels only during low-intensity exercise in patients with coronary artery disease. Cardiopulm Phys Ther J. 2018;29(4):144–151.
5. Shoemaker MJ. Commentary on “clinical markers of exercise intensity as a surrogate for blood lactate levels only during low-intensity exercise in patients with coronary artery disease”. Cardiopulm Phys Ther J. 2018;29(4):152–153.
6. Larsen RT, Tang LH, Keller C, et al. Comparison of physical fitness and cardiopulmonary exercise test performance using arm versus leg cycling in patients with cardiovascular or pulmonary disease—a systematic review and meta-analysis. Cardiopulm Phys Ther J. 2018;29(4):154–165.
7. Brown C, Tseng SC, Mitchell K, Roddey T. Body position affects ultrasonographic measurement of diaphragm contractility. Cardiopulm Phys Ther J. 2018;29(4):166–172.