Secondary Logo

Journal Logo

AAPA Members can view Full text articles for FREE. Not a Member? Join today!

Interpreting the methodology

JENNINGS, PATRICIA R. DrPH, PA-C

Journal of the American Academy of PAs: April 2007 - Volume 20 - Issue 4 - p 1–2
Research Corner Online
Free
SDC

Patricia Jennings is Associate Professor, Division of Physician Assistant Studies, University of Alabama, Birmingham, and a member of the JAAPA Editorial Advisory Board.

Our profession demands that we stay current with medical practices by reading original research articles. These manuscripts have become increasingly sophisticated, and the complexity of the information presented, especially in the methodology section, almost encourages us to “skip” directly to the conclusion. This article offers help by explaining the methodology described in the abstracts featured in this month's Research Corner.

LET US BEGIN with the two abstracts by Morgan, “A national comparison of characteristics of patients and patient visits attended by PAs or physicians in office-based care”1 and “Do PAs still care for the underserved?”2 Both utilized data from the Medical Expenditure Panel Survey (MEPS). The MEPS began in 1996 as a large-scale survey of individuals and their medical providers.

“MEPS collects data on the specific health services that Americans use, how frequently they use them, the cost of these services and how they are paid for, as well as the cost, scope and breadth of health insurance held by and available to United States (U.S.) workers. The Household Component (HC) collects data from a sample of families and individuals in selected communities across the United States, drawn from a nationally representative subsample of households that participated in the prior year's National Health Interview Survey (conducted by the National Center for Health Statistics).”3

The panel design of the survey, which features several rounds of interviews covering 2 calendar years, makes it possible in this case to determine how changes in respondents' health status, income, employment, eligibility for public and private insurance coverage, use of services, and payment for care are related.

In the first abstract, Morgan states, “This project uses data from the MEPS to compute weighted national estimates of patient characteristics and examine trends in the case mix of patients seen by PAs and MDs between 1996 and 2003.” How are “weighted estimates” calculated, and why is the calculation important?

In some comparisons of two sets of percentages, each observation has an equal impact on the result. In this case, however, as some practices (or populations noted in this abstract) contributed fewer participants, we would like these participants to have a lesser effect on the estimate of the difference (so we weigh them differently in the calculation). Ordinarily, to calculate the mean percentage in each group, we simply add the observations together and divide by the number of observations. To calculate the weighted mean, we multiply each observation by the weight (previously ascribed), add, and then divide by the sum of the weights. By calculating a weighted average (in contrast to a simple or unweighted average) of results in this instance, we can put more emphasis on the most statistically precise trial findings.

IN THE ABSTRACT by Dehn, “The distribution of PAs, NPs, and MDs in Iowa,”4 the county datasets were stratified by population from most to least populous and then grouped into quartiles of equal populations. We recall that stratification is the process of grouping members of the population into relatively homogeneous subgroups before sampling (or in the analysis). In Dehn's project, the population density varied within a region; stratifying the sample ensures that estimates can be made with equal accuracy in different parts of the region and that comparisons of subregions can be made with equal statistical power.

The advantages of stratification are that it

  • Focuses on important subpopulations but ignores irrelevant ones
  • Improves the accuracy of estimation
  • Is efficient
  • Provides equal numbers from strata varying widely in size (this may be used to equate the statistical power of test differences between strata).

The disadvantages of stratification are that it

  • Can be difficult to select relevant stratification variables
  • Is not useful when there are no homogeneous subgroups
  • Can be expensive
  • Requires accurate information about the population or bias is introduced
  • Looks randomly within specific subheadings.

LASTLY, in the abstract by Orcutt, the methodology section mentions that “supply and demand equations will be analyzed using ordinary least squares estimation techniques for trend analyses.”5 Least squares is a mathematical optimization technique that, when given a series of measured data, assesses how well the data conform to a specified relationship (ie, function).6

The least squares technique is commonly used in curve fitting. How do you fit a regression line when data appear in the form represented below?

Figure

Figure

We could “eyeball” the data and draw a straight line that is not too distant from any of the points. However, this approach is difficult and can be quite imprecise with either a large number of points or a lot of scatter. A better method is to set up a specific criterion that defines the closeness of a line to a set of points and to find the line closest to the sample data according to this criterion. The least-square line is the line that minimizes the sum of squared distances of the sample points. In the methodology section, this may be represented by the term R-square or adjusted R-square. The closer this number is to 1, the better the line “fits” (often referred to as the “goodness of fit”).

IN SUMMARY, Understanding the principles that guide the selection and application of statistical tests aids critical reading. When researchers use appropriate tests to analyze their data, their results gain credibility.7 The following points are key:

  • Many medical researchers now consider the services of a biostatistician essential in helping with the design and analysis of a study.
  • No statistical test can transform poor data into useful information.
  • Finally, statistical significance does not equal clinical importance.
Back to Top | Article Outline

REFERENCES

1. Morgan P, Strand J, Albanese M. A national comparison of characteristics of patients and patient visits attended by physician assistants or physicians in office-based care (preliminary report). Presented at: Physician Assistant Education Association Education Forum; October 28, 2006; Quebec City, Quebec.
2. Morgan P. Do physician assistants still care for the undeserved? (Preliminary report). Presented at: Physician Assistant Education Association Education Forum; October 28, 2006; Quebec City, Quebec.
3. United States Department of Health and Human Services; Agency for Healthcare Research and Quality. MEPS: Medical Expenditure Panel Survey. Available at: http://www.meps.ahrq.gov/ mepsweb/about_meps/survey_back.jsp. Accessed March 7, 2007.
4. Dehn RW. The distribution of physicians, advanced practice nurses, and physician assistants in Iowa. Journal of Physician Assistant Education. 2006;17(1):36-38.
5. Orcutt VL. A supply and demand model for the physician assistant profession. Presented at: Physician Assistant Education Association Education Forum; October 28, 2006; Quebec City, Quebec.
6. Least squares. Wikipedia: the free encyclopedia. Available at: http://en.wikipedia.org/wiki/ Least_squares. Accessed March 7, 2007.
7. Gehlbach SH, ed. Interpreting the Medical Literature. 5th ed. New York, NY: McGraw-Hill; 2006:8-12.
© 2007 American Academy of Physician Assistants.