Medical Care

Skip Navigation LinksHome > July 2008 - Volume 46 - Issue 7 > Comparisons of Health Plan Quality: Does the Sampling Strate...
Text sizing:
Medical Care:
doi: 10.1097/MLR.0b013e3181789300
Brief Report

Comparisons of Health Plan Quality: Does the Sampling Strategy Affect the Results?

Goldman, L Elizabeth MD, MCR*; Chattopadhyay, Arpita PhD†‡; Bindman, Andrew B. MD†‡§

Collapse Box


Background: Health plan evaluations typically focus on beneficiaries continuously enrolled throughout a calendar year.

Objective: To examine whether the sampling strategy used in evaluating health plans affects their assessment.

Research Design: We completed a cross-sectional analysis of California Medicaid managed care plans aggregating individual level data by health plan.

Measures: We calculated the correlation between a health plan's annual age-sex adjusted hospitalization rate for ambulatory care sensitive (ACS) conditions sampling among beneficiaries continuously enrolled throughout the year with all health plan beneficiaries enrolled for a minimum of 1 month. For health plan beneficiaries with a minimum of 1 month of enrollment we calculated hospitalization rates in 2 ways: (1) counting only those ACS hospitalizations while enrolled in the plan and (2) counting any ACS hospitalization throughout the year regardless of whether it was during the period of plan enrollment.

Subjects: California Medicaid beneficiaries 18–64 years of age in 2001.

Results: Forty-four Medicaid plans representing approximately 750,000 beneficiaries were included. On average, 50% (range, 26–69% across plans) of the beneficiaries were enrolled continuously during the year. Plan rankings based on the ACS hospitalization rates for each of the 3 sampling strategies were variably correlated [Spearman correlations 0.26 (P = 0.086), 0.33 (P = 0.031), and 0.71 (P < 0.0001) for pairwise comparisons]. The agreement among the sampling strategies in labeling a health plan as an outlier was not statistically different than random chance (κ = 0.069, P = 0.21).

Conclusions: Judgments regarding health plan performance are affected by limiting the evaluation sample to only those beneficiaries continuously enrolled in a health plan throughout the year. Policymakers should consider the goal of health care measurement when selecting a sampling strategy and explicitly acknowledge the bias that might be introduced by a particular approach.

© 2008 Lippincott Williams & Wilkins, Inc.


Article Tools


Article Level Metrics

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.