In 2005, the Group Insurance Commission of Massachusetts required participating health plans to control costs by stratifying physicians into tiers1. Physician tiering, or physician clinical performance assessment, is a new concept in health care that ranks physicians into two or more categories on the basis of cost efficiency and adherence to accepted clinical practice standards2. “Top-tier” physicians, with better cost-efficiency or quality ratings, are assigned a lower copayment than physicians in the second tier. Ultimately, patients are encouraged to visit top-tier physicians because of lower out-of-pocket fees, while health insurance companies reduce spending by endorsing more cost-efficient practices. Tiering is primarily used in specialties with heavy spending, wide variation in resource use, and established practice guidelines2.
At the present time, there is little information about programs of physician stratification through clinical performance assessment. We therefore performed a cross-sectional study, investigating the tiering of orthopaedic surgeons in Massachusetts. We hypothesized that if the tiering methodologies were robust, health plans would have a high degree of concordance and would tend to rate the same physicians as top-tier. We also studied the physician demographic and practice characteristics associated with top-tier status.
Materials and Methods
We collected demographic, insurance, training, and licensing data on 615 orthopaedic surgeons practicing in the state of Massachusetts. We used the Commonwealth of Massachusetts Board of Registration in Medicine (www.massmedboard.org, updated July 2, 2007) to identify surgeons for the present study. These surgeons were then cross-referenced with physicians listed by the three Massachusetts health plans that publish physician tiering results. We included only surgeons who were licensed and practicing in Massachusetts and who accepted one or more of these three health plans. We excluded surgeons who were practicing outside of Massachusetts. Two physicians who were listed as orthopaedic surgeons in the insurance company registries were found to be physical medicine and rehabilitation specialists, and they were removed from the analysis.
From the Commonwealth of Massachusetts Board of Registration in Medicine, we recorded demographic data on the surgeons, including board certification, Medicaid participation, and the number of malpractice claims filed against them in the last ten years. We calculated the number of years in practice by subtracting the year in which training was completed from 2007. We used the physician’s practice address listed with the Board of Registration to determine practice zip code. From the 2000 United States Census (www.census.gov), we then gathered data on the practice zip code’s population density, the percentage of individuals below the poverty level, and median household income. This approach to demographic data has been used in a number of other studies3-8. Population density data were further used to classify a surgeon’s practice setting as urban (>3000 persons per square mile), suburban (1000 to 3000 persons per square mile), or rural (<1000 persons per square mile). Individual surgeon volume for the physicians performing more than ten joint replacements in 2005 was obtained from the Commonwealth of Massachusetts.
Health Plan Orthopaedic Surgeon Tiering Methods
Table I briefly reviews insurance company methods for stratifying orthopaedic surgeons into tiers for the 2007 study year. We investigated each health plan web site and queried the three insurance companies for details regarding their physician-stratification methodology. However, the availability of in-depth information describing tiering methods was very limited.
All three health plans stratified surgeons on the basis of specific cost-efficiency and performance-quality benchmarks. Plan C analyzed surgeons at the group level only. Each health plan used Symmetry’s Episode Treatment Group (ETG) methodology to assess cost efficiency9. This model analyzes inpatient, outpatient, and pharmaceutical and auxiliary claims data from a comprehensive database originally compiled by Mercer human resource consulting. Claims data were combined and classified into distinct, complete episode treatment groups, with each episode treatment group linked to the treating physician. For example, an episode of back pain would be identified on the basis of billing data and the rate of utilization of imaging studies measured. Next, the method computed a physician’s average resource use for each episode treatment group and compared these values with average resource usage by physicians in the same specialty treating a clinically similar mixture of episode treatment groups.
The three health plans primarily based quality ratings on physician adherence to selected medical practice guidelines and other evidence-based standards. For example, Plan A used an osteoporosis-management measure identifying “the percentage of women sixty-seven years and older who suffered a first bone fracture and who had either a bone mineral density test or a prescription for a drug to treat or prevent osteoporosis during the six months after the date of fracture.”10
Two health plans (Plans B and C) also relied on quantitative patient satisfaction ratings, use of e-prescribing, use of electronic medical records, and physician use of self-made patient registries to gauge quality. The quality measures that were used were diverse and differed by health plan. Quality measures were taken from such sources as the Healthcare Effectiveness Data and Information Set (HEDIS), Resolution Health, the Agency for Healthcare Research and Quality (AHRQ), and Massachusetts Health Quality Partners, among others. Quality scores were calculated and, in conjunction with cost-efficiency data, were used to determine final tier status. Plans A and B classified individual surgeons. Plan C placed all surgeons within a group in the same tier. Plan A required surgeons to meet both cost and quality metrics to reach the top tier.
We computed descriptive univariate data on the surgeon cohort. For ease of interpretation, surgeons were classified by whether they practiced in high-income (top quartile of median income, >$56,000) or low-poverty (bottom quartile, <6% poverty) zip codes. We also analyzed these variables as continuous variables and found no change in results (data not shown).
We then compared the concordance of top-tier classification of surgeons between health plans. We limited the analysis to surgeons who participated in multiple health plans and calculated the kappa statistic. Agreement was defined as poor (κ < 0.20), fair (κ = 0.21 to 0.40), moderate (κ = 0.41 to 0.60), substantial (κ = 0.61 to 0.80), or very good (κ > 0.80)11. For Plan A, zero physicians were in Tier 1, so we analyzed Tier 2 as the effective top tier.
For each health plan, we screened for associations between surgeon demographic factors and top-tier status. Factors that were found to be significantly associated (p < 0.05) were included in multivariate models. We used binary logistic regression, with tier as the dependent variable, and calculated odds ratios and 95% confidence intervals for each factor. We repeated the process for each of the three health plans.
In order to understand the geographic distribution of top-tier orthopaedic surgeons, we used practice zip codes to calculate the number of surgeons practicing in each city or town in Massachusetts. We then stratified the number of surgeons per city/town into quartiles and created a surgeon-distribution map. We proceeded to calculate the percentage of orthopaedic surgeons in a city/town who were rated as top-tier by at least one health plan and then created a map of the results.
Source of Funding
This research was supported in part by the Intramural Research Program of the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health. The funding was used to pay salaries.
The majority (84%) of orthopaedic surgeons were board certified, and 96% reported one or fewer malpractice claims in the last ten years (Table II). Four hundred and eight surgeons (66%) participated in all three health plans.
Health Plan Tier Agreement
The percentage of surgeons who were rated as top-tier varied widely among the health plans, from 21% to 62% (Table III). None of the surgeons participating in Plan A met both of the health plan’s cost-efficiency and quality benchmarks, and therefore, none was rated in the top tier.
Agreement between the health plans as to which surgeons were top-tier was low, ranging from 8% (between Plans B and C) to 28% (between Plans A and B). Only 5.2% of surgeons were rated top-tier by all three health plans. The kappa statistic ranged from 0.04 (Plan B to Plan C) to 0.25 (Plan A to Plan B).
Surgeon Demographics and Top-Tier Status
On univariate analysis, board certification, longer time in practice, more malpractice claims, practicing in a low-poverty area, and Medicaid acceptance were all associated with top-tier status for Plans A and B (p < 0.02 for all). For Plan C, only Medicaid acceptance and suburban practice location were associated with top-tier status (p < 0.029 and p < 0.0001, respectively).
Table IV shows physician factors that were independently associated with top-tier status for each health plan. Practicing in a suburban location was significantly associated with being ranked as top-tier for all three health plans. For Plans A and B, board certification and Medicaid acceptance were also significantly associated with top-tier status. On multivariate analysis, the number of malpractice claims filed against a physician in the last ten years and the number of years in practice were not significantly associated with top-tier ratings for any health plan.
Geographic Distribution of Top-Tier Surgeons
The geographic distribution of orthopaedic surgeons generally reflects the population distribution of the state (see Appendix). However, health plans tended to assign top-tier rankings to physicians who practiced in suburban locations (see Appendix). The Boston urban area had a lower percentage of top-tier surgeons compared with the surrounding suburban areas.
Surgeon Volume and Top-Tier Status
Surgeon volume was not correlated with tiering status for Plans A and B (p = 0.75 and 0.97, respectively). For Plan C, a top-tier status was associated with a lower annual volume of total joint replacements (average, eighty-one compared with fifty-five cases per year; p = 0.01).
Our review of physician clinical performance assessment by three health plans in Massachusetts shows only modest physician tiering agreement between health plans. Top-tier physicians tended to be board certified, to practice in suburban locations, and to accept Medicaid. More years in practice or fewer malpractice claims were not associated with top-tier status.
Physician clinical performance assessment, or physician tiering, posits that physicians may be graded on their quality of care on the basis of administrative and claims data. Physicians who are higher-quality and are highly cost-efficient are assigned to a lower copayment. The lower copayment fee encourages patients to visit more cost-efficient “top-tier” practitioners, thus lowering the cost of health care for the community. The field of physician stratification, however, is in its infancy12. Given that the assessment of physician clinical performance and rewarding for better outcomes is part of many health-care reform strategies, rigorous analysis of current programs is necessary.
Because “high performance” is a quality that is specific to individual surgeons, we hypothesized that, while there would be some variability, the same surgeons would be rated top-tier by multiple health plans. However, agreement between health plans on top-tier status was modest at best. There are three reasonable inferences: (1) the plans are not measuring physician quality, (2) they are measuring very different aspects of quality, or (3) they are giving markedly different weights to the different aspects of quality. For example, two plans may in fact be measuring accurately but placing much more weight on cost effectiveness versus adherence to practice guidelines. The measurement may be further distorted by the limitations of using billing data to make clinical judgments.
Why would practicing in a suburban location, or accepting Medicaid, be associated with top-tier status? There are no data to indicate that higher-quality care is delivered in suburban areas or that lower-quality care is delivered in urban areas. While it is possible that high-quality surgeons may tend to self-cluster into suburban locations, physician tiering methods are represented to be based on adherence to practice guidelines and cost efficiency, which should have little relation to geography. We hypothesize that the clustering of top-tier physicians into suburban areas reflects the failure of tiering methodologies to adequately adjust for the increased severity and complexity of patients who seek care at urban academic medical centers, and we suggest that practice location be used as a controlling factor for risk adjustment13,14. Furthermore, we noted that Plan C would tend to steer patients to lower-volume surgeons, although higher surgeon volume has been associated with better outcomes for patients15,16. However, our data can only document associations and cannot definitively exclude the possibility that high-quality surgeons tend to choose to practice in suburban locations.
Implications for Physicians
Individual physicians receive their tiering report cards from insurance companies in isolation, without access to relevant background data. Most surgeons are discouraged to find that they have not been classified into the top tier. However, in Plan A, for example, no orthopaedic surgeons were classified in the top tier. We strongly encourage health plans to supply data concerning how many physician peers and physicians in their practice area were ranked as top-tier and to clarify the measures used to stratify physicians.
Changing practice patterns seems to be unwarranted because the system is not sufficiently developed. The lack of agreement between health plans demonstrates that many physicians will be top-tier for one plan and lower-tier for another. For now, the difference in copayment level is a nominal $10.00 and is probably unlikely to impact patient choices.
Implications for Patients
There is a true need for greater patient education about the current status of physician tiering. While health plans prominently display a physician’s tier in online and printed directories, information about stratification methodology and its relation to quality of care is limited. For example, a patient may be frustrated to find that his or her surgeon is in the second tier. However, there is no evidence to suggest that second-tier physicians provide lower-quality care, and there is a good possibility that the surgeon may be ranked as top-tier by another health plan. Tiering programs may also disrupt existing referring patterns between physicians that have been developed on the basis of years of trust and good results.
Implications for the Health-Care System
With physician stratification increasing in popularity, payors must be attuned to its limitations. The systems for identifying high and low-quality physicians are nascent and have yet to be validated. If the methodologies for stratifying physicians are unreliable, health plans may not realize substantial cost savings with these programs because patients are not truly steered to high-quality physicians. Finally, greater transparency of tiering methodology will benefit both physicians and health plans.
The present study had several limitations. First, we explored tiering data for just one specialty, and thus our conclusions may not be applicable to other medical specialties. Second, it may be that the lack of agreement between health plans may be a result of measuring different aspects of health-care quality. For example, Plan A may place much greater weight on cost-efficient care, whereas Plan B may emphasize completion of all recommended tests. In such a scenario, the payors are unlikely to agree on physician-tiering status. However, from the patient’s perspective, “high quality” is a global characteristic, and the goal should be to measure and classify physicians on all aspects of quality. We analyzed the first year of this new system, and the methodologies have changed over time. Finally, there is no consensus gold-standard for high physician clinical performance that can be used as a benchmark to validate the physician-tiering systems.
Further research is needed by both physicians and health plans in the area of physician tiering. Ongoing efforts to validate the tiering methods against clinical data are needed. The development of a robust technique for identifying high-quality physicians would certainly prove beneficial for all parties.
Maps depicting orthopaedic surgeon density and distribution by tier are available with the electronic version of this article on our web site at jbjs.org (go to the article citation and click on “Supporting Data”).
Investigation performed at the National Institute of Arthritis and Musculoskeletal and Skin Diseases, National Institutes of Health, Bethesda, Maryland
Disclosure: In support of their research for or preparation of this work, one or more of the authors received, in any one year, outside funding or grants of less than $10,000 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), National Institutes of Health. In addition, one or more of the authors or a member of his or her immediate family received, in any one year, payments or other benefits of less than $10,000 or a commitment or agreement to provide such benefits from commercial entities (Stryker, AO North America, and Best Doctors, Inc.).