The chronic disease burden in the United States represents a significant economic and clinical challenge to the primary care system.1 In 2012, 117 million Americans had one or more chronic health conditions.2 This number is projected to grow to 155 million in 2020.3 In addition to high prevalence, significant shortcomings exist in the quality of care for these patients. Only about half of adult patients in the United States receive recommended care, which is often worse for patients with chronic diseases.4 For example, as few as 24% of patients with diabetes received 3 or more glycosylated hemoglobin tests over a 2-year period4 while the guidelines recommend between 6 and 8 tests within the same time frame.5 This suboptimal care compromises patients’ well-being and health outcomes.
The persistent reported shortage of primary care providers (PCPs) exacerbates these challenges. Currently, physicians, physician assistants, and nurse practitioners (NPs) provide the bulk of primary care services in the country. In 2013, 20% of all PCPs were NPs and 11% were physician assistants.6 By 2025, researchers estimate a 20% shortage in the overall PCP workforce compared with the demand for care.7 This is despite the fact that the NP workforce has grown steadily over the past several decades and is expected to increase by 93% from 2013 to 2025.6
Almost 90% of NPs are educated to take on a PCP role and are well-positioned to alleviate some of the primary care demand by delivering effective care, particularly for chronic diseases.8–10 Primary care practices employing NPs are more likely to complete recommended care management for patients with chronic diseases, and patients with chronic diseases counseled by NPs have significantly lower systolic blood pressure and cholesterol levels compared with patients counseled by physicians alone.11 A recent systematic review concluded patients treated by NP-physician teams have better care processes (eg, superior clinical guideline adherence) compared with physicians practicing alone.12 In addition, primary care delivered by NPs is associated with a decreased risk of preventable hospitalization for many chronic diseases.13
Although evidence shows that NPs provide high quality, safe care, policy, and organizational barriers hinder their optimal use and ability to deliver care to their fullest scope of practice (SOP).14,15 Policy barriers, such as SOP regulations in 29 states that require NPs to have physician oversight in care delivery, have been identified as potential impediments to high quality, accessible care.16,17 States with the least restrictive NP SOP regulations have a 2.5-fold greater likelihood of patients’ receiving primary care from NPs than the most restrictive states.18 Organizational barriers for NPs are also common, including unfavorable practice environments characterized by lack of access to resources and poor relationships with administrators.19,20 Researchers have given substantial attention to practice environments in health care organizations due to their direct and indirect impact on patient care and outcomes21,22; however, little attention has been paid to how NP practice environments affect the quality of care. In primary care, studies show that organizational structures affect clinical effectiveness23 and quality improvement.24 Despite the fact NPs are often utilized to deliver chronic care resulting from their training, which combines community, patient, and family aspects in patient care,25,26 no study has investigated the relationship between NP practice environment within primary care practices and quality of care, particularly for chronic diseases. The purpose of this study is to determine the association between NP practice environments and quality of care for chronic diseases measured by widely accepted Healthcare Effectiveness Data and Information Set (HEDIS) measures.27 We focused on HEDIS measures for asthma, diabetes, and cardiovascular disease (CVD) as these conditions affect millions of Americans and represent major public health concerns.28 Several studies have linked HEDIS measures to organizational structures of practices employing primary care physicians,29,30 yet no studies have been conducted in practices employing NPs.
Theoretical underpinnings for this study were drawn from Donabedian’s31 quality of care model. The model demonstrates structures affect processes, which in turn affect outcomes. In this study, the NP practice environments were identified as the structural attribute of primary care practices, and NPs served as informants about their environment. The process attribute was identified by evaluating the HEDIS performance measures for asthma, diabetes, and CVD.
We used a cross-sectional design and linked survey data from NPs on their practice environments with clinical performance data—HEDIS measures for 3 chronic diseases. We hypothesized better NP practice environments within primary care practices are associated with higher HEDIS measure performance. Approval was obtained from the Institutional Review Board of Columbia University Medical Center.
Setting and Sample
The study was conducted in Massachusetts. Primary care NPs were recruited from the Massachusetts Provider Database (MPD)—a statewide database containing detailed information about health care providers contracting with the 5 major health plans covering over 50% of Massachusetts’s commercially insured residents.32 The MPD is developed and maintained by Massachusetts Health Quality Partners (MHQP)—a coalition of health care, business, and community leaders dedicated to promoting measurable improvements in the quality of health care services in Massachusetts. MHQP annually contacts practices in Massachusetts to collect information about clinicians and update the MPD. In 2012, we obtained the MPD and extracted the practice addresses of NPs who were indicated in the database as PCPs. The MPD is a unique database that allows for identification of practices employing NPs and also their role as PCPs or specialists.
NP Survey Measures
NP practice environments were measured using an instrument with previously demonstrated reliability and validity: the 29-item Nurse Practitioner Primary Care Organizational Climate Questionnaire (NP-PCOCQ).33,34 The NP-PCOCQ has 4 subscales measuring important domains of NP practice environment: NP-Physician Relations (NP-PR), Independent Practice and Support (IPS), Professional Visibility (PV), and NP-Administration Relations (NP-AR). NP-PR contains 7 items measuring aspects of NP-physician relations such as whether physicians trust patient care decisions made by NPs and whether NPs and physicians practice as a team. IPS has 9 items measuring aspects of organizational support for NPs’ ability to independently deliver care such as providing access to resources and staff help as well as whether the organization creates an environment where NPs can practice independently. PV has 4 items that measure how visible the NP role is as a clinician within their employment settings. NP-AR encompasses 9 items measuring the relationship between NPs and administrators including open communication between NPs and administrators or administrators’ awareness about NPs’ skills and competencies. The 4 subscales have high internal consistency reliability with Cronbach α’s ranging from 0.87 to 0.95.34 The tool also exhibits construct, discriminant, and predictive validity.33
NP practice environment measures were collected at the individual NP-level, which then were aggregated to the organizational-level (the practice environment is a property of the organization and not of the individual).35,36 This approach also reduces measurement error. First, we computed the mean score on each NP-PCOCQ subscale for each NP and then aggregated the responses of all NPs within the same practice to create a single organizational-level practice environment score for each practice on each subscale. Higher organizational-level mean scores on each subscale indicate better NP practice environments within the practice.
NPs also completed measures of age, sex, race, and provided information on the characteristics of their practices including number of NPs, geographic location, patient panel, and practice setting type (physician office, community health center, or hospital-based clinic).
HEDIS Performance Measures
MHQP collects clinical performance data using HEDIS measures from the 5 commercial health plans in Massachusetts. HEDIS measures are a nationally standardized reporting system for health plans to measure performance on important dimensions of care delivery.27 Four of the 5 plans provide MHQP with de-identified data at the individual patient level and 1 health plan reports the data at the PCP-level.
MHQP links the data at the PCP-level and then practice-level to assess practice performance. For each HEDIS measure, a higher score reflects higher quality of care in the practice and is calculated by dividing the number of eligible patients who received the recommended care by the number of patients eligible for inclusion for the measure. If a patient is eligible for the measure, they are designated with either a “1” if the service was achieved or a “0” if not.32 Next, MHQP sums the numerators and denominators for all providers (both NPs and physicians) within the same practice and divides it by the aggregate numerator and denominator to present clinical performance at the practice level.
Data Collection and Merging
Using mail survey procedures, we sent surveys to 807 NPs listed as PCPs in the MPD using their practice addresses to gather data on their practice environment. We followed a modified Dillman et al37 method to send 1 reminder and conduct a second mailing for nonresponders. Overall, 314 NPs from 163 primary care practices in Massachusetts completed and returned the surveys, yielding a response rate of 40%. We also obtained the clinical performance dataset from MHQP which contained the following practice-level HEDIS measures for adult patients with asthma, diabetes, and CVD in 201232: (1) appropriate asthma medications for patients ages 12–50 (the proportion of the patients in the practices who were identified as having persistent asthma during the year before the measurement year and who were appropriately prescribed medications during the measurement year); (2) comprehensive diabetes care HbA1c testing [the proportion of patients aged 18–75 y with diabetes (type 1 and type 2) who had a HbA1c test in 2012]; and (3) cholesterol management for patients with CVD, low density lipoprotein (LDL) screening [the proportion of patients who received screening for LDL cholesterol (LDL-C) test within 1 year among adult patients ages 18–75 y old discharged alive for acute myocardial infarction, coronary artery bypass graft or percutaneous coronary interventions during the year before the measurement year, or who had a diagnosis of ischemic vascular disease during the measurement year and the year before the measurement year].
Using the unique practice identifiers both in the NP survey and the clinical performance dataset, we merged the data at practice level. While NP survey data was available for 163 practices, the clinical performance data was available for only 118 practices. MHQP collects HEDIS measures only for practices with at least 3 providers—only 118 practices from which we had 221 NP surveys met this criterion.
We first calculated the descriptive statistics on the demographic characteristics of NPs and practice attributes. Next, we assessed bivariate relationship between the outcome measures and each of the following potential organizational-level covariates: average age of NPs, female ratio, white ratio, proportion of NPs with their own patient panel, proportion of NPs with master’s degree or higher, practice site type, and practice site urbanicity. Covariates with P-value<0.2038 as well as the 4 organizational-level NP-PCOCQ subscale scores—the main predictor variables—were entered into the final multivariable regression models.
The outcome variable in the models were the value of the practice-level HEDIS measures (ranging from 0 to 1) indicating the proportion of the patients in the practices receiving the recommended service. Fractional logistic regression models were used. This type of model helps assess the effect on continuous proportion outcomes with a range of 0–1. The mean scores from 4 NP-PCOCQ subscales were entered into the models simultaneously. All predictors in final models were at organizational-level variables and for meaningful interpretation, all continuous variables were standardized scores with mean of 0 and SD of 1. The multicollinearity of the 4 main predictors—the organizational-level mean scores of the NP-PCOCQ subscales—were checked. PROC GLIMMIX in SAS 9.439 was used to conduct the fractional logistic regression analysis. Odds ratios (OR) and 95% confidence intervals (CI) were reported to assess the strength and direction of the relationship. As we only had 3 planned tests, to avoid inflated type II error, we did not perform multiple testing adjustment.40,41
The organizational-level characteristics of NPs and their practices are reported in Table 1. The average number of NPs in practices was 1.87 (SD=1.48) ranging from 1 to 12 NPs. The average age of NPs was 50.58 years (SD=11.67). The average proportion of NPs across the practices with a master’s degree was 88% (SD=28%). About 90% (SD=33%) of NPs were white. Physician offices comprised the largest group of practices (37%) while community health centers represented the next largest group (31%). Over half of practices (57%) were located in urban areas.
The organizational-level mean scores on NP-PCOCQ subscales are presented in Table 2. They ranged from 2.88 (SD=0.65) on NP-AR to 3.49 (SD=0.36) on IPS. The mean score on the PV subscale was 3.15 (SD=0.57) and 3.38 on the NP-PR subscale (SD=0.47). The outcome variables—HEDIS measures—represent a proportion with a mean score of 0.89 (SD=0.18), 0.91 (SD=0.13), and 0.94 (SD=0.07) for medication management for patients with asthma, LDL-C screening for CVD, and HbA1c testing for patients with diabetes, respectively. Variance of inflation factor was calculated to estimate the severity of multicollinearity. The variance of inflation factors of the NP-PCOCQ’s subscales fell below the cutoff value of 10 for high multicollinearity.42 Thus, multicollinearity is not a concern. Of the 118 practices, 6 had missing data in the NP-PCOCQ subscale scores or any of the significant covariates. Rate of missing data is small (5.1%), thus listwise deletion was used when building final multivariable models.
Table 3 presents the results from the final models assessing the effect of NP practice environment measures on the quality of care measures after controlling for confounding variables. NP-AR subscale score was a significant predictor for asthma medication management. The higher the organizational-level NP-AR subscale mean score, the better the medication management for patients with asthma (OR=1.96; 95% CI, 1.23–3.12): with 1 SD increase in the mean score of this subscale, the odds of medication management almost doubled. In this model, female ratio and practice setting were significantly associated with the outcome as well. Compared with physician offices, hospital-based clinics were less likely to optimally manage asthma medications.
For CVD, IPS represented the only significant predictor among NP-PCOCQ subscales. The higher the organizational-level IPS subscale mean score, the better the quality of care measure (OR=1.60; 95% CI, 1.03–2.47): with one SD increase in the mean score of this subscale, the odds of receiving recommended screening increased 60%. In this model, the patient panel ratio—the number of NPs with their own patient panel among all NPs within the practice—was significantly associated with the outcome; the practices with higher patient panel ratios were less likely to deliver the recommended service.
The models showed no effect of the NP-PCOCQ subscales on the outcome measure for diabetes (HbA1c testing). Nonurban location of the practice was the only variable to be significantly associated with the outcome variable in the bivariate analysis; however, in the full model it was not significant.
We investigated the relationship between NP practice environments in primary care practices and HEDIS measures evaluating quality of care for asthma, diabetes, and CVD. Our findings indicate that the likelihood of patients with asthma and CVD to receive high quality of care in their practices improves when NPs deliver care in practices with favorable environments. Specifically, if practices promote a favorable relationship between NPs and administrators, patients in these practices with asthma have increased likelihood of receiving adequate medication management. Administrators supporting NP practice may create flexible schedules for NPs allowing NPs to allocate more time towards patient education hence promoting asthma medication management.
In addition, support for NP independent practice improves the likelihood of patients with CVD to receive LDL-C screening. Providing adequate support for NP independent practice may allow NPs to deliver care within their SOP, follow clinical guidelines, and proactively screen patients leading to higher LDL-C screening.
No effect of NP practice environment on diabetes care was detected. Compared with the other 2 outcome measures (medication management for asthma and LDL-C screening for CVD) the measure for diabetes lacked variability with a SD of 0.07 and a higher mean score. Also, on the other outcome variables a full range (0–1) was observed. The measure for diabetes had a narrower range (0.56–1.0). It appears that practices deliver comparable diabetes care and are better at caring for patients with diabetes than for patients with asthma or CVD.
A small sample of NPs from primary care practices in one state participated in our study, which might limit the generalizability of the findings. However, the sample’s demographic characteristics are comparable to those of Massachusetts’ nursing workforce. For example, 90% of NPs in our sample were white, which is comparable to 89.8% of the nursing workforce being white in Massachusetts.43 The mean age of NPs in our sample is about 50 years. The mean age of nurses in Massachusetts is 49 years.43
This is the first study to demonstrate how NP practice environment affects quality of care for patients with asthma and CVD and provides evidence that promoting organizational structures of practices employing NPs could potentially improve quality of care. The study findings have important policy, practice, and research implications. As policymakers are in constant search of strategies to deliver better care to patients, it is critical to support the independent practice of NPs. Not only is NP independent practice restricted by the SOP regulations in many states,44 but practices employing NPs often do not provide support for NPs to practice independently. Our study findings demonstrate that a lack of support for NP independent practice may negatively impact the quality of care delivered to patients with chronic diseases, particularly to those with CVD. Given the large demand for primary care services and the increasing number of NPs nationwide,6 it is vital both for states and organizations to create policies that support the ability of NPs to independently care for their patients and concurrently improve the quality of care.
Our study findings can help practice administrators take actions to modify NP practice environments to improve quality of care for asthma and CVD. Two particular factors should be promoted. First, the independent practice of NPs within their employment settings should be supported which may include ensuring that NPs have access to care delivery resources such as support staff. Studies demonstrate that within primary care practices, NPs often lack staff support which leads them to taking on patient care responsibilities typically delegated to medical assistants or registered nurses.15,45 Such underutilization of NPs’ advanced skills and competencies leads to delays in patient care and could increase the cost of caring for patients by 9%–12%.46 As more NPs will be employed in primary care practices, administrators can play a critical role in improving the quality of care within their practices by supporting the NP independent practice. Second, attention should be paid to the relationship between NPs and administrators. Studies consistently demonstrate that NPs and physicians have favorable relationships within practices whereas the relationship between NPs and administrators is challenging, which in turn negatively affects NP-physician teamwork.19,47 In this study, the relationship between NPs and administrators also affects the quality of care for patients with asthma. Thus, efforts should be made to improve this relationship by promoting the communication between NPs and administrators and increasing administrators’ awareness about NPs’ skills and competencies to potentially improve teamwork and care quality.
The study has several limitations. We were not able to link patients directly to NPs as we only had access to practice-level HEDIS measures. However, practice-level measures are still useful as NPs are often utilized in team care models and these measures capture the contributions of NPs to practice-level quality of care. Practice environment measures relied on self-reports. Nonresponse bias might be an issue. However, previous research48 surveying nurses and then reaching out to nonresponders showed no differences in nurse-reported organizational measures between responders and nonresponders. MHQP did not compute the asthma HEDIS measure for adult patients. Thus, our measure included both pediatric and adult patients which is a limitation. In addition, we used a small sample of practices. Although the effect of IPS on LDL-C screening was significant (P=0.035), the P-value was close to 0.05 cutoff. Future research with a large sample of practices should be conducted. Studies should also explore the aspects of NP practice environments that are important for care delivery for specific conditions. We found different dimensions of NP practice environment to predict outcomes. Future research should investigate how to modify NP practice environments to improve care for other conditions and also include interventions to improve NP practice environments. Research studies could also test the effect of NP practice environment on patient outcomes (eg, hospitalization or emergency department visits for ambulatory care sensitive conditions). Further research should be conducted to explore care delivery in hospital-based clinics and in practices where NPs’ are assigned a large number of patients. Finally, future studies should be expanded to other states given the variable SOP regulations for NPs.
To our knowledge, this is the first study linking quality of care measures for chronic diseases to NP practice environments. This study provides evidence that promoting NP practice environments leads to better chronic care delivery. More research examining the association between NP practice environments and additional HEDIS measures is recommended. Efforts to improve NP practice environments should be implemented and evaluated.
The authors thank Massachusetts Health Quality Partners for providing them with data.
1. DeVol R, Bedroussian A. An Unhealthy America: The Economic Burden of Chronic Disease
. Santa Monica, CA: Milken Institute; 2007.
2. Ward BW, Schiller JS, Goodman RA. Multiple chronic conditions among US adults: a 2012 update. Prev Chronic Dis. 2014;11:E62.
3. Wu SY, Green A. Projection of Chronic Illness Prevalence and Cost Inflation. Santa Monica, CA: RAND Corporation; 2000.
4. McGlynn EA, Asch SE, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645.
7. Sargen M, Hooker RS, Cooper RA. Gaps in the supply of physicians, advance practice nurses, and physician assistants. J Am Coll Surg. 2011;212:991–999.
9. Newhouse RP, Stanik-Hutt J, White KM, et al. Advanced practice nurse outcomes 1990-2008: a systematic review. Nurs Econ. 2011;29:1–22.
10. Auerbach DI, Chen PG, Friedberg MW, et al. Nurse-managed health centers and patient-centered medical homes could mitigate expected primary care physician shortage. Health Aff. 2013;32:1933–1941.
11. Ohman-Strickland PA, Orzano AJ, Hudson SV, et al. Quality of diabetes care in family medicine practices: influence of nurse-practitioners and physician’s assistants. Ann Fam Med. 2008;6:14–22.
13. Kuo Y-F, Chen N-W, Baillargeon J, et al. Potentially preventable hospitalizations in Medicare patients with diabetes: a comparison of primary care provided by nurse practitioners versus physicians. Med Care. 2015;53:776–783.
14. Poghosyan L, Boyd D, Clarke SP. Optimizing full scope of practice for nurse practitioners in primary care: a proposed conceptual model. Nurs Outlook. 2016;64:146–155.
15. Poghosyan L, Nannini A, Smaldone A, et al. Revisiting scope of practice facilitators and barriers for primary care nurse practitioners: a qualitative investigation. Policy Polit Nurs Pract. 2013;14:6–15.
16. Graves JA, Mishra P, Dittus RS, et al. Role of geography and nurse practitioner
scope of practice in efforts to expand primary care system capacity: health reform and the primary care workforce. Med Care. 2016;54:81–89.
17. Xue Y, Ye Z, Brewer C, et al. Impact of state nurse practitioner
scope-of-practice regulation on health care delivery: systematic review. Nurs Outlook. 2016;64:71–85.
18. Kuo Y-F, Loresto FL, Rounds LR, et al. States with the least restrictive regulations experienced the largest increase in patients seen by nurse practitioners. Health Aff. 2013;32:1236–1243.
19. Poghosyan L, Liu J. Nurse practitioner
autonomy and relationships with leadership affect teamwork in primary care practices: a cross-sectional survey. J Gen Intern Med. 2016;31:771–777.
20. Poghosyan L, Aiken L. Maximizing nurse practitioners’ contributions to primary care through organizational changes. J Ambul Care Manage. 2015;38:109–117.
21. Aiken LH, Sloane DM, Clarke S, et al. Importance of work environments on hospital outcomes in 9 countries. Int J Qual Health Care. 2011;23:357–364.
22. Carthon BM, Lasater KB, Sloane DM, et al. The quality of hospital work environments and missed nursing care is linked to heart failure readmissions: a cross-sectional study of US hospitals. BMJ Qual Saf. 2015;24:255–263.
23. Brazil K, Wakefield D, Cloutier M, et al. Organizational culture predicts job satisfaction and perceived clinical effectiveness in pediatric primary care practices. Health Care Manage Rev. 2010;35:365–371.
24. Dugan DP, Mick SS, Scholle SH, et al. The relationship between organizational culture and practice systems in primary care. J Ambul Care Manage. 2011;34:47–56.
25. Litaker D, Mion LC, Planavsky L, et al. Physician–nurse practitioner
teams in chronic disease
management: the impact on costs, clinical effectiveness, and patients’ perception of care. J Interprof Care. 2003;17:223–237.
26. Watts SA, Gee J, O’Day ME, et al. Nurse practitioner
-led multidisciplinary teams to improve chronic illness care: the unique strengths of nurse practitioners applied to shared medical appointments/group visits. J Am Assoc Nurse Pract. 2009;21:167–172.
29. Friedberg MW, Coltin KL, Safran DG, et al. Associations between structural capabilities of primary care practices and performance on selected quality measures. Ann Intern Med. 2009;151:456–463.
30. Mehrotra A, Epstein AM, Rosenthal MB. Do integrated medical groups provide higher-quality medical care than individual practice associations? Ann Intern Med. 2006;145:826–833.
31. Donabedian A. Evaluating the quality of medical care. Milbank Q. 1966;44:166–206.
33. Poghosyan L, Chaplin WF, Shaffer JA. Validation of Nurse Practitioner
Primary Care Organizational Climate Questionnaire: a new tool to study nurse practitioner
practice settings. J Nurs Meas. 2017;25:142–155.
34. Poghosyan L, Nannini A, Finkelstein SR, et al. Development and psychometric testing of the Nurse Practitioner
Primary Care Organizational Climate Questionnaire (NP-PCOCQ). Nurs Res. 2013;62:325–334.
35. Litwin GA, Stringer RA. Motivation and Organizational Climate. Boston, MA: Harvard University Press; 1968.
36. Lake ET. The nursing practice environment
. Med Care Res Rev. 2007;64 (suppl):104S–122S.
37. Dillman DA, Phelps G, Tortora R, et al. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet. Soc Sci Res. 2009;38:1–18.
38. Hosmer DW, Lemeshow S. Applied Logistic Regression, 2nd ed. New York, NY: John Wiley & Sons Inc.; 2000.
39. SAS Institute. SAS (for Windows) (Computer Program) Version 94. Cary, NC: SAS Institute; 2013.
40. Anderson NH. Empirical Direction in Design and Analysis. New York, NY: Routledge; 2014.
41. Thompson BThompson B. Planned versus unplanned and orthogonal versus nonorthogonal contrasts: the neo-classical perspective. Advances in Social Science Methodology (Vol 3). Greenwich, CT: JAI Press; 1994:3–27.
42. Allison PD. Multiple Regression: A Primer. Thousand Oaks, CA: Pine Forge Press; 1999.
45. Poghosyan L, Norful A, Martsolf G. Primary care nurse practitioner
practice characteristics: barriers and opportunities for interprofessional teamwork. J Ambul Care Manage. 2017;40:77–86.
46. Liu N, Finkelstein SR, Poghosyan L. A new model for nurse practitioner
utilization in primary care: increased efficiency and implications. Health Care Manage Rev. 2014;39:10–20.
47. Poghosyan L, Nannini A, Stone P, et al. Nurse practitioner
organizational climate in primary care settings: implications for professional practice. J Prof Nurs. 2013;29:338–349.
48. Aiken LH, Sloane DM, Cimiotti JP, et al. Implications of the California nurse staffing mandate for other states. Health Serv Res. 2010;45:904–921.