Skip Navigation LinksHome > May 2009 - Volume 39 - Issue 5 > Psychometric Evaluation of the Revised Professional Practice...
Text sizing:
A
A
A
Journal of Nursing Administration:
doi: 10.1097/NNA.0b013e3181a23d14
Articles

Psychometric Evaluation of the Revised Professional Practice Environment (RPPE) Scale

Erickson, Jeanette Ives MS, RN, FAAN; Duffy, Mary E. PhD, FAAN; Ditomassi, Marianne MSN, RN, MBA; Jones, Dorothy EdD, RN, FAAN

Free Access
Article Outline
Collapse Box

Author Information

Authors' Affiliation: Senior Vice President (Ms Erickson), Senior Nurse Scientist (Dr Duffy), Executive Director (Ms Ditomassi), and Director of Nursing Research (Dr Jones), Massachusetts General Hospital, Boston.

Corresponding author: Dr Duffy, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114 (mduffy4@partners.org).

Collapse Box

Abstract

Objective: The purpose was to examine the psychometric properties of the Revised Professional Practice Environment (RPPE) scale.

Background: Despite renewed focus on studying health professionals' practice environments, there are still few reliable and valid instruments available to assist nurse administrators in decision making.

Methods: A psychometric evaluation using a random-sample cross-validation procedure (calibration sample [CS], n = 775; validation sample [VS], n = 775) was undertaken.

Results: Cronbach α internal consistency reliability of the total score (r = 0.93 [CS] and 0.92 [VS]), resulting subscale scores (r range: 0.80-0.87 [CS], 0.81-0.88 [VS]), and principal components analyses with Varimax rotation and Kaiser normalization (8 components, 59.2% variance [CS], 59.7% [VS]) produced almost identical results in both samples.

Conclusions: The multidimensional RPPE is a psychometrically sound measure of 8 components of the professional practice environment in the acute care setting and sufficiently reliable and valid for use as independent subscales in healthcare research.

Over the past 20 years, there has been a renewed emphasis on the organizational context within which healthcare is delivered.1-2 Despite this increased focus on the work environment, research initiatives evaluating organizational outcomes have not kept pace. This is due in part to the paucity of psychometrically sound instruments that measure specific aspects of the professional practice environment.3 Lake,4 in her literature review of the types of research studies that used specific instruments to evaluate practice environments, found 203 articles meeting the inclusion criteria. Of this number, only 7 multidimensional instruments were found, 5 of which were developed for nursing research and 2 for behavioral or management science. Thus, it is essential that nursing leadership faced with increasing demands and diminishing resources have reliable and valid data upon which to base their decision making to deliver safe, efficient, and effective patient care. The purpose of this article was to report on the psychometric properties of the Revised Professional Practice Environment (RPPE) scale.

Like its predecessor, the Professional Practice Environment (PPE) scale,3 the RPPE is a conceptually grounded, multidimensional measure of 8 components of professional clinical practice in the acute care setting. In the late 1990s, a strategic planning process was initiated at the Massachusetts General Hospital (MGH) in Boston to create a shared vision for the 6 clinical disciplines within the newly created structure called Patient Care Services (PCS). One of the outcomes of that effort was the development of the interdisciplinary MGH Professional Practice Model that provided a comprehensive view of professional practice. The Professional Practice Model's core elements are: professional staff leadership and autonomy in practice; control over one's practice; collaborative governance stressing staff participation in decision making about patient care and the environment within which care is delivered; interdisciplinary communication and teamwork; use of a problem-solving approach to handle disagreements and conflict; enhanced internal work motivation; and delivery of culturally sensitive, competent care to patients of all ethnic groups.5 This model guided the development of the original version of the PPE scale.

Back to Top | Article Outline

Backgrounds

Much of the work on developing professional practice environment instruments sprang from the first Magnet hospital study.6 Using the findings from this study, Kramer and Hafner7 developed the Nursing Work Index (NWI), a 65-item scale designed to measure what nurses in Magnet hospitals believed were important characteristics of their professional practice environments. Two scores were obtained: job satisfaction and quality care. Eleven years later, Aiken and Patrician8 examined the 65 NWI items from a conceptual perspective and developed the Revised Nursing Working Index (R-NWI). The 57-item R-NWI, comprising 55 of the original NWI items and 2 additional items, measured 4 subscales: autonomy, control over work environment, relationship with physicians, and organizational support of caregivers.

Using factor analytic techniques on the NWI data, Lake9 developed the Practice Environment Scale of the NWI, which measured 5 components: nurse participation in hospital affairs; nursing foundations for quality of care; nurse manager ability, leadership, and support; staffing and resource adequacy; and nurse-physician relations. In addition, higher order factor analysis of the 5 subscales resulted in one major composite, called the Practice Environment Scale. Also using factor analytic techniques, Estabrooke and colleagues10 developed a 1-factor 26-item scale, the Practice Environment Index, from the R-NWI items.

Back to Top | Article Outline

Development of the Original MGH PPE Scale

The PPE scale was first developed in late 1998 to evaluate the effectiveness of the PCS' new practice environment in supporting clinicians in their delivery of patient care. This 35-item scale was designed to measure 8 clinical practice environment characteristics: leadership and autonomy in clinical practice (5 items), staff relationships with physicians (2 items), control over practice (6 items), communication about patients (3 items), teamwork (4 items), handling disagreement and conflict (8 items), internal work motivation (4 items), and cultural sensitivity (3 items).

Back to Top | Article Outline

Definitions

The 8 professional practice environment characteristics were defined as follows. Leadership and autonomy in clinical practice is the quality or state of being self-governing and exercising professional judgment in a timely fashion.11 Staff relationships with physicians are those associations with physicians that facilitate exchange of important clinical information.11 Control over practice signifies sufficient intraorganizational status to influence others and deploy resources when necessary for good patient care.12 Communication about patients is defined as the degree to which patient information is related promptly to the people who need to be informed through open channels of interchange.13 Teamwork is viewed as a conscious activity aimed at achieving unity of effort in the pursuit of shared objectives.14 Handling disagreement and conflict represents the degree to which managing discord is addressed using a problem-solving approach.14 Internal work motivation is self-generated encouragement completely independent of external factors such as pay, supervision, or coworkers.5,15,16 Cultural sensitivity is a set of attitudes, practices, and/or policies that respects and accepts cultural differences.5

After completing the test pool, 7 PCS staff members reviewed each item for readability, clarity, meaning, and congruence with the conceptual category it was designed to measure. After minor editing, all items were retained. Each item was then placed on a 4-point Likert scale of strongly agree, agree, disagree, and strongly disagree for participants' responses.

This version of the PPE scale was used for 3 years (1999-2001) to evaluate the effectiveness of the MGH professional practice environment and to monitor changes made in the environment in response to previous data. At the end of this period, we evaluated the internal consistency of the PPE subscales and noted that the internal work motivation scale composed of 4 items had low internal consistency (r = 0.63). When the distribution of scores was examined, we found high homogeneity of staff responses on these items. Thus, we developed 4 additional items to generate greater response variation on this scale. These items were reviewed for conceptual congruence with the scale definition and added to the scale. In addition, there was one item in the handling disagreement and conflict scale that contained 2 ideas. This item was edited to form 2 items in an effort to eliminate possible confusion for respondents. Now 40 items in length, this version of the PPE scale was then examined for psychometric adequacy and reported elsewhere.3 Because there are unequal numbers of items defining each subscale, average scores were used so that all subscale scores have equal weight.

Back to Top | Article Outline

Development of the RPPE Scale

The 40-item PPE scale mentioned above underwent further revision in 2005 when the MGH senior vice president for PCS and associate chief nurses and directors revised strategic goals. Nursing leadership reviewed all items and edited them for greater clarity. Two additional items were added to the handling disagreement and conflict scale, namely, "Most conflicts occur with members from my own discipline" and "Most conflicts occur with members from other disciplines." These items were designed to more clearly pinpoint where conflicts and disagreements originated. In addition, the now named RPPE scale was developed as an online version so as to provide greater ease in respondent participation and to decrease data preparation time since the surveys would be completed electronically and data would be directly entered into a database for subsequent analysis. The RPPE scale, 42 items in length, was the version used in the MGH 2006 Staff Perceptions of the Professional Practice Environment study, which received exempt institutional review board approval. The scale, distributed electronically to MGH professional practice staff, yielded a 61% response rate (n = 1,837).

Back to Top | Article Outline

Sample

Psychometric evaluation of the RPPE was then undertaken on all staff in the 2006 sample who had no missing data on the scale (n = 1,550). Because the sample size was large enough, a random sample cross-validation procedure17,18 was used to test whether the 8 original components in the RPPE could be derived in one sample and validated in a comparable sample drawn from the same population of MGH staff. The calibration sample (n = 775) was used to derive the underlying components; the validation sample (n = 775) was used to confirm the component structure. If both samples yielded the same or very similar results, this would provide further evidence of construct validity.19 As Table 1 shows, the 2 samples were comparable, with no significant differences on the demographic characteristics of age, sex, highest educational level, number of years in the profession, and number of years at MGH. Sample size for both samples (n = 775) was more than adequate to undertake principal components analyses (PCAs) with each sample having an approximate 20:1 case-to-variable ratio.20,21

Table 1
Table 1
Image Tools
Back to Top | Article Outline

Findings

Psychometric evaluation of the RPPE included (a) internal consistency reliability using Cronbach α and item analysis; (b) confirmatory PCA using the previously described random sample, cross-validation technique; and (c) internal consistency reliability of resulting components using Cronbach α.

Back to Top | Article Outline

Initial Reliability Estimates and Item Analyses

Item-total correlations were computed for the 42-item RPPE in both the calibration sample (n = 775) and the validation sample (n = 775). The Cronbach α was .93 for the calibration sample and .92 for the validation sample. In both analyses, the same 5 items, shown in bold italics in Tables 2 and 3, had item-total correlations below 0.30. Because of the multidimensional nature of the PPE construct, however, we decided to keep the items in the scale at this time and include them in PCAs to determine how well they would fare.

Table 2
Table 2
Image Tools
Table 3
Table 3
Image Tools
Back to Top | Article Outline

Calibration Sample-PCAs

Principal components analysis followed by Varimax rotation and Kaiser normalization was next performed on the calibration sample (n = 775) specifying 8 components. Examination of the rotated component matrix revealed a parsimonious and interpretable solution. All but 3 items loaded greater than the 0.30 component loading cutoff on one of the 8 components. There were very few substantial side loadings. Table 2 displays the RPPE items and their component loadings on the PCA-derived scales, which accounted for a total of 59.2% of initially extracted common variance. Component 1, handling disagreement and conflict, defined by 9 items with an eigenvalue of 11.2, accounted for 26.7% of variance. Component 2, leadership and autonomy in clinical practice, composed of 5 items with an eigenvalue of 3.1, explained an additional 7.3% of variance. Component 3, internal work motivation, defined by 8 items, had an eigenvalue of 2.6 and added 6.1% of variance. Components 4 through 8 with eigenvalues of 1.9 (control over practice-5 items), 1.7 (teamwork-4 items), 1.7 (communication about patients-3 items), 1.4 (cultural sensitivity-3 items), and 1.3 (staff relationships with physicians-2 items), added 4.6%, 4.1%, 4.0%, 3.4%, and 3.0% of variance, respectively. The 3 items that did not load significantly (>0.30) on any component were "Patient care assignments foster continuity of care," "I am asked to do things against my professional judgment," and "Most conflicts occur with members from other disciplines."

Back to Top | Article Outline

Validation Sample-PCAs

The same type of PCA was next undertaken on the validation sample (n = 775) and produced almost identical results. As Table 3 shows, all 8 components were defined by the same items and in the same order that were demonstrated in the calibration sample PCA (Table 2). The same 3 items that were dropped from the component structure due to components loadings less than 0.30 in the calibration sample were also dropped in the validation sample PCA. The total amount of shared variance in the second PCA was 59.7%, only 0.5% higher than the explained variance in the calibration sample PCA.

Back to Top | Article Outline

Internal Consistency Reliability of RPPE Subscales

Before computing RPPE mean subscale scores, Cronbach α internal consistency reliabilities for each of the 8 PCA-derived components were next computed on both samples' scores. As Tables 2 and 3 show, subscale reliabilities ranged from .80 to .87 in the calibration sample and from .81 to .88 in the validation sample. Thus, the now 39-item RPPE scale's 8 components of the professional practice environment were judged sufficiently reliable for use as independent measures in subsequent research. In addition, these findings demonstrate that the RPPE is psychometrically equivalent to its predecessor, the PPE scale.22

Back to Top | Article Outline

Administration

The RPPE is self-administered. Respondents are directed to a specific agency-based, secure Web site where they are instructed to complete the RPPE online. It takes approximately 10 minutes to provide answers on the RPPE. Since the RPPE scale is scored so that high scores represent high amounts of the construct being measured, 7 items need to be reverse scored. Because there are unequal numbers of items defining each RPPE subscale, average scores need to be used so that all subscale scores have equal weight. All mean subscale scores are formed by adding the subscale items together and then dividing that sum by the number of items in the subscale.

Back to Top | Article Outline

Discussion

Results from this psychometric evaluation of the now 39-item MGH RPPE scale indicated that all 8 subscales are reliable and construct valid for use as independent dimensions of the professional practice environment in today's acute care setting. In contrast to the NWI and its derivative scales, the RPPE offers a more comprehensive picture of today's professional practice environment. In addition to measuring all the professional characteristics springing from the Magnet hospital studies, the RPPE also measures professional staffs' ability to handle disagreement and conflict using a problem-solving approach, their internal work motivation, communication about patients, and cultural sensitivity.

The RPPE serves as an effective report card of the health of the professional practice environment and is linked to a model of practice that aspires to achieve these outcomes. Such information can help nursing leadership design and/or improve the various components of an individual unit or department practice setting and provide evaluative feedback to leadership about whether such changes have made a difference in practice. At MGH, PCS management and staff have used PPE and RPPE item data in this way for more than 9 years. They report that the RPPE subscale and item scores provide valuable information describing effective professional practice environments. The RPPE data linked to new initiatives and changes in practice serve as evidence to support or refute leadership response to professional staff concerns (Figure 1).

Figure 1
Figure 1
Image Tools

If unit or department identifiers are available and linked to respondent data, subscale scores can also be created at the unit or department level by averaging individual scores from the appropriate unit or department staff. However, moving from the individual to the unit or department level changes the unit of analysis, making it much smaller, depending on the number of units/departments in the study sample.

For Magnet-recognized organizations or for organizations pursuing Magnet recognition, the RPPE scale is an effective tool to measure baseline and ongoing perceptions of clinicians' impressions of their professional practice model, which are aligned with the 5 model elements of Magnet recognition, namely, transformational leadership; structural empowerment; exemplary professional practice; new knowledge, innovations, and improvements; and empirical outcomes.23 Through annual administration of the RPPE, a greater understanding of organizational concepts that enhance clinical practice can be achieved. Such data help illustrate which support structures are needed to hardwire the Institute of Medicine's 6 aims (patient centeredness, safety, effectiveness, efficiency, timeliness, and equity of care) into practice.24

Back to Top | Article Outline

Summary

These findings indicate that the multidimensional RPPE is a psychometrically sound measure of 8 components of the professional practice environment in an acute care setting, namely, handling disagreement and conflict, leadership and autonomy in clinical practice, internal work motivation, control over practice, teamwork, communication about patients, cultural sensitivity, and staff relationships with physicians. As well as being psychometrically sound, the RPPE demonstrates substantive coherence and application at both the individual and one or more organizational levels of analysis.

Back to Top | Article Outline

References

1. Mark B. Methodological issues in nurse staffing research. West J Nurs Res. 2006;28(6):694-709.

2. McGillis Hall L, Doran D. Nurses' perceptions of hospital work environments. J Nurs Manag. 2007;15:264-273.

3. Ives Erickson J, Duffy M, Gibbons M, Fitzmaurice J, Ditomassi M, Jones D. Development and psychometric evaluation of the Professional Practice Environment (PPE) scale. J Nurs Scholarsh. 2004;6(3):279-285.

4. Lake E. The nursing practice environment: measurement and evidence. Med Care Res Rev. 2007;64(104):1048-1228.

5. Ives Erickson J. Keeping in touch with staff perceptions of the professional practice environment. Caring Headlines. July 20, 2000:2.

6. McClure M, Poulin M, Sovie M, Wandelt M. Magnet Hospitals: Attraction and Retention of Professional Nurses. Kansas City, MO: American Nurses Association; 1981.

7. Kramer M, Hafner L. Shared values: impact on nurse job satisfaction and perceived productivity. Nurs Res. 1989;38:172-177.

8. Aiken L, Patrician P. Measuring organizational traits of hospitals: the Revised Nursing Work Index. Nurs Res. 2000;49:146-153.

9. Lake E. Development of the Practice Environment Scale of the Nursing Work Index. Res Nurs Health. 2002;25(3):176-188.

10. Estabrooke C, Tourangeau A, Humphrey C, et al. Measuring the hospital practice environment: a Canadian context. Res Nurs Health. 2002;25(4):256-268.

11. Zimmerman J, Shortell S, Rousseau D, et al. Improving intensive care: observations based on organizational case studies in nine intensive care units. Crit Care Med. 1993;21(10):1443-1551.

12. Hackman J, Oldham G. Motivation through the design of work: test of a theory. Organ Behav Hum Perform. 1976;16(2):250-279.

13. Hackman J, Oldham G. Work Re-design. Reading, MA: Addison-Wesley; 1980.

14. Aiken L, Havens D, Sloane D. The Magnet Nursing Services Recognition Program: a comparison of 2 groups of magnet hospitals. Am J Nurs. 2000;100(3):26-36.

15. Aiken L, Sochalski J, Lake E. Studying outcomes of organizational change in health services. Med Care. 1997;35(11 suppl):NS6-NS18.

16. Shortell S, Rousseau D, Gillies R, Devers K, Simons T. Organizational assessment in intensive care units (ICUs): construct development, reliability and validity of the ICU Nurse-Physician Questionnaire. Med Care. 1991;29:709-723.

17. Cudeck R, Brown M. Cross-validation of covariance structures. Multivariate Behav Res. 1983;18:147-167.

18. Fry S, Duffy M. Development & psychometric evaluation of the Ethical Issues Scale. J Nurs Scholarsh. 2001;33(3):273-277.

19. Li F, Harmer P, Duncan T, Duncan S, Acock A, Yamamoto T. Confirmatory factor analysis of the Task and Ego Orientation in Sports Questionnaire with cross-validation. Res Q Exerc Sport. 1998;69(3):276-283.

20. Tabachnick B, Fidell L. Using Multivariate Statistics. 5th ed. New York, NY: Harper Collins College Publishers; 2007.

21. Comrey A. Factor analytic methods of scale development in personality and clinical psychology. J Consult Clin Psychol. 1988;56:754-761.

22. Buchanan T, Ali T, Heffernan T, et al. Nonequivalence of on-line and paper-and-pencil psychological tests: the case of the Prospective Memory Questionnaire. Behav Res Methods. 2005;37:148-154.

23. American Nurses Credentialing Center. Announcing a new model for ANCC's Magnet Recognition Program, 2008. Available at http://cms.nursecredentialing.org/Magnet/NewMagnetModel.aspx. Accessed November 10, 2008.

24. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.

Cited By:

This article has been cited 1 time(s).

Journal of Nursing Scholarship
Validation of the Professional Practice Environment Scale in Australian General Practice
Halcomb, EJ; Davidson, PM; Caldwell, B; Salamonson, Y; Rolley, JX
Journal of Nursing Scholarship, 42(2): 207-213.
10.1111/j.1547-5069.2010.01349.x
CrossRef
Back to Top | Article Outline

© 2009 Lippincott Williams & Wilkins, Inc.

 

Login