The Psychometric Properties and the Development of the Indicators of Quality Nursing Work Environments in Taiwan

Abstract

Background: 

The nursing shortage in medical institutions in Taiwan averaged 9% in 2012, considerably higher than the 5% indicated in the literature. As a result, many hospitals have been forced to close wards or reduce beds. Despite the acute need, the percentage of registered nurses who are employed as nurses in Taiwan (60.4%) is considerably lower than those in Canada or the United States. This low rate may be because of the poor working environment for nurses in Taiwan.

Purpose: 

This study aimed to develop a set of nursing work environment quality indicators for Taiwan and to test the reliability and validity of the resulting survey tool.

Methods: 

Multiple methods were used in this study. In Phase 1, we organized an expert panel, reviewed the literature, and conducted seven rounds of expert panel discussion and six focus group discussions with nursing directors. The goal was to draft indicators representing a quality nursing work environment to fit current conditions in Taiwan. In Phase 2, we conducted an expert review for content validity, held three public hearings, and conducted a survey. Four hundred twenty-seven questionnaires were sent out, with 381 returned. The goal was to test the content validity, construct validity, and internal consistency reliability.

Results: 

The study produced a set of indicators of a quality nursing work environment with eight dimensions and 65 items. The content validity index for importance and suitability dimensions were 1.0, whereas the internal consistency was 0.91. The eight dimensions were safe practice environment (16 items), quality and quantity of staff (four items), salary and welfare (seven items), professional specialization and teamwork (seven items), work simplification (five items), informatics (five items), career development (nine items), and support and caring (12 items). The overall load for the indicators was 77.57%.

Conclusions/Implications for Practice: 

The developed indicators may be used to evaluate the quality of nursing work environments. Furthermore, the indicators may be used in hospital surveys to establish baseline conditions and for outcome research that measures improvement in nursing work environments after interventions.

Introduction

Taiwan is facing a nursing shortage tsunami. The average deficit in nursing staff among medical institutions was 8.42%, 9.65%, and 9.00% in 2010, 2011, and 2012, respectively (^{Ministry of Health and Welfare, 2013}). These nursing shortage rates are much higher than the 5% that was reported in the literature (^{Chiang & Lin, 2009; Liou & Cheng, 2009}). Because of the staff shortages and work overload, there have been waves of departures from the nursing practice. Since 2012, many hospitals have been forced to close wards or reduce numbers of beds as a result (^{Lin, Huang, Kao, & Lu, 2013}). According to an estimate by the ^{Taiwan Department of Health (2012)}, there will be a shortfall of 8000 nurses (about 6% of nurses needed) in the next 5 years if the work environment for nurses does not improve.

Nurses in Taiwan typically take care of 7–12 patients during the day shift in acute general wards (^{Lin et al., 2013}). This number increases during evening and night shifts. In addition to the workload issues, the work schedule is deemed unfavorable for many nurses and their families because of the shift rotation and holiday duty requirements. Nurses in Taiwan may need to spend much more times to work with family members. Moreover, persons with nursing training are in demand as airline flight attendants, health insurance assessors, and other careers that offer competitive payments and benefits or stable and flexible work schedules. For example, about 120 graduated nurses currently work for one Taiwanese airline, accounting for about 4% of that airline’s flight attendants.

A further indicator of the unattractive work environment for nurses in Taiwan is the low percentage of registered nurses who actually work as nurses. In Taiwan, only 60.4% of registered nurses are employed as nurses (^{The National Union of Nurses’ Association, 2013}), which is much lower than those in Canada (86.6%; ^{Canadian Federation of Nurses Unions, 2012}) and the United States (84%; ^{Juraschek, Zhang, Ranganathan, & Lin, 2012}).

Nursing shortages have many causes. Although, in many countries, nursing shortages are because of insufficient nursing manpower development (^{MacKusick & Minick, 2010; Ritter, 2011; Rudel, Moulton, & Arneson, 2009}), in Taiwan, the shortage is largely because of poor working conditions, which deter nurses from pursuing a nursing career (^{Chang, Lu, & Lin, 2010; Lin et al., 2013}). These poor working conditions are exacerbated by insufficient manpower allocations, the high patient-to-nurse ratio, and long working hours (^{Lin, Huang, & Lu, 2013}). To cope with this shortage, the national Department of Health promotes a program consisting of 10 reforms (^{Taiwan Department of Health, 2012}). These include setting up a committee within the Taiwan Nursing Association to study the problem and then to design Taiwan-specific indicators of a quality nursing work environment.

A literature review identified more than 40 studies published since 2000 that explore nursing-work-environment-related issues. These studies were conducted in the United States, Canada, Spain, Brazil, Korea, Belgium, Taiwan, and Australia (^{Aiken & Patrician, 2000}; Chiang & Lin, ^{2009; Cho, Mark, Yun, & June, 2011}; Cohen, Stuenkel, & Nguyen, ^{2009; Cummings, Hayduk, & Estabrooks, 2006; De Pedro-Gómez et al., 2012}; Estabrooks et al., ^{2002; Gasparino, Guirardello Ede, & Aiken, 2011; Havens, Warshawsky, & Vasey, 2012; Lake & Friese, 2006; Li et al., 2007; Liou & Cheng, 2009; Lu, Lin, Chen, Chang, & Kao, 2008; Parker, Tuckett, Eley, & Hegney, 2010; Walker, Middleton, Rolley, & Duff, 2010; Warshawsky & Havens, 2011}). The studies evaluated autonomy, control of nursing practice, culture, education, equipment, innovation, leadership, nurse management ability, nurse–physician relations, organizational support, pay, peer cohesion, respect, safety, scheduling, supplies, and resources (^{Aiken & Patrician, 2000; Cho et al., 2011; Cohen et al., 2009; Kramer & Schmalenberg, 2004; Lake, 2002; Schmalenberg & Kramer, 2008; Ulrich, Buerhaus, Donelan, Norman, & Dittus, 2005; Walker et al., 2010}). However, because of the differences in culture and healthcare systems, nurses in different countries may differ in their conception of what constitutes a quality work environment for nurses (Chiang & Lin, ^{2009; De Pedro-Gómez et al., 2012}; Estabrooks et al., ^{2002; Gasparino et al., 2011; Hanrahan, 2007; Lake & Friese, 2006; Liou & Cheng, 2009}; Parker et al., ^{2010; Slater & McCormack, 2007; Van Bogaert, Clarke, Vermeyen, Meulemans, & Van de Heyning, 2009; Warshawsky & Havens, 2011}). Thus, the purpose of this study is to develop a set of indicators that represent a quality nursing work environment within the context of the current Taiwan healthcare system.

In 2007, the International Council of Nurses proposed that a positive environment for practicing nursing may lead to higher quality patient care and may facilitate the development of a tool for measuring the quality of nursing work environments. The tool includes five domains with 42 items. The five domains were organization (16 items), nurses (eight items), government (seven items), national nursing association (six items), and regulatory bodies (five items; ^{Baumann, 2007}).

The Taiwan Department of Health invited experts to brainstorm on issues related to the nursing shortage and to develop an appropriate retention plan. This plan addressed eight domains (new staff training and coaching, work arrangements, manpower allocation, nursing work environment, staff caring, leadership, salary and welfare, and professional growth and development), 24 strategies, and 70 items (^{Lu et al., 2008}). Moreover, Chen, Wu, Song, and Lin (²⁰¹⁰) proposed a list of characteristics that define a quality nursing work environment. This list includes six domains: safe work environment, adequate manpower allocation and suitable work assignments, staff support and caring, excellent leadership, sufficient salary and welfare, and adequate professional development training and career planning (^{Chen et al., 2010}). During the process of establishing Magnet hospital indicators, the American Academy of Nursing nominated hospitals as potential Magnet medical institutes and then surveyed nursing administrators and nursing staff from these hospitals via questionnaires and group interviews. The survey resulted in a set of Magnet characteristics (^{McClure, Poulin, Sovie, & Wandelt, 2010}). We referred to the above studies in planning the processes and methods to be used for developing indicators of a quality work environment for nurses in Taiwan.

Methods

The purpose of this research, as set by the Taiwan Department of Health, was to develop a set of indicators for quality work environments for nurses to provide a reference for national policy in Taiwan. We used multiple methods to approach the problem. The process involved two phases and six steps, and we used a criterion-referenced framework to guide the design and interpretation of measurements. The end result was the Taiwan version of the indicators of quality nursing work environment (IQN-WE).

Phase 1: Development of the New Indicators of Quality Nursing Work Environment

In Phase 1, we assembled a panel of experts, reviewed the literature, and conducted seven rounds of expert panel discussions and six focus group discussions with nursing directors.

Stage 1: develop a new framework and items for measuring indicators of quality nursing work environment

One of the authors (M. S. L.) assembled an expert panel by inviting nursing administrators who had authority to make policy changes in their hospitals or organizations to participate. The resulting panel of seven experts included the president of the National Union of Nurses’ Associations, ROC; the president of the Taiwan Nursing Management Association; four nursing directors who worked at hospitals that had received awards in 2011 for quality nursing work environments; and two hospital accreditation auditors. These experts reviewed the domestic and international literature and participated in seven rounds of panel discussions. The experts agreed that the indicators should be distinct from existing hospital accreditation indicators and that evaluation should minimize any additional work burden and paperwork for nurses. The panel recommended that quality nursing work environments should include the following characteristics: a safe work environment, quality staffing, adequate salaries and welfare, professional collaboration and teamwork, work simplification and informatics, professional development, and a supportive and caring atmosphere. Seven domains and 74 items resulted from this phase.

Stage 2: revise the framework and items of the indicators of quality nursing work environment

We invited 36 nursing administrators from Magnet hospitals that received awards from the Department of Health in 2011 to participate in focus group discussions. Six group discussion sections were conducted. Each section was composed of 10 administrators, and each administrator participated in two of the six sections. The focus groups discussed and revised the content of the indicators. The final decisions produced 65 items in seven domains.

Phase 2: Validity and Reliability

This phase conducted tests for face validity, content validity, construct validity, and reliability.

Stage 1: face validity

During public hearings in northern, central, and southern Taiwan, nursing administrators and nursing staff were invited to review the quality indicators and provide opinions or suggestions. Four hundred twenty-seven administrators and nurses (n = 171 from the north, n = 142 from central Taiwan, n = 114 from the south) from 17 academic medical centers, 42 regional hospitals, and 51 local community hospitals participated. A written questionnaire was also distributed during the public hearings to evaluate the suitability of these indicators. The participants were informed that they could refuse to respond without penalty. In addition, there was no identifiable information attached to their questionnaire to ensure anonymity. The participants rated each item for suitability using a 4-point Likert scale (4 = very suitable, 3 = suitable, 2 = unsuitable, 1 = very unsuitable). Three hundred eighty-one (89.2%) valid questionnaires were returned. Suitability ranged from 2.96 to 3.61.

Stage 2: content validity

To assess content validity, we invited the presidents of hospitals that had received Department of Health awards for their quality nursing environments in 2011 to participate in the discussion and revision process. Ten hospital presidents participated in the review of the contents of the seven indicator domains. In addition, 12 nursing experts from academic medical centers (n = 3), regional hospitals (n = 5), and local community hospitals (n = 4) participated in the content validity evaluation. These experts rated the importance and suitability of each item. After the panel discussion, the researchers modified the literal expressions and content based on the opinions and suggestions of the experts to complete the revised instrument. The content validity index was used to evaluate expert content validity with the level of >80% used as the testing standard. The expert scoring results were collected. The content validity indices for the importance and suitability dimensions were both 1.0.

Stage 3: construct validity

We applied the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy to evaluate sample suitability before performing the Bartlett’s test of sphericity. Measuring system analysis (MSA) was used to evaluate each item for its suitability for factor analysis. We used exploratory factor analysis (EFA) for factor extraction. Principal component analysis (PCA) was used to find the initial factor solution and factor rotation and to check for varimax. Factors were considered significant and were retained if their eigenvalue was 1 or greater. We retained items with factor loadings greater than 0.5. Definitions were created for each after factors and items had been constructed (Table 1).

TABLE 4: Definition of the Eight Domains

Stage 4: reliability

The Cronbach’s alpha reliability of the IQN-WE was evaluated by computing the internal consistency of each of the eight empirically derived subscales. Furthermore, the intraclass correlation was used to test for internal consistency among the eight subscales.

Results

To ensure the suitability of the developed IQN-WE Index for measuring the nursing work environment, we recruited nurses to complete the tool and applied factor analysis to the responses provided.

Study Subjects

Three hundred eighty-one nurses participated in this study. The most frequent type of workplace evaluated was teaching hospitals (n = 92, 24.1%; Table 2). Specialized hospitals were the least frequent because of the relatively small number of this type in Taiwan. Northern Taiwan (n = 150, 39.4%) was the most heavily represented region, whereas thinly populated eastern Taiwan had the fewest participants. In addition, no public hearings were conducted in eastern Taiwan, which likely further reduced participation from that area. Most of the participants were staff nurses (n = 249, 65.4%) who worked in hospital wards (n = 162, 42.5%). Most held university degrees (n = 148, 38.8%) and were married (n = 236, 61.9%). Current hospital tenure was defined in 3-year intervals. Most had worked at their current hospitals for at least 12 years (n = 123, 32.3%). In terms of total nursing work years, most had 6–9 years of experience (n = 76, 19.9%).

TABLE 2: Characteristics of Nurse Participants (N = 381)

Reliability

The reliability of the IQN-WE was evaluated by computing the internal consistency of each of the eight empirically derived subscales. Alpha coefficients for the subscales of staff practice environment (.65), staff quality (.65), salary and welfare (.74), professional specialization and team collaboration (.75), task simplification (.83), informatics (.84), staff training and professional development (.72), and support and care (.75) were all within acceptable limits. The intraclass correlation was also used to test for internal consistency among the eight subscales. On the basis of the 65 items of the questionnaire, the intraclass correlation of the eight subscales was .60.

Validity

The Kaiser criterion proposes that questions composed of factors should have an average communality of ≥ 0.70 (^{Stevens, 1992}) and that a KMO of > 0.80 indicates a “good” relationship among variables (^{Spicer, 2005}). The KMO measure of sampling adequacy was 0.97, indicating that the data set met the requirements for the Bartlett’s test of sphericity. We used MSA to evaluate each item for suitability for factor analysis. As all items had an MSA greater than 0.7, all were included in the analysis. Bartlett’s test of sphericity yielded a p value of .001, indicating intercorrelation of the items and suitability for factor analysis.

We conducted EFA to analyze total variance without a prespecified factor. The results yielded eight initial eigenvalues greater than 1.0. Eight domains were identified with 77.57% of the variance explained (Table 3). PCA was used with varimax rotation (Kaiser normalization) to generate the rotated component matrix (Table 4). This resulted in an instrument where all items loaded at > 0.30 on the PCA, suggesting that the instrument measures the underlying construct. When all items are loaded on a principal component, the scores of the four factors may also be combined, thus providing an overall score. ^{Fabrigar, Wegener, MacCallum, and Strahan (1999)} pointed out that there should be at least three to five measurable items for each common factor of a factor analysis (^{Fabrigar et al., 1999}). The eight factors (domains) were labeled (a) safe practice environment, (b) staff quality, (c) salary and welfare, (d) professional specialty and team collaboration, (e) task simplification, (f) informatics support, (g) professional training and career development, and (h) support and care.

TABLE 3: Total Variance Explained

TABLE 4: Factor Analysis of PPE Scale Using Principal Component Analysis Through Varimax Rotation

The concept of welfare may differ between Taiwanese nurses and nurses in other countries. Nurses in Taiwan may consider better manpower allocation and higher nurse-to-patient ratios as benefits because they are accustomed to nursing shortages in the workplace. Two items originally assigned to the staff quality domain were reclassified into the salary and welfare domain. These two items were (a) adequate nurse-to-patient ratio in each work shift (day, evening, and night) and (b) mechanisms for adjusting staff allocation (including licensed practice nurses, nursing assistants, and care service personnel) to maintain appropriate workloads. Because increases in the nurse-to-patient ratio and manpower allocation were perceived as welfare, we accepted the reclassification of these two items into the domain of salary and welfare. Work simplification and informatics support were originally combined in one domain. However, the results of EFA indicated that these were two distinct factors, each with its own set of items.

The final version of the IQN-WE included eight domains and 65 items, which were scored using a 4-point Likert scale from 1 (very unsuitable) to 4 (very suitable). The average score was 3.49 (SD = 0.675). Domain 1, safe practice environment, includes 16 items (M = 3.53, SD = 0.614) such as health and safety management, environmental hazard prevention, ergonomic risk control, and violence and sexual harassment prevention (Table 5). Domain 2, staff quality, includes four items (M = 3.21, SD = 0.779): licenses and certifications, level in nursing clinical ladder, years of experience, and academic degrees. Domain 3, salaries and welfare, includes seven items (M = 3.48, SD = 0.487) such as salary, nightshift fees, nurse-to-patient ratio, and welfare programs. Domain 4, specialization and teamwork, includes seven items (M = 3.47, SD = 0.673): decision making, execution, supervision, management of professional specialization, mutual respect, good communication, and collaboration among team members. Domain 5, task simplification, includes five items (M = 3.46, SD = 0.706): initiation of work simplification, implementation of work simplification, incentives to facilitate and implement work simplification, outcomes of work simplification efforts, and cross-team support and assistance in work simplification. Domain 6, informatics support, includes five items (M = 3.53, SD = 0.674) such as hardware, security maintenance, clinical informatics, and management. Domain 7, staff training and professional development, includes (M = 3.47, SD = 0.659) directions, strategies, resources, evaluation, outcomes of staff training, multiple role functions, professional development and innovation, and professional recognition by others. Domain 8, support and care, includes 12 items (M = 3.56, SD = 0.655) such as management mode and support, empowerment in management, flexible work schedules, responsiveness to staff work and life needs, and activities and interventions to promote stress relief.

TABLE 5: Quality Nursing Work Environment Indicators

Discussion

Tool Development Process

This study was led by the President of the National Union of Nurses’ Associations, who also serves as the President of the Taiwan National Union of Nurses’ Associations. The process of completing the IQN-WE included assembling an expert panel, conducting seven rounds of expert panel discussions, guiding six focus group discussions, and hosting three public hearings with staff nurses and then distributing questionnaires to them. This process differs from the group discussions and open-ended questionnaires used by the American Academy of Nursing to develop Magnet hospital indicators (^{McClure et al., 2010}). Because all of the hospitals in Taiwan are understaffed, no domestic hospital is suitable as a reference for constructing nursing work environment quality indicators.

The process we used also differed from that used by a Canadian study, which sent questionnaires to 17,965 nurses in 415 hospitals (^{Estabrooks et al., 2002}). Because the concept of a quality nursing work environment is still unfamiliar to nurses in Taiwan, we conducted public hearings before distributing the questionnaire survey.

Psychometric Properties of the Indicators of Quality Nursing Work Environment

Similar to other studies (^{Chiang & Lin, 2009; Estabrooks et al., 2002; Lake, 2002; Parker et al., 2010}), we evaluated the IQN-WE for face validity, content validity, construct validity, and internal consistency reliability. We analyzed the structure of the IQN-WE questionnaire using PCA with iteration followed by varimax rotation and Kaiser normalization for the factor analysis. The final version of the IQN-WE included eight domains and 65 items, with 77.57% of the total variance explained. Its suitability scores ranged from 2.96 to 3.61 (M = 3.49). The tool has the greatest number of items for evaluating the nursing practice environment of any survey instrument to date and has the highest total variance explained as well as the highest suitability score. ^{Lake (2002)} developed a 57-item nursing work index (NWI), the Practice Environment Scale, with an overall loading of 59%. The suitability of the items ranged from 2.48 to 3.17. ^{Aiken and Patrician (2000)} also reported a revised NWI for measuring the characteristics of nursing practice environments. The revised NWI consists of four domains with 15 items. The alpha coefficient between .65 and .84 indicates the good internal consistency of the individual subscales. The overall internal consistency indicated by intraclass correlation (r = .60) revealed that each of the subscales addresses a distinct dimension of the IQN-WE. Although researchers would prefer a shorter instrument, maintaining adequate loading with fewer items remains a challenge. For the newly developed IQN-WE, future studies may focus on reducing the number of items.

Differences in Personal Opinions

Results for suitability under various domains may differ by type of hospital. For the domain of safe practice environment, the teaching hospitals received higher scores than the regional hospitals or the local community hospitals. For the domain of staff quality, there were also statistically significant differences, with the teaching hospitals receiving the highest scores, whereas the local community hospitals had the lowest scores for suitability. These findings indicate that the staff quality is better at teaching hospitals than at the other two types of hospitals. These results may explain why the suitability score for Item 4, “percentage of nurse managers with masters’ degrees,” averaged only 2.96. This item was less applicable to local community hospitals where resources are limited.

Job position may also affect the ratings of the indicators. In every domain, nurse managers gave scores that were higher than those given by staff nurses, indicating that nurse managers or administrators had different views on indicator suitability. These findings are similar to those of ^{Lee, Pai, and Yen (2008)}, which showed wide differences in perceptions of the nursing practice environment between nurse administrators and staff nurses.

For work units, we found that all but the “salaries and welfare” domain differed significantly, with higher scores for those working in the nursing departments than in the other units. This is likely because most nurses working in the nursing departments were at least supervisors in rank, and their perceptions of the suitability of the indicators were higher than those working in clinical wards. This finding was consistent with the finding for job position.

There were also different perceptions of the suitability of the indicators among nurses with different educational levels. We found that nurses with higher education (university or master degrees) rated the suitability higher than those with college or vocational school training. This finding was similar to that of a previous study by ^{Lee et al. (2008)}, who found that nurses with different educational backgrounds had different perceptions of their work environment, especially regarding safe practices and salary and welfare. Nurses with a graduate education degree tended to rate the suitability of indicators higher than those with either college degrees or vocational school training.

Nurses’ tenure in their current workplace likewise affected how they rated the indicators across all domains. Nurses with different tenure statuses rated the suitability of all indicators differently with the exception of the salary and welfare domain and the professional specialization and teamwork domains. Furthermore, nurses with longer work tenure rated the suitability indicators higher.

Study Limitations

Although we used focus groups to elicit expert opinions, we did not apply the Delphi technique to collect data. This may limit the depth of the data collected from the focus group. However, we engaged 36 nursing administrators and conducted six group discussions to ensure the width of the issues covered.

Conclusions

The reliability and validity of the IQN-WE support the use of this instrument in both clinical and research practice settings in ethnic Chinese populations. The IQN-WE provides a unique set of subscales (safe practice environment, staff quality, salary and welfare, professional specialty and team collaboration, task simplification, informatics support, professional training and career development, and support and care) that are designed to assess the quality of nursing work environments. This instrument shows adequate internal consistency reliability, moderate intraclass correlation across subscales, appropriate content validity, and strong construct validity.

The positive nursing work environment assessment tool proposed in 2007 by the International Council of Nurses for adoption by its member states was designated by the Taiwan Department of Health as the basis for developing indicators to evaluate the quality of nursing work environments as part of the national hospital accreditation process. Although the IQN-WE was constructed using a robust development process, time and budget constraints led the authors to focus on the more densely populated areas of northern, southern, and central Taiwan. Public hearings were not held in more sparsely populated eastern Taiwan. In addition, all participating nurses were assigned by their hospitals or institutions to represent their workplaces. In future studies, participants should be recruited from a more diverse population and include adequate representation from hospitals in eastern Taiwan.

Acknowledgments

This study was funded by the Department of Health, Executive Yuan of Taiwan, ROC (DOH101-TD-M-113-101009). The authors thank Liang-Hua Huang, Yu-Tzu Chen, Ju-Hua Kao, Li-Chu Hsu, Chung-Yi Huang, and Fu-Chi Lai for their participation on the expert panel and thank the 36 nurse administrators, 10 hospital presidents and vice presidents, and 10 experts who performed the validity testing for this study. The authors are grateful for the time and efforts made by the participating nurses and nursing managers in the public hearings and survey.