Effect of Ergonomics Training on Agreement Between Expert and Nonexpert Ratings of the Potential for Musculoskeletal Harm in Manufacturing Tasks

Fethke, Nathan B. PhD; Merlino, Linda MS; Gerr, Fred MD

Journal of Occupational & Environmental Medicine:
doi: 10.1097/JOM.0000000000000038

Objective: To evaluate the effect of ergonomics training on nonergonomists' ability to recognize and characterize the potential for musculoskeletal harm in manufacturing tasks.

Methods: Ergonomics training was delivered to members of a participatory ergonomics team in a manufacturing facility. Before and after training, participatory ergonomics team members and the research team rated the potential for musculoskeletal harm for each of 30 tasks. Measures of agreement included Pearson, concordance, and intraclass correlation coefficients.

Results: Measures of agreement generally improved after training. The greatest agreement was observed for ratings of the potential for musculoskeletal harm to the low back. The greatest improvement in agreement was observed for ratings of the potential for musculoskeletal harm to the neck/shoulder.

Conclusions: The training seemed to improve nonexperts' ability to identify the potential for musculoskeletal harm.

Author Information

From the Department of Occupational and Environmental Health, College of Public Health, the University of Iowa, Iowa City, Iowa.

Address correspondence to: Nathan B. Fethke, PhD, Department of Occupational and Environmental Health, College of Public Health, the University of Iowa, 105 River Street, CPHB S347, Iowa City, IA 52242 (nathan-fethke@uiowa.edu).

This research was supported by a pilot project grant from the Healthier Workforce Center for Excellence (HWCE) at the University of Iowa. The HWCE is supported by Cooperative Agreement No. U19OH008858 from the Centers for Disease Control and Prevention (CDC)/National Institute for Occupational Safety and Health (NIOSH). The contents are solely the responsibility of the authors and do not necessarily represent the official views of the CDC, NIOSH, or the HWCE.

The authors declare no conflicts of interest.

Article Outline

Occupational exposure to physical risk factors, such as forceful muscular exertions, awkward postures, and highly repetitive activities, has been associated with increased risk of work-related musculoskeletal disorders.1–3 Many employers have adopted participatory ergonomics (PE) methods to guide efforts to control exposure to physical risk factors. The hallmark of PE is the meaningful contributions of workers in both the identification/analysis of risk factors and the development of controls.4 Worker participation capitalizes on their knowledge and experience, and may promote acceptance of workplace changes.5

Reported benefits of PE interventions include reductions in musculoskeletal symptom prevalence,6–8 musculoskeletal disorder claims rates and claims costs,7,9–12 sick leave and absenteeism,7,9,10,13 and exposure to physical risk factors.6,14,15 The PE framework has also been suggested as a viable model for integrating workplace health protection activities with workplace health promotion activities, a core concept of the Total Worker Health program of the National Institute for Occupational Safety and Health. In particular, the scope of PE (which has typically focused on physical aspects of the work environment) can be broadened to also address psychosocial and organizational factors that influence worker health and well-being.16,17

Despite considerable acceptance of the PE approach and applicability of PE to the Total Worker Health paradigm, only one previous study was identified that empirically examined the ability of nonergonomists to learn and apply newly acquired knowledge and skills within a PE framework.18 Although the results suggested that ergonomics training could lead to improved working conditions, the participants were college students without industrial experience. Because some members of a PE team in a real-world setting are experienced workers intimately familiar with industrial processes, some inherent baseline understanding of ergonomics can be expected, even if only informal or anecdotal. To better characterize the value of PE, the specific objective of this study was to evaluate the effect of ergonomics training (delivered as a component of a PE intervention) on nonergonomists' ability to characterize the potential for musculoskeletal harm in manufacturing tasks.

Back to Top | Article Outline


We implemented a PE intervention at a manufacturing facility in Iowa. The facility manufactures vinyl-sided window assemblies for residential construction applications. The facility employs 250 to 400 production workers, depending on seasonal variation in product demand. Most workers perform cyclic, light assembly tasks (mean cycle time ∼65 seconds) involving manual manipulation of parts, use of powered and nonpowered hand tools, and some lifting. The Institutional Review Board at the University of Iowa approved all study procedures.

Back to Top | Article Outline
Description of the Training Program

The training component of the PE intervention included two distinct activities: (1) ergonomics process training, and (2) support meetings. The purpose of the ergonomics process training was to provide relevant, practical information on how to create an ergonomics process within their organizational structure. Content included (a) didactic instruction in musculoskeletal anatomy, physical risk factors, dimensions of exposure, and exposure–effect relationships; (b) instruction in the use of formal exposure assessment instruments (eg, the Strain Index,19 the Rapid Entire Body Assessment,20 and the NIOSH Lifting Equation21); (c) hands-on, team-based assessments of tasks performed at the facility; (d) discussion of ergonomics process implementation, with the goal of developing the framework of a strategic plan; (e) examples of the development, implementation, and evaluation of controls; and (f) cost–benefit analyses. The ergonomics process training was delivered by a Certified Professional Ergonomist (NF) over two one-half day workshops.

The purpose of the support meetings was to reinforce training; refine the ergonomics process implementation plan; prioritize development and implementation of controls; discuss control options with PE team members, management, and affected workers; and discuss issues related to workplace ergonomics. Research team members met with the PE team for 2 hours once per month for 1 year after the ergonomics process training.

Back to Top | Article Outline
Composition of the PE Team

The PE team included the facility's safety manager, two additional safety personnel, the production manager, the human resources manager, a representative from maintenance, and three production employees (n = 9 from the facility). The general manager served as an ex-officio member of the PE team, but did not contribute data to the current analyses.

Back to Top | Article Outline
Study Procedures

Our evaluation of training effectiveness was based on pre- and posttraining agreement between the research team's consensus rating and the PE team's median rating of the potential for musculoskeletal harm associated with specific production tasks. Furthermore, we evaluated pre- and posttraining interrater agreement between the PE team members' ratings.

Before the ergonomics process training, we randomly selected 30 cyclic production tasks and obtained representative 10-minute video recordings of each task (or a minimum of five cycles). For each task, a separate worker was filmed and recordings were obtained simultaneously of the frontal and sagittal planes.

During a meeting convened 1 week before the ergonomics process training, each PE team member viewed each task video and provided his/her ratings of the potential for low back, neck/shoulder, elbow, and hand/wrist musculoskeletal harm on 10-cm visual analog scales (VAS) provided by the investigators. For the upper extremity, ratings were made only for the body side clearly within the sagittal plane camera field of view. Employees at the facility referred to the potential for musculoskeletal harm as an “ergonomic hazard.” Therefore, we used descriptive anchors on the VAS to reflect such informal terminology. A VAS rating of 0 cm was used to indicate “no ergonomic hazard” and a VAS rating of 10 cm was used to indicate a “very harmful ergonomic hazard.” The PE team members were instructed to complete the scales independently and not to communicate while the task videos were played. No identifying information (eg, PE team member name or job title) was collected with the VAS rating. Before this meeting, the research team viewed the same video recordings and rated (by consensus) each of the 30 tasks using identical scales.

After the ergonomics process training and 1 year of monthly support meetings, and with procedures identical to those used just before the ergonomics process training, the research team and the PE team again completed VAS ratings for a second set of 30 randomly selected cyclic production tasks.

Back to Top | Article Outline
Statistical Analyses

During the year after the ergonomics process training, four of the original nine PE team members left the facility because of reassignment or termination. Therefore, the posttraining VAS ratings completed by the five remaining members of the original PE team were used for the current analyses.

For each task and body region, we computed the median of the PE team members' VAS ratings. We then calculated, for each body region separately, Pearson correlation coefficients between the PE team's median VAS rating and the research team's consensus VAS rating for the set of 30 task videos obtained before the ergonomics process training (rpre). Similarly, we calculated Pearson correlation coefficients between the PE team's median VAS rating and the research team's consensus VAS rating for the set of 30 task videos obtained after the ergonomics process training and 1 year of support meetings (rpost). The one-sample t test for a correlation coefficient was used to test the null hypotheses that rpre = 0 and rpost = 0. Fisher's z-transformation was used to estimate 95% confidence intervals for the Pearson correlation coefficients.22 We also used Fisher's z-test for comparing two correlation coefficients to test the null hypothesis that rpre = rpost. Because we expected training to improve agreement in VAS ratings between the PE team and the research team, this test was one sided (ie, the alternative hypothesis was rpost > rpre).

We also estimated the pre- and posttraining concordance between the PE team's median VAS ratings and the research team's consensus VAS ratings by computing the concordance correlation coefficient (Pc).23 In contrast to the Pearson correlation coefficient, Pc incorporates corrections for shifts of the linear relationship away from the ideal model (ie, least-squares linear regression slope = 1.0, and offset = 0.0). Methods described in Lin23 were used to estimate 95% confidence intervals for the concordance correlation coefficients (Pc-pre and Pc-post). Fisher's z-test for comparing two correlation coefficients was used to test the null hypothesis that Pc-pre = Pc-post. As above, this test was one sided and separate analyses were performed for each body region.

Finally, the intraclass correlation coefficient (ICC; two-way, random effects model with absolute agreement) was used to estimate the pre- and posttraining agreement in the VAS ratings among PE team members. Confidence limits and tests of significance (null hypothesis: ICC = 0) for the pre- and posttraining ICC estimates were calculated.24 Because (1) posttraining VAS ratings were available for only five of the original nine PE team members and (2) we did not collect identifying information with the VAS, we examined the possibility that a difference between the pre- and posttraining ICCs was an artifact of the five remaining PE team members and not a training effect. Specifically, in addition to the pretraining ICC for all nine original PE team members, we estimated the distribution (mean, standard deviation) of the pretraining ICC for all possible combinations of five original PE team members.

Statistical procedures were performed using Microsoft Excel (version 2010, Microsoft Co, Redmond, WA) and SPSS (version 21, IBM Co, Armonk, NY).

Back to Top | Article Outline


In general, measures of agreement between the PE team's median VAS ratings and the research team's consensus VAS ratings were improved after the ergonomics process training and 1 year of support meetings (Table 1). The largest improvements were observed for the neck/shoulder region (rpre = 0.13 vs rpost = 0.46; Pc-pre = 0.07 vs Pc-post = 0.36). Nevertheless, no posttraining agreement value was statistically significantly different from its corresponding pretraining agreement value.

For the low back, a small decrease was observed for the post-training Pearson correlation compared with the pretraining Pearson correlation, whereas a small increase was observed for the posttraining concordance correlation compared with the pretraining concordance correlation. In this case, the improvement in the posttraining concordance correlation was the result of a reduced offset (ie, smaller intercept) of the least-squares regression line (Fig. 1).

The ICCs of the VAS ratings among the PE team members also improved after the ergonomics process training and 1 year of support meetings. Before the training, only the ICC of the VAS ratings of the potential for musculoskeletal harm to the low back was significantly greater than zero. After training, all ICC estimates were significantly greater than zero. Inspection of Table 1 shows that for the elbows, the 95% confidence intervals around the pre and posttraining ICCs did not overlap, suggesting an improvement not likely because of chance.

The distributions (mean, standard deviation) of the pretraining ICCs for all possible combinations of five PE team members were, 0.16 (0.06) for the low back, 0.04 (0.07) for the neck/shoulder, −0.06 (.06) for the elbow, and 0.02 (0.08) for the hand/wrist. For all body areas except the low back, the estimate of the posttraining ICCs from the five remaining original PE team members exceeded the mean of the distribution of pretraining ICC estimates for all possible combinations of five PE team members by more than one standard deviation.

Back to Top | Article Outline


Considerable methodological heterogeneity is apparent in available literature describing the delivery and evaluation of ergonomics training, in general, and PE interventions, in particular.25,26 Several studies report evidence of training effectiveness as improvements of scores on tests of knowledge about physical risk factors, the design of workspaces using ergonomics principles, and other ergonomics-related constructs.27–29 In contrast, we evaluated a PE team's ability to characterize by observation the potential for musculoskeletal harm, using a process on the basis of a conceptual understanding of ergonomics rather than the rote application of any particular formal exposure assessment instrument. In general, the agreement in VAS ratings of the potential for musculoskeletal harm to the low back, neck/shoulder, elbows, and hand/wrist improved after training activities, although the observed effects were modest in size.

The agreement (Pearson and concordance) between the PE team's median VAS ratings and the research team's consensus VAS ratings was highest for the low back for both the pre and posttraining analyses. Because we did not instruct PE team members to focus on physical risk factors (eg, posture, force, and repetition) when completing the VAS ratings, we are unable to evaluate specific drivers of the observed results. Nevertheless, discussion of the results with the PE team suggested several circumstances unique to the facility that may have contributed to this result. Specifically, many of the production tasks involve manual handling of products weighing up to 100 lb and a facility policy requires team lifts of more than 51 lb. Furthermore, employees receive a brief orientation to ergonomics upon hire and complete a 30-minute web-based ergonomics training module annually. The orientation and web-based materials contain substantial information about lifting biomechanics. Therefore, the training may not have increased knowledge about factors associated with low back musculoskeletal outcomes to the same extent as knowledge about factors associated with neck/shoulder, elbow, or hand/wrist musculoskeletal outcomes.

Improvement in the ICCs of VAS ratings suggests that the training was at least partially effective in transferring knowledge to PE team members. Estimates of the ICC depend strongly on the specific model (eg, two-way, random effects vs two-way, mixed effects) and type (absolute agreement vs consistency) selected.30 The ICC model we used treated the PE team members as a random sample of a larger population of similar individuals.

The results of this study should be interpreted cautiously. The ergonomics process training and support meetings seemed to improve the PE team's ability to characterize the potential for musculoskeletal harm over a 1-year time frame. The effectiveness and impact of the PE intervention over a longer period have not been evaluated. The loss of four original PE team members during the year after the ergonomics process training affected our analytical strategy. Nevertheless, negative long-term effects of PE team member turnover have been minimized through adoption of a strategic plan to guide ongoing intervention activities, which includes provisions for maintaining “institutional memory” of ergonomics.

The PE intervention is a component of an ongoing study of the combined effects of PE and workplace health promotion on exposure to physical risk factors, musculoskeletal symptom prevalence, musculoskeletal injury rate, workers' compensation claims costs, health insurance costs, and indicators of chronic disease risk (eg, hypertension, obesity, and cholesterol). The health promotion component uses motivational interviewing to encourage health behavior change and a participatory approach to implement facility-wide wellness activities.

Back to Top | Article Outline

The authors acknowledge Mr Steven Hanson for his contributions to data collection and analysis and Ms Marie Yanacek for her assistance with data collection.

Back to Top | Article Outline


1. Bernard BP. Musculoskeletal Disorders and Workplace Factors: A Critical Review of Epidemiologic Evidence for Work-Related Disorders of the Neck, Upper Extremity, and Low Back. Cincinnati, OH: DHHS (NIOSH) Publication No. 97-141; 1997.
2. National Research Council and Institute of Medicine. Musculoskeletal Disorders and the Workplace. Washington, DC: National Academy Press; 2001.
3. Punnett L, Gold J, Katz JN, Gore R, Wegman DH. Ergonomic stressors and upper extremity musculoskeletal disorders in automobile manufacturing: a one year follow up study. Occup Environ Med. 2004;61:668–674.
4. Haims MC, Carayon P. Theory and practice for the implementation of “in-house”, continuous improvement participatory ergonomic programs. Appl Ergon. 1998;29:461–472.
5. Habes DJ. Participatory Ergonomic Interventions in Meatpacking Plants. Cincinatti, OH: DHHS (NIOSH) Publication No. 94-124; 1994.
6. Vink P, Kompier MA. Improving office work: a participatory ergonomic experiment in a naturalistic setting. Ergonomics. 1997;40:435–449.
7. Evanoff BA, Bohr PC, Wolf LD. Effects of a participatory ergonomics team among hospital orderlies. Am J Ind Med. 1999;35:358–365.
8. Bohr PC. Efficacy of office ergonomics education. J Occup Rehabil. 2000;10:243–255.
9. Lanoie P, Tavenas S. Costs and benefits of preventing workplace accidents: the case of participatory ergonomics. Safety Sci. 1996;24:181–196.
10. Moore JS, Garg A. The effectiveness of participatory ergonomics in the red meat packing industry: evaluation of a corporation. Int J Ind Ergonom. 1998;21:47–58.
11. Moreau M. Corporate ergonomics programme at automobiles Peugeot-Sochaux. Appl Ergon. 2003;34:29–34.
12. Butler MP. Corporate ergonomics programme at Scottish & Newcastle. Appl Ergon. 2003;34:35–38.
13. Joseph BS. Corporate ergonomics programme at Ford Motor company. Appl Ergon. 2003;34:23–28.
14. St Vincent M, Chicoine D, Beaugrand S. Validation of a participatory ergonomic process in two plants in the electrical sector. Int J Ind Ergonom. 1998;21:11–21.
15. Laing AC, Frazer MB, Cole DC, Kerr MS, Wells RP, Norman RW. Study of the effectiveness of a participatory ergonomics intervention in reducing worker pain severity through physical exposure pathways. Ergonomics. 2005;48:150–170.
16. Punnett L, Cherniack M, Henning R, Morse T, Faghri P, CPH-NEW research team. A conceptual framework for integrating workplace health promotion and occupational ergonomics programs. Public Health Rep. 2009;124(suppl 1):16–25.
17. Henning R, Warren N, Robertson M, Faghri P, Cherniack M, CPH-NEW Research Team. Workplace health protection and promotion through participatory ergonomics: an integrated approach. Public Health Rep. 2009;124(suppl 1):26–35.
18. Saleem JJ, Kleiner BM, Nussbaum MA. Empirical evaluation of training and a work analysis tool for participatory ergonomics. Int J Ind Ergonom. 2003:387–396.
19. Moore JS, Garg A. The Strain Index: a proposed method to analyze jobs for risk of distal upper extremity disorders. Am Ind Hyg Assoc J. 1995;56:443–458.
20. Hignett S, McAtamney L. Rapid entire body assessment (REBA). Appl Ergon. 2000;31:201–205.
21. Waters TR, Putz-Anderson V, Garg A, Fine LJ. Revised NIOSH equation for the design and evaluation of manual lifting tasks. Ergonomics. 1993;36:749–776.
22. Rosner BA. Fundamentals of Biostatistics. 7th ed. Boston, MA: Cengage Learning; 2011:506–514.
23. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45:255–268.
24. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.
25. Wijk K, Mathiassen SE. Explicit and implicit theories of change when designing and implementing preventive ergonomics interventions—a systematic literature review. Scand J Work Environ Health. 2011;37:363–375.
26. Rivilis I, Van Eerd D, Cullen K, et al. Effectiveness of participatory ergonomic interventions on health outcomes: a systematic review. Appl Ergon. 2008;39:342–358.
27. Silverstein BA, Richards SE, Alcser K, Schurman S. Evaluation of in-plant ergonomics training. Int J Ind Ergonom. 1991;8:179–193.
28. King PM, Fisher JC, Garg A. Evaluation of the impact of employee ergonomics training in industry. Appl Ergon. 1997;28:249–256.
29. Robertson M, Amick BC, DeRango K, et al. The effects of an office ergonomics training and chair intervention on worker knowledge, behavior and musculoskeletal risk. Appl Ergon. 2009;40:124–135.
30. Lee KM, Lee J, Chung CY, et al. Pitfalls and important issues in testing reliability using intraclass correlation coefficients in orthopaedic research. Clin Orthop Surg. 2012;4:149–155.
Copyright © 2013 by the American College of Occupational and Environmental Medicine