Secondary Logo

Journal Logo

A Cluster Randomized Controlled Trial to Evaluate HeadCoach

An Online Mental Health Training Program for Workplace Managers

Gayed, Aimée MCrim; Bryan, Bridget T. MSc; LaMontagne, Anthony D. ScD; Milner, Allison PhD; Deady, Mark PhD; Calvo, Rafael A. PhD; Mackinnon, Andrew PhD; Christensen, Helen PhD; Mykletun, Arnstein PhD; Glozier, Nicholas PhD; Harvey, Samuel B. PhD

Journal of Occupational and Environmental Medicine: July 2019 - Volume 61 - Issue 7 - p 545–551
doi: 10.1097/JOM.0000000000001597
ORIGINAL ARTICLES
Free

Objective: Mental ill-health is now the leading cause of sickness absence and occupational incapacity in high-income countries. This study evaluated HeadCoach online manager training, designed to improve confidence, and managerial behaviors that create mentally healthy workplaces.

Methods: A cluster randomized controlled trial was conducted comparing managers who received HeadCoach (N = 87) to waitlist control (N = 123). Managers’ confidence and behavior were investigated at baseline, postintervention, and follow-up. Psychological distress of direct reports was evaluated.

Results: Confidence significantly increased postintervention only; however, per-protocol analyses indicated a significant improvement for program completers compared with control at both time points. Responsive and preventive behaviors significantly improved. Psychological distress of direct reports remained unchanged.

Conclusions: HeadCoach online mental health training is an effective and scalable way to improve managers’ confidence and workplace practices around mental health. The impact on direct reports remains unknown.

School of Psychiatry, University of New South Wales, Sydney, Australia (Ms Gayed and Dr Mykletun); Black Dog Institute, Faculty of Medicine, University of New South Wales, Sydney, Australia (Ms Bryan, Dr Deady, Dr Mackinnon, Dr Christensen, and Dr Harvey); Centre for Population Health Research, Deakin University, Geelong, Victoria, Australia (Dr LaMontagne); School of Population and Global Health, The University of Melbourne, Melbourne, Australia (Dr LaMontagne and Dr Milner); School of Electrical and Information Engineering, University of Sydney, Sydney, Australia (Dr Calvo); Centre for Mental Health, University of Melbourne, Melbourne, Australia (Dr Mackinnon); Department of Mental Health and Suicide, Norwegian Institute of Public Health, Oslo, Norway (Dr Mykletun); Department of Community Medicine, University of Tromsø, Tromsø, Norway (Dr Mykletun); Centre for Work and Mental Health, Nordland Hospital Trust, Bodø, Norway (Dr Mykletun); Centre for Research and Education in Forensic Psychiatry and Psychology, Haukeland University Hospital, Bergen, Norway (Dr Mykletun); and Brain and Mind Centre & Central Clinical School, Faculty of Medicine and Health, University of Sydney, Sydney, Australia (Dr Glozier).

Address correspondence to: Aimée Gayed, MCrim, School of Psychiatry, University of New South Wales, Black Dog Institute, Hospital Road, Randwick, NSW 2031, Australia (a.gayed@unsw.edu.au).

This study has received ethical approval from the South Eastern Sydney Local Health District Human Research Ethics Committee HREC ref. no: 16/348 (HREC/16/POWH/684). It has been registered with the Australian New Zealand Clinical Trials Registry: ACTRN12617000279325 before the commencement of participant recruitment.

Authors’ contributions: AG, SBH, NG, AMi, AMy, ADL, MD, and RAC contributed to the design of the study. AG, SBH, and AMa conducted the analyses. AG and SBH prepared the first draft of the manuscript. All authors approved the final manuscript for submission.

Conflict of interests: AG, ADL, NG, and SBH co-own the intellectual property for HeadCoach. HC, AMa, and SBH are employed by the Black Dog Institute who provide manager training to workplaces.

Sources of interest: This study was funded by beyondblue with donations from the Movember Foundation (Project Code: LK 7139). Additional funding was provided by the icare Foundation and the Mental Health Branch of NSW Health.

Clinical significance: Managers, who completed HeadCoach online workplace mental health training, had significantly higher confidence and application of both preventive and responsive behaviors to support the mental health of staff they supervise. Such evidence-based online training can offer workplaces a practical and effective means to improve managers’ workplace practices around mental health.

Over recent decades, occupational impairment due to psychiatric conditions has steadily increased,1 with mental health conditions now becoming the leading cause of long-term sickness absence and work incapacity in most high-income countries.2–4 Mental health conditions seen in the workplace, such as depression, anxiety, and stress-related disorders, may often be precipitated by characteristics of the workplace.5 This relationship is now acknowledged as a major public health concern.6

Managers play a key role in the well-being of staff they supervise through the application of appropriate preventive and responsive managerial strategies.7,8 Their knowledge of workplace issues, and ability to implement changes to working conditions for staff, place managers in an influential position to minimize or prevent the impact of work-related mental health risk factors. Additional preventive strategies include modeling accepting attitudes toward mental ill-health and supporting the mental health needs of staff.9,10 The corollary of this is that managers who behave in an inappropriate or bullying manner can contribute to the development of mental health problems.11,12 The way managers respond to staff experiencing mental ill-health can also impact the recovery process for workers,7,8 with evidence suggesting a positive association between manager contact and the recovery and return to work of employees reporting directly to them.7,13

Despite the importance of their role, many managers report uncertainty about how to best support mental health needs of their staff.10,14 This has led to the development of a range of mental health training programs specifically for managers. A recent systematic review and meta-analysis found such training can improve managers’ mental health knowledge, reduce stigmatizing attitudes toward mental illness, and increase implementation of supportive managerial behaviors.15 To date, manager mental health training programs have principally focused on face-to-face training. Although potentially effective,16 such training may be expensive and logistically difficult to deliver, especially if regular booster sessions are needed. Online training has the benefits of standardized delivery combined with the flexibility to tailor content to target audience, and be scheduled around users’ job demands,17 with the opportunity to revisit content within the learning environment to enhance the consolidation of course material.

Considering these factors, and following recently recommended best practice frameworks for workplace mental health initiatives,9,18 we developed an online training intervention for managers called HeadCoach. HeadCoach is the first mental health program for managers delivered entirely online offering a suite of both responsive and preventive strategies that offer a suite of both responsive and preventive strategies to help managers better understand and support the mental health needs of their staff. This randomized controlled trial (RCT) was conducted to test the effectiveness of HeadCoach to improve managers’ confidence in implementing evidence-based responsive and preventive managerial techniques to create a mentally healthy workplace.

Back to Top | Article Outline

METHODS/DESIGN

Study Design and Participants

A cluster RCT was conducted in partnership with three organizations. Two organizations were state-specific ambulance services across Australia. The third was an Australia-wide building equipment hire company. The study protocol, including selection of the primary outcome and detailed analysis plan, was published before the study being completed19 (HREC ref.no.16/348-HREC/16/POWH/684).

Back to Top | Article Outline

Randomization

Within each organization, clusters of managers were defined by preexisting geographical work zones comprising offices or stations at which managers and the staff they supervised were based. An independent researcher based at the University of New South Wales used a computer-generated program to conduct the stratified randomization of workplace clusters within each of the three organizations.

Back to Top | Article Outline

Inclusion Criteria

All participants at the manager and direct report employee levels were required to be 18 years or older; be residing in Australia; have good English comprehension; and work for one of the collaborating industry partners. In addition, managers were required to be supervising three or more staff members. Direct report employees who met the above criteria were included in the analyses if at least half of the managers from their site had enrolled for the trial.

Back to Top | Article Outline

Trial Procedures

Managers

Following completion of the online baseline questionnaire, managers in the intervention group received immediate access to the online HeadCoach manager training program. For managers in both the intervention and control groups, notification of the postquestionnaire was emailed at 6 weeks following baseline. If a manager in the intervention group completed all components of the online program earlier within the 6-week training period, they received the postquestionnaire at that time point. At 4-month postbaseline, notification regarding the final questionnaire was sent to both groups irrespective of rate of adherence to subsequent stages of the trial. At the completion of this final questionnaire, the control group received access to the HeadCoach program. Managers from both groups who completed this questionnaire were entered into a prize draw for one of three vouchers to the value of AUD$250.

Back to Top | Article Outline

Direct Report Employees

Direct report employees completed the baseline questionnaire in the month before any managers commenced the intervention. This questionnaire was the only activity of participation required until the follow-up questionnaire, which was distributed via email 5 months after their baseline to coincide with the managers’ 4-month follow-up questionnaire. The email also informed participants of the opportunity at completion to enter a prize draw for one of three AUD$250 vouchers.

Back to Top | Article Outline

Primary Outcome

The primary outcome for this study was a change in managers’ self-reported confidence to create a mentally healthy workplace in which the mental health needs of their direct report employees are appropriately supported. This primary outcome was selected a priori and specified in our previously published protocol.19 Mangers’ confidence was assessed at each assessment point using a modified version of a previously published supervisor scale.7 This modified scale has been used in published RCTs of other manager interventions16 and validated against manager behavior.20 This scale describes six workplace scenarios which managers are asked to indicate their level of confidence in dealing with on a five-point Likert scale ranging from not at all to extremely confident, resulting in an overall confidence score ranging from 6 to 30. Scenarios included “Initiating contact with staff on sickness absence leave that you believe might be due to mental illness” and “Creating a work environment that prevents and reduces stress within my team.”21

Back to Top | Article Outline

Secondary Outcomes

Changes in managers’ behaviors were measured using an adapted version of the Health and Safety Executive (HSE) Management Standards Indicator Tool.22 This evaluated both responsive behaviors to staff experiencing mental ill-health and preventive behavior to reduce mental ill-health risks within the workplace to create a more mentally healthy workplace. Direct report employees were asked about their level of psychological distress using the 6-item Kessler Psychological Distress Scale (K6).23

Back to Top | Article Outline

Statistical Analyses

The primary analysis was undertaken within an intent-to-treat framework utilizing mixed-model repeated measures (MMRM).19 Clustering was accommodated by a random cluster membership factor, and an unstructured variance–covariance matrix was used to accommodate the relationships between observations at different occasions of measurement. In addition to the group-by-time interaction, differences between the intervention and control groups at each of the follow-up time points were examined against baseline using planned contrasts. A priori planned per-protocol analyses were conducted to assess the effectiveness of the program among those who completed differing numbers of the online modules compared with the waitlist control group. Before undertaking the analysis, raw data were examined for outliers. Where outliers were excluded for the primary analysis, sensitivity analyses for the primary and secondary outcomes were conducted which included the outliers. Analyses were conducted in SPSS version 23.

Back to Top | Article Outline

Role of the Funding Source

This project was developed with funding from beyondblue with donations from the Movember Foundation. Additional funding was provided by the icare foundation and the Mental Health Branch of NSW Health, Australia. The funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Back to Top | Article Outline

RESULTS

Manager Outcomes

As outlined in Figure 1, a total of 229 managers consented to participate in this study. Of those who completed baseline, follow-up data were available for 41 intervention group managers (47.1%) and 78 control group managers (63.4%). Examination of the response trajectory of each manager participating in the study generated for the primary outcome determined improbable outlier responses supplied by one manager at follow-up. As a result, this participant was excluded. However, sensitivity analysis including responses from this outlier confirmed that the exclusion of this individual did not alter the results reported. Figure 1 details the trial profile and participant retention for managers and their direct reports in both conditions.

FIGURE 1

FIGURE 1

Demographic details of the manager sample at baseline are shown in Table 1. There were no differences between intervention and control participants at baseline (all P > 0.05).

TABLE 1

TABLE 1

The intraclass correlation (ICC) for the primary outcome measuring managers’ confidence in supporting the mental health needs of their staff was 0.0004. For the primary outcome, there was a significant interaction of condition group and time (F[2, 121.7] = 4.3, P = 0.015). Analyses of postintervention measurements for the primary outcome indicated manager confidence significantly improved over time in the intervention group compared with the control (P = 0.004), although this difference was attenuated at follow-up (P = 0.082) (Fig. 2A). These differences equate to Cohen's d effect sizes of 0.44 (95% confidence interval [CI], 0.16 to 0.72) and 0.35 (95% CI, 0.08 to 0.63), respectively. Figure 2B and C displays the pattern of change for the two behavioral outcomes. At postintervention, a significant difference was detected between the intervention and control groups for responsive behavior (P = .012). This difference remained significant at follow-up (P = 0.036). Similarly, for preventive behavior, significant differences between the conditions were found postintervention (P = 0.003) and follow-up (P = 0.026).

FIGURE 2

FIGURE 2

In the intervention group, 32 (36.8%) managers completed all HeadCoach modules. As outlined in our published analysis plan,19 per-protocol analyses were also conducted. As shown in Figure 3A, a significant difference was found in the confidence scores between HeadCoach completers and the control group (P = 0.015). This effect was not found when comparing partial program completers with the control group (P = 0.97). Figure 3B and C shows the similar patterns that were observed for both behavioral outcomes.

FIGURE 3

FIGURE 3

Back to Top | Article Outline

Direct Report Employee Outcomes

Of the 391 direct report employees who provided baseline data, 173 (44%) responded to the follow-up questionnaire. There were no detectable differences in K6 scores between the two groups over the follow-up period (F[1, 184.65] = 0.57, P = 0.57) with levels of psychological symptoms in both groups remaining stable over time.

Back to Top | Article Outline

DISCUSSION

This cluster RCT is the first to examine the impact of a manager mental health training program that addresses both responsive and preventive strategies and delivered entirely online. Our findings suggest that this program can improve managers’ confidence and lead to changes in responsive and preventive behavior important in creating a mentally healthy working environment for staff. Given the rising costs of mental ill-health among workers in most developed countries, these are key findings with major implications. There is a growing concensus that workplaces should be one of the key domains in which public mental health initiatives are focused,24 but to date there have been very few scalable, evidence-based interventions that can be used in workplaces.25 The availability of an evidence-based, simple online training program for managers represents a major step toward the hope of achieving more supportive, mentally healthy workplaces.

The choice of managers’ confidence as the primary outcome was driven by recent observational research showing that confidence is the most important predictor of positive manager behavior regarding mental health in the workplace.20 This was also in keeping with theories, suggesting that people are more likely to engage in a particular behavior when their confidence or self-efficacy to succeed is higher.26 In line with these assumptions, our results showed that as well as improving manager's confidence, HeadCoach also led to changes in managers’ behavior. Importantly, these differences in manager behavior remained apparent at follow-up, supporting the potential of this online manager training to generate sustained changes to the way managers handle mental health issues in the workplace by minimizing work-related mental health risk factors, and supporting staff through episodes of mental illness. However, this study was not able to demonstrate that HeadCoach produced changes in the mental health and well-being of workers reporting to the managers undertaking the additional training. The inability to capture the proposed flow on benefits to staff is similar to previous findings from other studies of manager training.15,27 In contrast, one previous trial of face-to-face manager training, which had very similar content to HeadCoach, was able to detect a change in sickness absence among direct reports.16 However, the type of linked sickness absence data used to evaluate face-to-face training was not available for this trial, so a direct comparison of employee level results is not possible. In addition, there are a number of other potential methodological reasons for the lack of detectable change among direct reports. Although follow-up in this study was longer than for many previously published controlled trials, the duration was less than for Milligan-Saville et al's positive study of face-to-face training,16 so may still not have been sufficient to allow improvement in manager behaviors to impact the experiences of direct reports. In addition, we refrained from asking direct reports to supply the name of their manager to maximize reponse rate, with employee–manager linkage conducted based on worksite location. Following the collection of data, it became apparent that this method of linking data did not produce a robust means of matching employees to managers, as although employees and managers could be identified as working at the same site, it was uncertain whether the employee was directly managed by a supervisor who had opted into the study. This potential misclassification of direct reports combined with the relatively small sample may have led to type 2 errors. Therefore, it remains uncertain whether online training is able to produce a meaningful change at the level of direct reports similar to that of face-to-face training.

There are a number of other limitations to this study that should be considered. The adherence rate for program completion and follow-up surveys, although not dissimilar to previously published studies on internet interventions,28–30 were low. Personalized email reminders included in this study to enhance participant engagement may have increased response rate somewhat; however, future evaluations may consider additional strategies, such as text messages, to further promote streamlined progress. A further limitation of this study is the reliance on self-reported measures to evaluate the effectiveness of the program. There is risk of managers reporting their confidence and managerial practices more favorably; however, the anonymity of an online survey should have reduced the likelihood of this occurring. Future research may also find value in evaluating employee sickness absence records and staff turnover data to determine effectiveness of the intervention at the direct report level. It is also important to acknowledge is that this intervention was implemented in isolation to allow the evaluation on the impacts of this strategy alone.

This study is the first to demonstrate the benefits a training program delivered entirely online can have on managers’ confidence and behavior. Given the rising costs of mental ill-health among workers in most developed countries, there is an urgent need for evidence-based, feasable workplace mental health programs. Although there remains value in future examinations of manager training to compare face-to-face and online training to determine the equivalence of delivering content via these different methods, our results suggest that online training programs can offer a practical, efficient, and effective means to enhance the way managers support the mental health needs of their employees.

Back to Top | Article Outline

Acknowledgments

The authors are extremely grateful to Louise Ashelford and Zoe Wooldridge from NSW Ambulance, Anita Savic from Ambulance Victoria, Kate Robinson and Martine Briers from Coates Hire, and all participating staff from these three organizations for their contribution to this study.

Back to Top | Article Outline

REFERENCES

1. Harvey S, Deady M, Wang MM-J, et al. Is the prevalence of mental illness increasing in Australia? Evidence from national health surveys and administrative data, 2001–2014. Med J Aust 2017; 206:490–493.
2. Harvey SB, Henderson M, Lelliott P, Hotopf M. Mental health and employment: much work still to be done. Br J Psychiatry 2009; 194:201–203.
3. McLellan RK, Pransky G, Shaw WS. Disability management training for supervisors: a pilot intervention program. J Occup Rehabil 2001; 11:33–41.
4. Tsutsumi A. Development of an evidence-based guideline for supervisor training in promoting mental health: literature review. J Occup Health 2011; 53:1–9.
5. Harvey SB, Modini M, Joyce S, et al. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup Environ Med 2017; 74:301–310.
6. Harvey SB, Henderson M. Occupational psychiatry. Psychiatry 2009; 8:174–178.
7. Nieuwenhuijsen K, Verbeek JH, de Boer AG, Blonk RW, van Dijk FJ. Supervisory behaviour as a predictor of return to work in employees absent from work due to mental health problems. Occup Environ Med 2004; 61:817–823.
8. Lane TJ, Lilley R, Hogg-Johnson S, LaMontagne AD, Sim MR, Smith PM. A prospective cohort study of the impact of return-to-work coordinators in getting injured workers back on the job. J Occup Rehabil 2018; 28:298–306.
9. Petrie K, Joyce S, Tan L, et al. A framework to create more mentally healthy workplaces: a viewpoint. Aust N Z J Psychiatry 2017; 52:15–23.
10. Harvey SB, Joyce S, Tan L, et al. Developing a mentally healthy workplace: a review of the literature. Available at: http://www.mentalhealthcommission.gov.au/media/116414/Developing%20a%20mentally%20healthy%20workplace_Final%20November%202014.docxMentalHealthCommission,AustralianGovernment; 2014. Accessed January 2019.
11. Verkuil B, Atasayi S, Molendijk ML. Workplace bullying and mental health: a meta-analysis on cross-sectional and longitudinal data. PLoS One 2015; 10:e0135225–e135230.
12. Finne LB, Knardahl S, Lau B. Workplace bullying and mental distress—a prospective study of Norwegian employees. Scand J Work Environ Health 2011; 37:276–287.
13. CIPD. Chartered Institute of Personnel and Development. Manager support for return to work following long-term sickness absence: Chartered Institute of Personnel and Development. Available at: https://www.cipd.co.uk/Images/manager-support-for-return-to-work-following-long-term-sickness-absence_2010_tcm18-10818.pdf. 2010. Accessed January 2019.
14. Henderson M, Harvey SB, Overland S, Mykletun A, Hotopf M. Work and common psychiatric disorders. J R Soci Med 2011; 104:198–207.
15. Gayed A, Milligan-Saville JS, Nicholas J, et al. Effectiveness of training workplace managers to understand and support the mental health needs of employees: a systematic review and meta-analysis. Occup Environ Med 2018; 75:462–470.
16. Milligan-Saville JS, Tan L, Gayed A, et al. Workplace mental health training for managers and its effect on sick leave in employees: a cluster randomised controlled trial. Lancet Psychiatry 2017; 4:850–858.
17. Kawakami N, Kobayashi Y, Takao S, Tsutsumi A. Effects of web-based supervisor training on supervisor support and psychological distress among workers: a randomized controlled trial. Prev Med 2005; 41:471–478.
18. LaMontagne AD, Martin A, Page KM, et al. Workplace mental health: developing an integrated intervention approach. BMC Psychiatry 2014; 14:131.
19. Gayed A, Bryan BT, Petrie K, et al. A protocol for the HeadCoach trial: the development and evaluation of an online mental health training program for workplace managers. BMC Psychiatry 2018; 18:25.
20. Bryan BT, Gayed A, Milligan-Saville JS, et al. Managers’ response to mental health issues among their staff. Occup Med 2018; 68:464–468.
21. Gayed A, LaMontagne AD, Milner A, et al. A new online mental health training program for workplace managers: pre-post pilot study assessing feasibility, usability, and possible effectiveness. JMIR Ment Health 2018; 5:e10517.
22. Health and Safety Executive. Stress Management Competency Indicator Tool - Manager version. 2009.
23. Kessler RC, Andrews G, Colpe LJ, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32:959–976.
24. Davies SC. Annual Report of the Chief Medical Officer 2013, Public Mental Health Priorities: Investing in the Evidence. London: Department of Health; 2014.
25. Joyce S, Modini M, Christensen H, et al. Workplace interventions for common mental disorders: a systematic meta-review. Psychol Med 2016; 46:683–697.
26. Sweet SN, Fortier MS, Strachan SM, Blanchard CM, Boulay P. Testing a longitudinal integrated self-efficacy and self-determination theory model for physical activity post-cardiac rehabilitation. Health Psychol Res 2014; 2:1008.
27. Kawakami N, Takao S, Kobayashi Y, Tsutsumi A. Effects of web-based supervisor training on job stressors and psychological distress among workers: a workplace-based randomized controlled trial. J Occup Health 2006; 48:28–34.
28. Christensen H, Griffiths KM, Farrer L. Adherence in Internet interventions for anxiety and depression: systematic review. J Med Internet Res 2009; 11:e13.
29. Fear NT, Jones M, Murphy D, et al. What are the consequences of deployment to Iraq and Afghanistan on the mental health of the UK armed forces? A cohort study. Lancet 2010; 375:1783–1797.
30. Cook C, Heath F, Thompson RL. A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Meas 2000; 60:821–836.
Keywords:

behavior; eHealth; manager; mental health education; online intervention; randomized controlled trial; supervisor training; workplace mental health

Copyright © 2019 by the American College of Occupational and Environmental Medicine