In this study, we aim to
- Examine the feasibility and acceptability of a digital psychosocial intervention, Wysa for Return to Work, for individuals with a work-related injury, by measuring study qualification rate and app usage parameters; and
- Evaluate the demographics of these individuals, alongside their app usage, in order to assess their engagement and retention on the program.
A work-related injury or illness is broadly defined as any psychological or physical harm sustained in the course of one’s work duties.1,2 Workers’ compensation (WC) programs were set up to provide medical benefits and partial wage replacement to workers who were injured during employment. In 2019, over 144 million employees in the United States had WC protections, and the total benefits paid were $63 billion.3 The US Bureau of Labor Statistics reported 2.7 million private sector nonfatal injuries and illnesses in 2020. The most common injuries identified in WC claims were musculoskeletal in nature.4
Approximately 80% percent of WC compensation claim costs in the United States are attributed to the 5% to 10% of injured workers who suffer long-term disability and are on long-term claims.5,6 Sometimes, these occur because the injured workers are “stuck” in coping patterns that are not conducive to recovery and return to work (RTW).5 Psychosocial stressors such as negative expectations of RTW, pain catastrophizing, fear avoidance, low self-efficacy, perceived injustice, and a lack of access to multidisciplinary resources have been shown to be strong predictors of delayed recovery.7–10 At the same time, a longer RTW duration has been shown to increase the likelihood of experiencing mental health concerns,8,11 which can impact other domains, including reducing quality of life.12,13 This psychosocial impact may be accompanied by significant financial burdens, with some disabled workers losing up to 30% of their income and earnings even years after an injury.14,15
Effective psychosocial resources have been shown to promote recovery from illness and injury.16–18 Digital psychosocial interventions have been found to be effective in filling the demand-supply gap in the provision of psychosocial support services.19–21
Digital psychosocial interventions for recovery and resilience can be delivered through AI-based conversational agents (CAs).22,23 CAs have been found to be effective in promoting stress management techniques,24 and are feasible as an intervention for pain self-management.25
An analysis of user feedback on Wysa, an AI-based CA, has indicated high levels of acceptability, with users highlighting its nonjudgmental approach and easy conversational style,26 which enables the creation of a therapeutic alliance.27 An evaluation of a novel, customized version of Wysa built to facilitate pain acceptance and behavioral activation for orthopedic patients with chronic musculoskeletal pain found high retention and engagement rates among users, with high engagement users reporting greater improvements in anxiety scores than low engagement users. Patients who received Wysa reported greater improvements on measures of anxiety, depression, pain interference, and physical function than those who received usual orthopedic care.28,29
In this study, we examine the feasibility and acceptability of a digital psychosocial intervention, Wysa for Return to Work, for individuals with a work-related injury. This study evaluates the demographics of these individuals, alongside their app usage, in order to understand engagement and retention on the program. The findings of this study could help identify individuals who would be more inclined to use digital interventions and who may benefit from them.
The Wysa app is publicly available as an app on the Android and iOS app stores. It has been designed to prioritize safety, privacy, and security by design. There is no user registration required, and no personally identifiable information is asked at any time during app use.
The study involved analyzing app usage data of injured workers on WC claims administered by Travelers, a national insurance carrier. This study only utilizes deidentified data after receiving explicit consent from study participants.
For ethical and privacy reasons, neither the authors nor Travelers had access to user messages or conversations. Only anonymized app usage data were extracted for this research, for 15 months from March 2021 to June 2022. The study data set was deidentified using one-way cryptographic functions. User data were adequately secured according to the organization’s privacy, security, and safety policies.
This was a single-armed, prospective cohort pilot study that offered a resilience and recovery-based intervention to the claimants as part of their claim. Participants were enrolled throughout the study period, from March 2021 to June 2022.
Eligibility Criteria and Recruitment
Participants were recruited from a cohort of WC claims after an absence from work due to an injury. The injury could be of various types, from low severity with minimal lost time from work, to complex claims with longer durations of lost time and potentially long recovery timeframes. Individuals with catastrophic injuries were excluded from this study.
In the course of a regular claims journey, demographic information is collected from the claimants, and their assigned nurse case manager asks them five psychosocial screening questions. Claimants who received a positive score (as defined by the criteria given below in Fig. 1) on any one of the five questions qualified for the study, and were recommended the use of the Wysa RTW program by the nurse case manager. Based on the number of questions they scored positively on, they each had a total score ranging from 1 to 5.
Qualified participants were randomly assigned to either the control group or the test group (see Fig. 2 below).
All study participants received a complimentary, 1-year subscription to Wysa for Return to Work, a novel, customized version of Wysa to facilitate recovery management and RTW. This version is not available commercially. After the 12-week RTW program, the remaining period provided the users with continued support in the form chosen by the user.
Wysa is a multicomponent intervention that uses AI-based technology to deliver therapeutic content such as cognitive behavioral therapy, dialectical behavioral therapy, motivational interviewing, mindfulness training, deep-breathing techniques, and sleep meditations through a CA.30,31
Using behavioral activation and resilience principles, this RTW version of Wysa supported the user with building a recovery mindset. The AI-enabled free-text CA offered users a 12-week program where the CA acted as a companion in the participant’s journey of recovering and returning to work, and the goal of the program was to improve recovery outcomes, help in recovery management, and reduce time taken to RTW.
Data were collected from March 23, 2021, to June 29, 2022, and were obtained from two sources: participants’ self-reported information during screening and nurse case management, collected by Travelers, which included details about their demographics (gender, age), injury (part of body/nature of injury), and psychosocial screening responses; and app usage reports from Wysa, which included user onboarding dates and assessment data, energy scores reported, and sessions/tools completed and their dates.
No personally identifiable data were shared with Wysa by the Travelers study team and vice versa. Screened participants with missing data on gender, and those with “undetermined” (unclassified) injury severity levels, were excluded from the analysis.
For the analysis, participants were stratified on the basis of the following demographic criteria: gender (male, female), age range (17–24, 25–40, 41–56, and 57–75 years), and injury severity level (low, medium, and high). Participants were also stratified on the basis of the number of psychosocial risk-factor indicators they scored positively on.
The primary study objectives were related to the feasibility and acceptability of introducing a digital psychosocial intervention (Wysa) for WC claimants. This analysis aims to understand the individuals who would be inclined to use a digital intervention like Wysa, and infer the feasibility of such a support mechanism. Significance was set a priori at P < 0.05. Statistical analyses were performed using R (v4.2.0).
Objective 1: Feasibility
The first outcome of this study assessed the need for a psychosocial intervention like Wysa in the target population through the study recruitment rate, evaluated as the proportion of screened participants who qualified for the study by scoring positively on at least one of the psychosocial risk-factor indicator questions presented to them at screening. The likelihood of injury types in the qualified group was also examined.
To determine whether any significant differences existed in recruitment rates across demographic strata, the rate was calculated on the basis of gender, age-range, and severity. We conducted a chi-squared test of proportions for the significance test across genders, with multiple pairwise tests between different age ranges and severity levels, and adjusted the P values for the results found positive using the Bonferroni correction.
Objective 2: Acceptability
To assess the acceptability and likelihood of usage of a psychosocial intervention like Wysa, four key outcomes were measured: (1) onboarding rate, (2) retention, (3) engagement, and (4) change in user-reported mood.
The onboarding rate was measured as the percentage of qualified participants who onboarded on the Wysa RTW program. To determine whether any significant differences existed in onboarding rates across demographic strata, the onboarding rate was calculated on the basis of gender, age range, and severity also. A chi-squared test of proportions was utilized for the significance test across genders and psychosocial risk-factor indicators, and multiple pairwise tests between different age ranges and severity levels were conducted, and the P values were adjusted for the results found positive using the Bonferroni correction.
During onboarding, participants also reported what they were looking forward to doing after recovery, the challenges that were affecting their journey, the activities that gave them joy, and how much time they could spare for their daily interactions with the app, and these responses were then used to customize their app experience. The frequencies of these responses were analyzed.
User rate of engagement was defined as the proportion of participants who engaged in at least one Wysa intervention after onboarding to the program. The rate of engagement was calculated across genders, age ranges, severity levels, and psychosocial risk factor indicators.
Descriptive statistics (total sessions, average sessions) of the different types of interventions undertaken by users helped us determine the most commonly used tools. The difference between average sessions across genders, age ranges, severity levels, and total psychosocial risk factor indicators was also tested. The Wilcoxon signed rank test was used for the significance test across genders, age ranges, severity levels, and psychosocial risk factor indicators, since the data were not normally distributed (as per the Shapiro-Wilk normality test).
Retention was defined as the number of days spent on the app before either disengagement or the end of the study period (June 29). Participants who stopped using the app at least a month before June 29 were considered to have disengaged. The participants who onboarded after April 29, 2022, 60 days before the end of the study period were not included. The mean number of retention days was calculated for those who disengaged. The mean difference between retention days across genders, age ranges, severity levels, and total psychosocial risk factor indicators was tested. The Wilcoxon signed rank test was used for these significance tests, since the data sets across genders, age ranges, severity levels, and psychosocial risk factor indicators were not normally distributed (as per the Shapiro-Wilk test for normality). Ages 17 to 24 years and low severity levels were excluded from this analysis because of the low number of users in these cohorts (12 and 2, respectively).
When the participants accessed Wysa through the “talk” button on the home screen, or they started a morning check-in, they were asked about their energy levels and were provided with a visual sliding scale where they could rate their energy levels from 0 to 100 points. Energy scores are rated “high” if they are above 65 points, “neutral” if they are between 35 and 65 points, and “low” if they are below 35 points. We calculated the average energy score rating and (after filtering out users with only one reported energy score, n = 51) the average rate of change of participant’s energy scores over time.
Objective 1: Feasibility
Forty-one percent (5150/12663) of the screened population qualified for the study, having scored positively on at least one of the psychosocial risk-factor indicator questions presented to them at screening (Table 1).
TABLE 1 -
Numbers of participants who were screened, qualified, and onboarded across demographic stratification
||Qualified (Test) for Wysa RTW
| 57–75 years
| 41–56 years
| 25–40 years
| 17–24 years
When examined by demographic variables, 42.4% of women (1910/4497) qualified for the study, and 39.6% of men (3241/8166) qualified, and this difference was statistically significant (P = 0.002423; α = 0.05); that is, women were found to be more in need of psychosocial interventions like Wysa (see Table, Supplemental Digital Content 1, https://links.lww.com/JOM/B235, which compares study recruitment rate across genders).
A total of 42.7% (495/1157) of people aged 17 to 24 years qualified for the study, as compared with 40.6% of those aged 25 to 40 years (1579/3887), 39.9% of those aged 41 to 56 years (1830/4585), and 38% (1220/2979) of those aged 57 to 75 years.
A total of 48.5% (813/1673) of the screened population with high severity injuries qualified for the study, compared with 39.5% (4300/10870) of participants with medium severity injuries and 31.6% (38/120) of participants with low severity injuries. Participants with high severity injuries were found to be statistically more in need of psychosocial interventions than those with medium (P = 3.042e-12; α = 0.05) and low (P = 0.0004783; α = 0.05) severity injuries (see Table, Supplemental Digital Content 2, https://links.lww.com/JOM/B236, which compares study recruitment rate across severity levels). Injuries to the shoulder (15%), lower back (11.6%), and knee (11.8%) represented the greatest proportion of the qualified population.
Objective 2: Acceptability
From those who downloaded the intervention, 190 users (93.5%) successfully onboarded onto Wysa, and 88.4% of all onboarded participants (168/190) engaged with one or more interventions, with an average of 10.8 sessions utilized across participants. Mood check-ins, mindfulness-based tools, behavioral activation interventions, sleep tools, and thought recording were the most commonly used tools across the population (Fig. 3).
Participants whose last day of app usage was 1 month before the end of the study period (n = 165) were retained for an average of 35 days. Participants with continued app usage (n = 10) were retained on the app for an average of 153.9 days.
Women engaged in a greater number of average sessions as compared with men, and this difference was statistically significant (P = 0.02239, α = 0.05) (see Table, Supplemental Digital Content 3, https://links.lww.com/JOM/B237, which compares average of total sessions undertaken across genders). Women retained for an average of 40.6 days on the RTW program before drop-off, and men retained for an average of 33.7 days.
Participants aged 57 to 75 years were retained for an average of 55.3 days on the program, with the difference between those aged 57 to 75 years and those aged 25 to 40 years found to be statistically significant (P = 0.025977; α = 0.05; see Table, Supplemental Digital Content 4, https://links.lww.com/JOM/B238, which compares average retention days across age ranges).
Participants with high severity injuries used the RTW program for 53.1 days on average, and the difference in retention compared with those with medium severity injuries was found to be statistically significant (P = 0.006074; α = 0.05; see Table, Supplemental Digital Content 5, https://links.lww.com/JOM/B239, which compares average retention days across severity levels).
When stratified by demographic criteria (n = 190), men formed 50.5% and women were 49.4% of those who onboarded. Women onboarded at a higher rate than men, and this difference was statistically significant (P = 0.000127; α = 0.05; see Table, Supplemental Digital Content 6, https://links.lww.com/JOM/B240, which compares the study onboarding rate across genders); that is, women were found to be more inclined toward using interventions like Wysa.
Individuals with more than three psychosocial risk factors retained for a greater number of days than those with fewer risk factors (68.2 days, P = 4.749e-07) and engaged in a greater number of sessions per user (18.9 sessions; P = 0.002153).
Participants aged 41 to 56 years formed 37.8% (72/190) of the onboarded population, followed by age ranges 25 to 40 years at 35.2% (67/190), 57 to 75 years at 19.4% (37/190), and 17 to 24 years at 7.3% (14/190).
Participants with medium severity injuries formed 74.7% (142/190) of the onboarded population, followed by those with high severity injuries at 24.2% (46/190), and low severity injuries at 1% (2/190). Participants with high severity injuries were found to be more inclined than those with medium severity injuries toward using digital interventions like Wysa, because of their statistically higher onboarding rates (P = 0.005447; α = 0.05; see Table, Supplemental Digital Content 7, https://links.lww.com/JOM/B241, which compares the study onboarding rate across severity levels). Injuries to the shoulder (14.1%), knee (13.6%), and lower back (7.3%) represented the greatest proportion of those who onboarded onto Wysa.
A total of 90.4% women engaged in an intervention after onboarding onto the app, as compared with 86.4% men. Participants in the 57 to 75 age group had the highest engagement rate across age groups (94.5%), with participants in age groups 41 to 56 years and 25 to 40 years close behind at 90.2% and 88%, respectively.
In the course of onboarding onto the app, 38% of the respondents stated that their recovery was taking longer than they had hoped, 76% stated that they were looking forward to getting back to work after their recovery, 78% wanted to feel like themselves again, and 46% wanted to feel independent. The key challenges mentioned by the respondents were pain (81%), sleep (42%), fatigue (39%), frustration (61%), fear (33%), and sadness (26%). They also stated that a lack of motivation (23%) and confidence (18%) was affecting their recovery.
Out of a total of 357 energy score ratings given by 99 users, 54.8 points was the average energy score per user. On average, participants’ energy scores increased by 3.58 points over time (SD = 13.14 points).
In this pilot study, we examined the feasibility and acceptability of a digital psychosocial intervention, Wysa for Return to Work, which was intended to facilitate and expedite recovery for individuals with a work-related injury and receiving WC benefits.
In the study, 41% of the WC claimants qualified by screening positively on at least one psychosocial risk-factor question, demonstrating a high need for psychosocial interventions among injured workers. This finding is supported by research that documents the large indirect toll on the lives of claimants, caused by disruptions to sleep and daily activities.13 The data also highlight women, adolescents, and young adults, and individuals with musculoskeletal or high severity injuries as more likely to indicate a need for psychosocial support. This is supported by research that indicates a higher risk of work injury and recurrence for women32 and young adults,33 and identifies behavioral health risk factors and musculoskeletal concerns as the leading causes of sick-absence in high-income countries.34
Individuals who screened positively on three or more psychosocial risk factor indicators onboarded at a higher rate than those with fewer risk factors, retained on the RTW program for much longer, and also engaged in a higher number of sessions per user, indicating that people with a greater need for psychosocial support were also more inclined to seek the support they needed through the program. Furthermore, participants with high severity injuries, which are associated with greater psychological morbidity,35 were also statistically more likely to qualify for the study and use the app for longer than those with less severe injuries. The retention potentially indicates that the app was able to facilitate a supportive space for those with heightened needs, as also indicated by an acceptability study of user reviews for Wysa.26
For users who completed usage within the study period, the mean engagement in the study was 10.8 sessions and the mean retention period was 35 days, and over 153.9 days of app usage for those who continued utilization. Evidence suggests an average retention period between 4 and 16 days in most digital health applications.36,37 These high rates of engagement and retention indicate that the participants found the app features helpful for their everyday challenges.38 The app’s CA has also demonstrated the ability to create a therapeutic alliance, which has also been positively linked with user engagement.39
Greater engagement with CAs has also been shown to lead to greater reductions in psychological distress,31,40–43 suggesting the benefits of using a digital psychosocial intervention for providing psychosocial support for injured workers.
According to a systematic review, a common aspect of successful RTW interventions was behavioral activation, psychoeducation, and a focus on the RTW context.44 It is perhaps the incorporation of behavior-change techniques (self-monitoring and goal setting)45,46 and coping strategies (mindfulness and meditation),47 alongside a behavioral activation framework that led to the sustained engagement and retention.48–50
These findings suggest that a digital mental health intervention is feasible and acceptable for an RTW population and that a fully powered study can help in understanding the extent of the impact that a digital mental health intervention could have on RTW, recovery, and lost time.
Strengths and Limitations
The primary strength of this feasibility study is that, to our knowledge, it is the first investigation to incorporate a digital psychosocial intervention into a WC program for injured workers. The primary limitations are that there was no true control arm/comparison group, which reduced our ability to establish causality for the potential factors responsible for retention and engagement, and delineate variables around type and length of injury as a comparative. The intervention was also only available in English during the study.
Receiving direct feedback from the users of the program, and being able to follow up with them, would help understand their experience better and customize psychosocial support services to fit their needs, and having larger sample sizes across demographic stratifications would help establish a greater degree of confidence in the findings.
Furthermore, information about the app was provided to the qualified population at the start of their claim journey, and there might have been individuals who needed the intervention but did not onboard because of the information overload that is common at this stage.
In summary, this pilot study explored the feasibility and acceptability of an AI-led digital psychosocial intervention, Wysa for Return to Work, which was intended to facilitate and expedite recovery for individuals with a work-related injury and receiving a WC claim in an accessible and affordable way. This study highlights the ability to attain high retention and engagement among injured workers with an app that uses a digital CA, and offers the potential to improve recovery outcomes for injured workers in a manner that is feasible and scalable to deliver in a WC program.
1. Lilley R, Davie G, Langley J, Ameratunga S, Derrett S. Do outcomes differ between work and non-work-related injury in a universal injury compensation system? Findings from the New Zealand Prospective Outcomes of Injury Study. BMC Public Health
2. 1904.5 - determination of work-relatedness. Available at: https://www.osha.gov/laws-regs/regulations/standardnumber/1904/1904.5
. Accessed May 17, 2022.
3. Workers’ Compensation Benefits, Costs, and Coverage. National Academy of Social Insurance
. 2021. Available at: https://www.nasi.org/research/workers-compensation/workers-compensation-benefits-costs-and-coverage/
. Accessed August 15, 2022.
5. MacEachen E, Kosny A, Ferrier S, Chambers L. The ‘toxic dose’ of system problems: why some injured workers don’t return to work as expected. J Occup Rehabil
6. Wickizer TM, Franklin G, Plaeger-Brockway R, Mootz RD. Improving the quality of workers’ compensation health care delivery: the Washington State Occupational Health Services Project. Milbank Q
7. Anderson JT, Haas AR, Percy R, Woods ST, Ahn UM, Ahn NU. Clinical depression is a strong predictor of poor lumbar fusion outcomes among workers’ compensation subjects. Spine
8. Cancelliere C, Donovan J, Stochkendahl M, et al. Factors affecting return to work after injury or illness: best evidence synthesis of systematic reviews. Chiropr Man Therap
9. Blank L, Peters J, Pickvance S, Wilford J, Macdonald E. A systematic review of the factors which predict return to work for people suffering episodes of poor mental health. J Occup Rehabil
10. Caruso G, Kertay L. Part I: psychological factors in delayed and failed recovery and unnecessary disability. Guides Newsletter
11. Franche RL, Carnide N, Hogg-Johnson S, et al. Course, diagnosis, and treatment of depressive symptomatology in workers following a workplace injury: a prospective cohort study. Can J Psychiatry
12. Dembe AE. The social consequences of occupational injuries and illnesses. Am J Ind Med
13. Keogh JP, Nuwayhid I, Gordon JL, Gucer PW. The impact of occupational injury on injured worker and family: outcomes of upper extremity cumulative trauma disorders in Maryland workers. Am J Ind Med
14. Boden LI, Galizzi M. Economic consequences of workplace injuries and illnesses: lost earnings and benefit adequacy. Am J Ind Med
15. Boden LI, Galizzi M. Income losses of women and men injured at work. J Hum Resour
16. Edwards RR, Dworkin RH, Sullivan MD, Turk DC, Wasan AD. The role of psychosocial processes in the development and maintenance of chronic pain. J Pain
17. Noorain S, Paola Scaparra M, Kotiadis K. Mind the gap: a review of optimisation in mental healthcare service delivery. Health Systems
18. Singh OP. Closing treatment gap of mental disorders in India: opportunity in new competency-based Medical Council of India curriculum. Indian J Psychiatry
19. MacIsaac A, Mushquash AR, Mohammed S, Grassia E, Smith S, Wekerle C. Adverse childhood experiences and building resilience with the JoyPop App: evaluation study. JMIR Mhealth Uhealth
20. Wind TR, Rijkeboer M, Andersson G, Riper H. The COVID-19 pandemic: the ‘black swan’ for mental health care and a turning point for e-health. Internet Interv
21. B. Inkster and Digital Mental Health Data Insights Group (DMHDIG). Early warning signs of a mental health tsunami: a coordinated response to gather initial data insights from multiple digital services providers. Front Digit Health
22. Cluver L. Solving the global challenge of adolescent mental ill-health. Lancet Child Adolescent Health
23. Dalton L, Rapa E, Stein A. Protecting the psychological health of children through effective communication about COVID-19. Lancet Child Adolesc Health
24. Shamekhi A, Bickmore T, Lestoquoy A, Gardiner P. Augmenting group medical visits with conversational agents for stress management behavior change. In: Persuasive Technology: Development and Implementation of Personalized Technologies to Change Attitudes and Behaviors
. 2017:55–67. Available at: https://link.springer.com/chapter/10.1007/978-3-319-55134-0_5
. Accessed December 14, 2022.
25. Hauser-Ulrich S, Künzli H, Meier-Peterhans D, Kowatsch T. A smartphone-based health care chatbot to promote self-management of chronic pain (SELMA): pilot randomized controlled trial. JMIR Mhealth Uhealth
26. Malik T, Ambrose AJ, Sinha C. Evaluating user feedback for an artificial intelligence-enabled, cognitive behavioral therapy-based mental health app (Wysa): qualitative thematic analysis. JMIR Hum Factors
27. Beatty C, Malik T, Meheli S, Sinha C. Evaluating the therapeutic alliance with a free-text CBT conversational agent (Wysa): a mixed-methods study. Front Digit Health
28. Leo AJ, Schuelke MJ, Hunt DM, et al. A digital mental health intervention in an orthopedic setting for patients with symptoms of depression and/or anxiety: feasibility prospective cohort study. JMIR Form Res
29. Leo AJ, Schuelke MJ, Hunt DM, Miller JP, Areán PA, Cheng AL. Digital mental health intervention plus usual care compared with usual care only and usual care plus in-person psychological counseling for orthopedic patients with symptoms of depression or anxiety: cohort study. JMIR Form Res
30. Wysa - Everyday Mental Health. In: Wysa - Everyday Mental Health
. Available at: https://www.wysa.io/
. Accessed June 27, 2022.
31. Inkster B, Sarda S, Subramanian V. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR Mhealth Uhealth
32. Capogna BM, Mahure SA, Mollon B, Duenes ML, Rokito AS. Young age, female gender, Caucasian race, and workers’ compensation claim are risk factors for reoperation following arthroscopic ACL reconstruction. Knee Surg Sports Traumatol Arthrosc
33. Breslin C, Koehoorn M, Smith P, Manno M. Age related differences in work injuries and permanent impairment: a comparison of workers’ compensation claims among adolescents, young adults, and adults. Occup. Environ. Med
34. Mikkelsen MB, Rosholm M. Systematic review and meta-analysis of interventions aimed at enhancing return to work for sick-listed workers with common mental disorders, stress-related disorders. Occup Environ Med
2018; [Online]. Available at: https://oem.bmj.com/content/75/9/675.abstract
. Accessed July 15, 2022.
35. Sareen J, Erickson J, Medved MI, et al. Risk factors for post-injury mental health problems. Depress Anxiety
36. Adjust Global App Trends 2019 - Get free report. Available at: https://www.adjust.com/resources/ebooks/adjust-global-app-trends-report-2019/
. Accessed August 16, 2022.
37. Pratap A, Neto EC, Snyder P, et al. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. NPJ Digit Med
38. Eysenbach G. The law of attrition. J Med Internet Res
39. Sinha C, Cheng AL, Kadaba M. Adherence and engagement with a cognitive behavioral therapy-based conversational agent (Wysa for chronic pain) among adults with chronic pain: survival analysis. JMIR Form Res
40. Ring L, Shi L, Totzke K, Bickmore T. Social support agents for older adults: longitudinal affective computing in the home. J Multimodal User Interfaces
41. Fulmer R, Joerin A, Gentile B, Lakerink L, Rauws M. Using psychological artificial intelligence (Tess) to relieve symptoms of depression and anxiety: randomized controlled trial. JMIR Ment Health
42. Burton C, Szentagotai Tatar A, McKinstry B, et al. Pilot randomised controlled trial of Help4Mood, an embodied virtual agent-based system to support treatment of depression. J Telemed Telecare
43. Ly KH, Ly A-M, Andersson G. A fully automated conversational agent for promoting mental well-being: a pilot RCT using mixed methods. Internet Interv
44. Venning A, Oswald TK, Stevenson J, et al. Determining what constitutes an effective psychosocial ‘return to work’ intervention: a systematic review and narrative synthesis. BMC Public Health
45. Lyzwinski LN, Caffery LJ, Bambling M, Edirippulige S. Consumer perspectives on mHealth for weight loss: a review of qualitative studies. J Telemed Telecare
46. Crane D, Garnett C, Brown J, West R, Michie S. Behavior change techniques in popular alcohol reduction apps: content analysis. J. Med. Internet Res
47. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement
with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res
48. Szinay D, Perski O, Jones A, Chadborn T, Brown J, Naughton F. Perceptions of factors influencing engagement with health and well-being apps in the United Kingdom: qualitative interview study. JMIR Mhealth Uhealth
49. Szinay D, Jones A, Chadborn T, Brown J, Naughton F. Influences on the uptake of and engagement with health and well-being smartphone apps: systematic review. J Med Internet Res
50. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med