With over 10,000 mental health– and psychiatry-related smartphone apps available today and expanding, there is a need for reliable and valid evaluation of these digital tools. However, the updating and nonstatic nature of smartphone apps, expanding privacy concerns, varying degrees of usability, and evolving interoperability standards, among other factors, present serious challenges for app evaluation. In this article, we provide a narrative review of various schemes toward app evaluations, including commercial app store metrics, government initiatives, patient-centric approaches, point-based scoring, academic platforms, and expert review systems. We demonstrate that these different approaches toward app evaluation each offer unique benefits but often do not agree to each other and produce varied conclusions as to which apps are useful or not. Although there are no simple solutions, we briefly introduce a new initiative that aims to unify the current controversies in app elevation called CHART (Collaborative Health App Rating Teams), which will be further discussed in a second article in this series.
*Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA;
†NICM, School of Science and Health, Western Sydney University;
‡Black Dog Institute, University of New South Wales, Sydney, New South Wales, Australia;
§Oxford Institute of Population Ageing, University of Oxford, Oxford, United Kingdom;
∥Gerontology Research Centre, Simon Fraser University, Burnaby, British Columbia, Canada;
¶Division of Psychology and Mental Health, Faculty of Biology, Medicine and Health University of Manchester;
#Youth Mental Health Research Unit, Greater Manchester West Mental Health Foundation Trust, Manchester, United Kingdom;
**Division of Hospital Medicine, Department of Psychiatry, Clinical Informatics, University of California San Francisco, San Francisco, CA;
††Department of Biomedical Informatics and Medical Education, University of Washington;
‡‡Sage Bionetworks, Seattle, WA;
§§Department of Psychiatry, University of California, Davis, Sacramento, CA;
∥∥Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King’s College London;
¶¶South London & Maudsley NHS Foundation Trust, Maudsley Hospital, London, United Kingdom;
##School of Psychiatry, Faculty of Medicine, University of New South Wales, Sydney, Australia.
Send reprint requests to John Torous, MD, MBI, Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Ave, Boston, MA, 02446. E-mail: firstname.lastname@example.org.
John Torous and Joseph Firth co-first authors.