NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Structured Abstract
Background:
Mental health mobile applications (apps) have the potential to expand the provision of mental health and wellness services to traditionally underserved populations. There is a lack of guidance on how to choose wisely from the thousands of mental health apps without clear evidence of safety, efficacy, and consumer protections.
Purpose:
This Technical Brief proposes a framework to assess mental health mobile applications with the aim to facilitate selection of apps. The results of applying the framework will yield summary statements on the strengths and limitations of the apps and are intended for use by providers and patients/caregivers.
Methods:
We reviewed systematic reviews of mental health apps and reviewed published and gray literature on mental health app frameworks, and we conducted four Key Informant group discussions to identify gaps in existing mental health frameworks and key framework criteria. These reviews and discussions informed the development of a draft framework to assess mental health apps. Iterative testing and refinement of the framework was done in seven successive rounds through double application of the framework to a total of 45 apps. Items in the framework with an interrater reliability under 90 percent were discussed among the evaluation team for revisions of the framework or guidance.
Findings:
Our review of the existing frameworks identified gaps in the assessment of risks that users may face from apps, such as privacy and security disclosures and regulatory safeguards to protect the users. Key Informant discussions identified priority criteria to include in the framework, including safety and efficacy of mental health apps. We developed the Framework to Assist Stakeholders in Technology Evaluation for Recovery (FASTER) to Mental Health and Wellness and it comprises three sections: Section 1. Risks and Mitigation Strategies, assesses the integrity and risk profile of the app; Section 2. Function, focuses on descriptive aspects related to accessibility, costs, organizational credibility, evidence and clinical foundation, privacy/security, usability, functions for remote monitoring of the user, access to crisis services, and artificial intelligence (AI); and Section 3. Mental Health App Features, focuses on specific mental health app features, such as journaling and mood tracking.
Conclusion:
FASTER may be used to help appraise and select mental health mobile apps. Future application, testing, and refinements may be required to determine the framework’s suitability and reliability across multiple mental health conditions, as well as to account for the rapidly expanding applications of AI, gamification, and other new technology approaches.
Contents
- Preface
- Acknowledgments
- Key Informants
- Peer Reviewers
- Executive Summary
- Introduction
- Methods
- Findings
- Summary and Implications
- References
- Appendixes
- Appendix A. List of Existing Frameworks
- Appendix B. Framework Criteria
- Appendix C. Modifications in uMARS Questions
- Appendix D. Pilot Rounds Results
- Appendix E. Framework Training Guide: Framework to Assist Stakeholders in Technology Evaluation for Recovery (FASTER) to Mental Health and Wellness Framework Assessment
- Appendix F. Framework
- Appendix G. Framework Assessment Examples
Suggested citation:
Agarwal S, Jalan M, Wilcox HC, Sharma R, Hill R, Pantalone E, Thrul J, Rainey JC, Robinson KA. Evaluation of Mental Health Mobile Applications. Technical Brief 41. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 75Q80120D00003.) AHRQ Publication No. 22-EHC016. Rockville, MD: Agency for Healthcare Research and Quality; May 2022. DOI: https://doi.org/10.23970/AHRQEPCTB41. Posted final reports are located on the Effective Health Care Program search page.
This report is based on research conducted by the Johns Hopkins University Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 75Q80120D00003). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.
The information in this report is intended to help healthcare decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of healthcare services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.
This report is made available to the public under the terms of a licensing agreement between the author and the Agency for Healthcare Research and Quality. This report may be used and reprinted without permission except those copyrighted materials that are clearly noted in the report. Further reproduction of those copyrighted materials is prohibited without the express permission of copyright holders.
AHRQ or U.S. Department of Health and Human Services endorsement of any derivative products that may be developed from this report, such as clinical practice guidelines, other quality enhancement tools, or reimbursement or coverage policies, may not be stated or implied.
AHRQ appreciates appropriate acknowledgment and citation of its work. Suggested language for acknowledgment: This work was based on an evidence report, Evaluation of Mental Health Mobile Applications, by the Evidence-based Practice Center Program at the Agency for Healthcare Research and Quality (AHRQ).
- NLM CatalogRelated NLM Catalog Entries
- Review Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps.[BMJ Open. 2021]Review Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps.Lagan S, Sandler L, Torous J. BMJ Open. 2021 Mar 19; 11(3):e047001. Epub 2021 Mar 19.
- An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process.[JMIR Mhealth Uhealth. 2018]An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process.Zelmer J, van Hoof K, Notarianni M, van Mierlo T, Schellenberg M, Tannenbaum C. JMIR Mhealth Uhealth. 2018 Jul 9; 6(7):e10016. Epub 2018 Jul 9.
- Insights from user reviews to improve mental health apps.[Health Informatics J. 2020]Insights from user reviews to improve mental health apps.Alqahtani F, Orji R. Health Informatics J. 2020 Sep; 26(3):2042-2066. Epub 2020 Jan 10.
- Systematic assessment of the quality and integrity of popular mental health smartphone apps using the American Psychiatric Association's app evaluation model.[Front Digit Health. 2022]Systematic assessment of the quality and integrity of popular mental health smartphone apps using the American Psychiatric Association's app evaluation model.Rickard NS, Kurt P, Meade T. Front Digit Health. 2022; 4:1003181. Epub 2022 Sep 29.
- Review Applying Digital Technology to Understand Human Experiences of Climate Change Impacts on Food Security and Mental Health: Scoping Review.[JMIR Public Health Surveill. 2...]Review Applying Digital Technology to Understand Human Experiences of Climate Change Impacts on Food Security and Mental Health: Scoping Review.Bhawra J, Elsahli N, Patel J. JMIR Public Health Surveill. 2024 Jul 23; 10:e54064. Epub 2024 Jul 23.
- Evaluation of Mental Health Mobile ApplicationsEvaluation of Mental Health Mobile Applications
Your browsing activity is empty.
Activity recording is turned off.
See more...