Preface

The VA Evidence-based Synthesis Program (ESP) was established in 2007 to provide timely and accurate syntheses of targeted healthcare topics of particular importance to clinicians, managers, and policymakers as they work to improve the health and healthcare of Veterans. QUERI provides funding for four ESP Centers, and each Center has an active University affiliation. Center Directors are recognized leaders in the field of evidence synthesis with close ties to the AHRQ Evidence-based Practice Centers. The ESP is governed by a Steering Committee comprised of participants from VHA Policy, Program, and Operations Offices, VISN leadership, field-based investigators, and others as designated appropriate by QUERI/HSR&D.

The ESP Centers generate evidence syntheses on important clinical practice topics. These reports help:

• Develop clinical policies informed by evidence;
• Implement effective services to improve patient outcomes and to support VA clinical practice guidelines and performance measures; and
• Set the direction for future research to address gaps in clinical knowledge.

The ESP disseminates these reports throughout VA and in the published literature; some evidence syntheses have informed the clinical guidelines of large professional organizations.

The ESP Coordinating Center (ESP CC), located in Portland, Oregon, was created in 2009 to expand the capacity of QUERI/HSR&D and is charged with oversight of national ESP program operations, program development and evaluation, and dissemination efforts. The ESP CC establishes standard operating procedures for the production of evidence synthesis reports; facilitates a national topic nomination, prioritization, and selection process; manages the research portfolio of each Center; facilitates editorial review processes; ensures methodological consistency and quality of products; produces “rapid response evidence briefs” at the request of VHA senior leadership; collaborates with HSR&D Center for Information Dissemination and Education Resources (CIDER) to develop a national dissemination strategy for all ESP products; and interfaces with stakeholders to effectively engage the program.

Comments on this evidence report are welcome and can be sent to Nicole Floyd, ESP CC Program Manager, at vog.av@dyolF.elociN.

Executive Summary

Measurement based care (MBC) is a care delivery approach involving the regular use of standardized measures in routine mental health care to identify individuals not improving as expected and to prompt treatment changes. In the US Department of Veterans Affairs (VA), MBC is specifically defined as: (1) Collect = use of “reliable, validated, clinically appropriate measures at intake and at regular intervals”, (2) Share = “results from the measures are immediately shared and discussed with the Veteran and other providers involved in the Veteran’s Care”, and (3) Act = “Together, providers and Veterans use outcome measures to develop treatment plans, assess progress over time, and inform shared decisions about changes to the treatment plan over time”. As of January 2018, the Joint Commission requires MBC use in all mental health treatment programs accredited under behavioral health standards both within and outside of VA. As MBC delivery has varied widely and shown equally variable clinically meaningful effects across studies, guidance is needed on which specific delivery approaches may operate most effectively and why. This rapid evidence synthesis builds on recent conflicting reviews by adding 14 new studies and focusing on the subset of approaches with the most clinically meaningful and highest-strength evidence and with the most relevance to the specific approach currently recommended by VA.

Despite the large volume of new studies, identification of the most promising delivery approaches for VA remains difficult, because the methodological quality of the evidence remains low, no studies were in Veterans, no studies evaluated the specific approach currently recommended by VA, and effects on other important clinical outcomes, patient satisfaction with care processes, and adverse effects or unintentional consequences remain unknown. The greatest weaknesses of this evidence are that 1) it lacks measurement of the hypothesized mechanism of action (eg, detection of non-response and change in treatment plan) and 2) it lacks information about MBC protocol fidelity.

The most promising MBC approach we identified was when MBC was used in a single Norwegian general outpatient psychiatric clinic in the course of an intense implementation strategy including extensive training provided by the PROMs tool creators, use of technology-assisted automated risk scoring, and strong management advocacy including moral and financial support for providers (48% vs 33%; OR 1.91; 95% CI 0.88 to 4.15; P = 0.1025; NNT = 7, Executive Summary Table). Key strengths of this study that increase our confidence that the mechanisms of effect could be specifically attributed to MBC are that it took extra measures to minimize confounding due to therapist variability and clients’ pretreatment distress levels and better protected against lack of blinding by using an independent outcome assessment measure. However, its use of a not-yet-VA-recommended assessment tool and an intense implementation facilitation strategy raises concerns about the feasibility of its widespread use across VA nationally in different clinical settings with variable resources.

The effects of MBC on suicide behavior, functioning, and quality of life are largely unknown. In addition to clinical outcomes, although it has been suggested that MBC has the potential to improve patient satisfaction with care and treatment adherence, and to reduce no-shows and drop-outs, to date there is limited randomized controlled trial evidence to support these proposed benefits.

The potential benefits of MBC have been best shown in populations with anxiety and/or depressive disorders. MBC has also shown some promise in couples’ therapy and in inpatient treatment of eating disorders, but not for outpatient treatment of eating disorders, the specific symptoms of schizophrenia, or for patients in severe psychiatric crisis seeking emergency help. We found no studies of MBC in PTSD, bipolar disorder, or for suicide prevention.

MBC is a complex, multicomponent, multidisciplinary, and nuanced care delivery process that can represent a major change to practice. However, it is inherently difficult to study because there are so many more sources of heterogeneity and confounding – system, provider, patient, MBC approach – than with a single intervention, such as with a new antidepressant. New research would be more meaningful if it adequately addressed a broader range of sources of confounding, demonstrated that MBC shortened time to identifying patients at risk of important below-expected progress, and improved the types of treatment plan changes made in the context of shared decision-making using a wider range of instruments (ie, VA-recommended instruments) and under implementation strategies that are feasible for a wider range of care settings.

Executive Summary Table

Summary of Findings.

Introduction

Methods

To identify articles relevant to the key questions, our research librarian searched Medline, PsycINFO, CENTRAL, and Google Scholar from 1/1/2015 through 11/16/2018, using terms for psychotherapy, feedback, and patient-reported outcomes (see Supplemental Materials for complete search strategies). We relied on the 2016 Cochrane review by Kendrick et al for identification of studies published through 2014.⁵² Additional citations were identified from hand-searching reference lists and consultation with content experts. We limited the search to published and indexed articles involving human subjects available in the English language. Study selection was based on the eligibility criteria described above. Titles and abstracts were reviewed by one investigator and checked by another (sequential review). Full-text articles were sequentially reviewed by 2 investigators. All disagreements were resolved by consensus.

We rated the internal validity of studies only for the subset of studies that used MBC approaches most relevant to the specific approach currently recommended by the VA that at least included collecting and sharing of feedback with patients. To rate the internal validity, we used a subset of items from the Cochrane’s Risk of Bias Tool,⁵⁴ that focused on randomization adequacy, balance of baseline characteristics, control for confounding variables, and adequacy of fidelity to the MBC protocol. We abstracted data from all included studies and results for each included outcome. All data abstraction and internal validity ratings were first completed by one reviewer and then checked by another. All disagreements were resolved by consensus.

We informally graded the strength of the evidence based on the AHRQ Methods Guide for Comparative Effectiveness Reviews, by considering risk of bias (includes study design and aggregate quality), consistency, directness, and precision of the evidence.⁵⁵ Ratings typically range from high to insufficient, reflecting our confidence that the evidence reflects the true effect. For this review, we applied the following general algorithm: evidence comprised primarily of RCTs with high risk of bias received ratings of ‘insufficient’; evidence consisting of a single fair- to good-quality RCT received a rating of ‘low strength’; and evidence consisting of multiple, consistent, precise, fair- to-good quality RCTs received a rating of ‘moderate strength’ or ‘high strength’. We found no ‘high-strength’ evidence, but this generally would have been comprised of multiple, good-quality, precise RCTs.

Where studies were appropriately homogenous, we synthesized outcome data quantitatively using Microsoft® Excel® for Windows (2016) to estimate pooled effects. Where meta-analysis was not suitable due to limited data or heterogeneity, we synthesized the evidence qualitatively.

The complete description of our full methods can be found on the PROSPERO international prospective register of systematic reviews (http://www.crd.york.ac.uk/PROSPERO/; registration number CRD42018107202). A draft version of this report was reviewed by peer reviewers as well as clinical leadership. Their comments and our responses are presented in the Supplemental Materials.

Results

Literature Flow

Figure 1

Literature Flowchart.

Literature Overview

Searches resulted in 807 unique and potentially relevant articles. We included 38 studies.^{1–20,34,56–72} No studies described use of explicit shared decision-making (see Figure 1 for specific criteria). Twenty-two studies were categorized as R1.^1–20,63,68 Sixteen studies were categorized as R2.^{34,56–62,64–67,69–72} For our synthesis, we focused on studies with at least a standard procedure for sharing and discussing feedback with patients (R1). For a list of ongoing and excluded studies, see Supplemental Materials. Figure 2 provides details about key study characteristics.

Figure 2

Selected Study Characteristics.

MBC Approaches and Applicability to MBC in Shared Decision-making

In general, findings from MBC studies are most applicable to populations with anxiety and/or depressive disorders as implemented into general outpatient treatment settings. Additionally, some studies focused on more specific populations including eating disorders,^10,14,19 relationship issues,^2,9 schizophrenia,¹⁶ and severe psychiatric crisis.²⁰ Although the number of MBC RCTs is increasing, the available evidence likely has unclear applicability to the specific practice of using any of the 4 VA-recommended tools for MBC in the context of shared decision-making in primary care mental health integrated care management models such as are primarily used in VA. This is because we found no studies that used an MBC approach as specifically defined by VA (ie, collect, share, and act with shared decision-making) in a care management setting, only one study in a military/Veteran population,⁶⁹ and only 2 studies that used any of the VA-recommended MBC tools (PHQ9 and GAD7).^13,15 Studies collected PROMS data, but typically either (1) combined that with guidance on how to share feedback with clients and made suggestions on how to act, which may or may not have included shared decision-making, or (2) did not provide explicit guidance on if/how to share and act on feedback. For example, in the RCT with the potential to be the most relevant to Veterans in terms of population characteristics – the only study in a military population, many of whom were returning Veterans from Iraq and Afghanistan – applicability is still limited because the MBC approach involved only giving PROMs results to therapists with no sharing with participants and no attempt to monitor therapist behavior regarding PROMs use for informing treatment planning.⁶⁹ Similarly, although a 2015 RCT by Guo et al⁶³ is frequently cited as providing strong evidence of MBC’s benefits, its findings also likely have limited applicability to MBC in shared decision-making because it was unclear whether MBC ratings were shared with the patients and clinicians made treatment decisions strictly based on an explicit and fixed dosing schedule of either paroxetine or mirtazapine, which did not appear to incorporate patient preference.⁶³

Methodological Quality

The methodological quality of the MBC literature remains low, making it difficult to attribute treatment effects to specific mechanisms of MBC. Low methodological quality is primarily due to (1) the lack of adequate information about the actual use of MBC or the quality of its implementation, (2) lack of information about potential for confounding due to between-group heterogeneity in provider characteristics (ie, experience, competence, treatment models, case mix, training, and attitudes toward MBC), patient illness severity and duration, comorbidities, previous experience with MBC, intensity of psychotherapy (ie, frequency and duration) and concomitant treatments (ie, pharmacotherapy, education, further testing), and (3) lack of use of an independent instrument to corroborate progress in the feedback group that was based on the feedback instrument alone. Only 5 RCTs reported on MBC fidelity.^3,6,18,20,68 Among those, they generally only reported on administering the PROMs (ie, administered every session in 93% of patients,³ or 67% of therapists reported applying PCOMS adequately in > 70% of sessions²⁰) and did not provide information about if and how they used the PROMs in treatment decisions. To control for provider variability, 3 studies randomized by provider^8,16,68 but typically little information was provided about patient characteristics and even less was provided about treatment type and/or intensity. Finally, as blinding the patients and therapists to whether or not they are in the feedback group in MBC studies is not feasible, there exists an inherently increased risk of more favorable outcomes in the feedback group due to expectations alone and the potential for more attention in general. Additionally, the feedback group is at risk of further favoring because patients may be extra motivated to improve when informed of scores indicating lower improvement than perceived as expected. Therefore, to better protect against this bias, use of another independent outcome measure to corroborate progress should be used as assessed by a blinded outcome assessor and without discussion by the therapist or client. However, only a single study used blinded outcome assessors¹⁹ and one study assessed outcomes using an independent instrument.³

Strength of Evidence

Our confidence in the strength of the findings on MBC’s effects is generally low because studies did not directly assess MBC as defined by the VA, they had serious methodological weaknesses as discussed above, and most MBC approaches were evaluated by single-study evidence bases (unknown consistency in direction and/or magnitude of effect).

Key Question 1. What is the effectiveness of measurement based care delivery practices in mental health care?

Clinically Significant Improvement in Overall Distress

MBC’s effects are mixed across studies that evaluated approaches that included collect and share components and encouraged but did not monitor acting on results (Table 1). For example, among the 14 studies that reported rates of patients with a clinically meaningful response, 57% of MBC approaches resulted in statistically significantly improved outcomes.^{1–3,8–10,12,63} However, determining what MBC conditions are most effective was difficult due to heterogeneity across studies in multiple patient, provider, setting, and implementation approach factors.

Among the studies demonstrating clinically important improvements, 2 studies stand out as providing the strongest support for MBC.^3,63 First is a 2018 RCT of 170 mostly women with mood and anxiety disorders seen in a hospital-based outpatient psychiatric clinic in Norway.³ In this RCT, under ideal implementation circumstances, MBC resulted in a small but significant improvement in outcomes over treatment as usual (d = 0.26), with improvements increasing over time. The advantage remained even after adjustments for therapist variability and clients’ pretreatment distress levels. What makes this possibly the best evidence we have are the following important strengths of this RCT: (1) high PCOMS measure administration fidelity was documented (93%), (2) an independent measure of symptom and function was used to assess outcome, (3) therapists were regularly trained and supervised (ie, obligatory 1-day face-to-face training twice a year given by developers of PCOMS system and training and supervision sessions once each month), (4) patients’ diagnoses were reliable based on use of a valid and standardized tool (M.I.N.I International Neuropsychiatric Interview), and (5) some potential confounding was minimized through adjustment for therapist variability and clients’ pretreatment distress levels. However, the strength of these findings is still limited by important methodological weaknesses common to this body of evidence. Weaknesses include not adequately minimizing other sources of potential bias including variation between groups in specific types or dose of psychotherapy, medical treatment, or treatment outside the clinic, if/how PCOMS results were discussed, if/how subsequent treatment changes were made, and the lack of blinding of outcome assessors. Additionally, this RCT involved use of a highly intensive implementation strategy that included highly trained experts who received extensive training provided by PCOMS creators and extensive implementation support that included minimization of provider paperwork burden through use of technology-assisted automated risk scoring and management that advocated PCOMS and provided moral and financial support. It is unclear whether implementation of the intensive implementation strategy used in this RCT would be feasible in more typical clinical settings with fewer implementation resources.

Second is the 2015 RCT by Guo et al,⁶³ which is frequently cited as providing strong evidence of MBC’s benefits. This RCT of 120 outpatients with moderate to severe major depressive disorder treated with a fixed dosing schedule of paroxetine or mirtazapine at a university-affiliated teaching hospital in China found that after 24 weeks, MBC with the Quick Inventory of Depressive Symptomatology-Self-Report (QIDS-SR) led to significantly greater rates of remission than usual care (Chinese version of the 17-item Hamilton Depression Rating Scale [HAM-D] score ≤ 7, 73.8% vs 28.8%, P<0.0001). The greatest strengths of this study are that it is the best example we identified of a design that better isolates MBC’s effects through explicit documentation of the stepped-care treatment algorithm used, which included specific medication choices. It also took measures to minimize performance and measurement biases through using blind raters to assess outcome and ensured high fidelity to MBC protocol via external compliance monitoring. However, its findings likely have limited applicability to the specific approach of using MBC in shared decision-making because it was unclear whether MBC ratings were shared with the patients, and clinicians made treatment decisions strictly based on an explicit and fixed dosing schedule of either paroxetine or mirtazapine, which did not appear to incorporate patient preference.

Suicide Behavior, Functioning, Quality of Life

The effects of MBC on suicide behavior, functioning, and quality of life are largely unknown. We did not identify any studies that reported on suicide behavior and real-life functional outcomes (eg, days missed from work). MBC significantly improved quality of life in only one¹⁶ of 3 studies that assessed quality of life outcomes using various instruments (eg, MANSA= Manchester Short Assessment of Quality of Life,¹⁶ EQ-5D-5L= EuroQol Quality of Life Scale,¹⁵ and WHO-5 score = WHO-Five Well-Being Index).¹⁴ However, serious methodological weaknesses – primarily lack of MBC fidelity assessment – preclude reaching any conclusions based on this evidence. Also, as the findings of improved quality of life came from a population with a primary diagnosis of schizophrenia or related psychotic disorder, they have unclear applicability to patients with more commonly diagnosed mental health conditions, such as mood and anxiety disorders.

Satisfaction with Care

Evidence on satisfaction with care is very limited.^15–17 Among patients with schizophrenia or related psychotic disorders, MBC significantly improved treatment satisfaction after 12 months as measured using the Client Satisfaction Questionnaire (CSQ) (adjusted mean 25.99 points vs 25.15, adjusted mean difference 0.92, 95% CI 0.22 to 1.56).¹⁶ However, in patients with more commonly diagnosed mental health conditions, MBC either did not change satisfaction with care¹⁷ or actually decreased patient satisfaction.¹⁵

No-shows, Drop-outs, Medication Adherence

There is little evidence to suggest that MBC improves care processes. No RCT reported medication use and/or adherence outcomes or no-show rates. Among 4 RCTs that reported attendance outcomes, all consistently found that MBC did not improve rates of attendance.^10,14,18,19

Table 1

Characteristics of Studies with Feedback to Patient and Clinician and Discussion.

Summary and Discussion

This rapid review built on previous evidence synthesis work^23,45,52 by adding 14 new studies and identifying the delivery circumstances under which MBC has the most reliable evidence of operating most effectively. Our review found that MBC can lead to clinically meaningful improvements in patient outcomes under certain circumstances. The most reliable evidence^3,6,18,20 points to use of the PCOMS to routinely monitor outcomes in outpatient treatment of anxiety and mood disorders as the most promising approach when paired with a high-intensity implementation strategy (clinically significant improvement for MBC vs usual care: 48% vs 33%, OR 1.91, 95% CI 0.88 to 4.15, P = 0.1025, NNT= 7).³ As the effectiveness of this MBC approach was demonstrated to increase over time, likely it was the intense and sustained implementation efforts – which appeared more intense than in other applications – that led to its success. PCOMS and the OQ-45 have also shown some promise for use in other diagnostic subgroups including couples therapy^13,32 and inpatient treatment of eating disorders,¹⁰ but not for improving specific symptoms of schizophrenia,¹⁶ or for patients in severe psychiatric crisis seeking emergency help.²⁰

Our findings differ somewhat from previous reviews^23,45,52 and this is likely due to differences in scope and methodology. For example, our findings are less favorable compared to the 2017 review by Fortney et al²³ which broadly stated that “virtually all randomized controlled trials with frequent and timely feedback of patient-reported symptoms to the provider during the medication management and psychotherapy encounters significantly improved outcomes.” While we agree that some randomized controlled trials did demonstrate improvement, we disagree with the implication that all improvements are equally clinically meaningful, reliable, and universally applicable. Alternatively, while we completely agree with the suggestion from the 2016 Cochrane review by Kendrick et al that “more research of better quality is needed”, our findings are slightly more positive than theirs, which broadly stated that “we found insufficient evidence to support the use of routine outcome monitoring using PROMs in the treatment of common mental health disorders in terms of improving patient outcomes or in improving management.”⁵² This is largely because the RCT we highlighted above as potentially representing the most promising MBC approach was not yet available at the time of the 2016 Cochrane review.

Conclusions

Effectiveness of the specific VA-recommended approach of using any of 4 recommended PROMs for implementing MBC in the context of shared decision-making in mental health is unknown. We identified other promising approaches to use of PROMs for MBC in general mental health settings, but raise important questions about their applicability and implementation feasibility into heterogenous VA primary care mental health integrated care settings. New research would be more meaningful if it evaluated the specific VA-recommended MBC approach, improved on identified methodological limitations, evaluated a wider range of clinically meaningful outcomes, and simultaneously compared MBC use under 2 or more implementation strategies that are feasible for a wider range of care settings.

Acknowledgments

This topic was developed in response to a nomination by the Office of Mental Health and Suicide Prevention (OMHSP) for the purpose of reviewing the evidence on measurement based care in mental health. The scope was further developed with input from the topic nominators (ie, Operational Partners), the ESP Coordinating Center, and the review team.

The authors gratefully acknowledge Julia Haskin, MS for editorial support, Sam Aldape for citation management support, Katherine Mackey, MD, MPP for clinical review and the following individuals for their contributions to this project:

References

1.

Amble I, GudeT, StubdalS, AndersenBJ, WampoldBE. The effect of implementing the outcome questionnaire-45.2 feedback system in norway: A multisite randomized clinical trial in a naturalistic setting. Psychotherapy Research. 2014;25(6):1–9. [PubMed: 25101527]

2.

Anker MG, DuncanBL, SparksJA. Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology. 2009;77(4):693–704. [PubMed: 19634962]

3.

Brattland H, KoksvikJM, BurkelandO, et al. The effects of routine outcome monitoring (ROM) on therapy outcomes in the course of an implementation process: A randomized clinical trial. J Couns Psychol. 2018;65(5):641–652. [PubMed: 30113180]

4.

De Jong K, TimmanR, Hakkaart-Van RoijenL, et al. The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research. 2014;24(6):1–11. [PubMed: 24386975]

5.

Hawkins EJ, LambertMJ, VermeerschDA, SladeKL, TuttleKC. The therapeutic effects of providing patient progress information to therapists and patients. Psychotherapy Research. 2004;14(3):308–327.

6.

Kellybrew-Miller A. The impact of systematic client feedback on client outcomes in a community mental health center. Dissertation Abstracts International: Section B: The Sciences and Engineering. 2017;77(7-B(E)):No Pagination Specified.

7.

Murphy KP, RashleighCM, TimulakL. The relationship between progress feedback and therapeutic outcome in student counselling: A randomised control trial. Counselling Psychology Quarterly. 2012;25(1):1–18.

8.

Reese RJ, NorsworthyLA, RowlandsSR. Does a continuous feedback system improve psychotherapy outcome?Psychotherapy: Theory, Research, Practice, Training. 2009;46(4):418–431. [PubMed: 22121836]

9.

Reese RJ, TolandMD, SloneNC, NorsworthyLA. Effect of client feedback on couple psychotherapy outcomes. Psychotherapy: Theory, Research, Practice, Training. 2010;47(4):616–630. [PubMed: 21198247]

10.

Simon W, LambertM, BusathG, et al. Effects of providing patient progress feedback and clinical support tools to psychotherapists in an inpatient eating disorders treatment program: A randomized controlled study. Psychotherapy Research. 2013;23(3):287–300. [PubMed: 23656489]

11.

Simon W, LambertMJ, HarrisMW, BusathG, VazquezA. Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure. Psychotherapy Research. 2012;22(6):1–10. [PubMed: 22755547]

12.

Slone NC, ReeseRJ, Mathews-DuvallS, KodetJ. Evaluating the efficacy of client feedback in group psychotherapy. Group Dynamics: Theory, Research, and Practice. 2015;19(2):122.

13.

Delgadillo J, de JongK, LucockM, et al. “Feedback-informed treatment versus usual psychological treatment for depression and anxiety: A multisite, open-label, cluster randomised controlled trial”: Correction. The Lancet Psychiatry. 2018;5(8):e17. [PubMed: 29937396]

14.

Davidsen AH, PoulsenS, LindschouJ, et al. Feedback in group psychotherapy for eating disorders: A randomized clinical trial. Journal of consulting and clinical psychology. 2017;85(5):484–494. [PubMed: 28333513]

15.

Kendrick T, StuartB, LeydonGM, et al. Patient-reported outcome measures for monitoring primary care patients with depression: Promdep feasibility randomised trial. BMJ Open. 2017;7(3):e015266. [PMC free article: PMC5387943] [PubMed: 28363932]

16.

Priebe S, McCabeR, BullenkampJ, et al. Structured patient-clinician communication and 1-year outcome in community mental healthcare: Cluster randomised controlled trial. The British journal of psychiatry: the journal of mental science. 2007;191:420. [PubMed: 17978322]

17.

Rise MB, EriksenL, GrimstadH, SteinsbekkA. The long-term effect on mental health symptoms and patient activation of using patient feedback scales in mental health out-patient treatment. A randomised controlled trial. Patient Education & Counseling. 2016;99(1):164–168. [PubMed: 26227577]

18.

McClintock AS, PerlmanMR, McCarrickSM, AndersonT, HimawanL. Enhancing psychotherapy process with common factors feedback: A randomized, clinical trial. Journal of Counseling Psychology. 2017;64(3):247–260. [PubMed: 28114769]

19.

Schmidt U, LandauS, Pombo-CarrilMG, et al. Does personalized feedback improve the outcome of cognitive-behavioural guided self-care in bulimia nervosa? A preliminary randomized controlled trial. British Journal of Clinical Psychology. 2006;45(1):111–121. [PubMed: 16480570]

20.

van Oenen FJ, SchipperS, VanR, et al. Feedback-informed treatment in emergency psychiatry; a randomised controlled trial. BMC Psychiatry. 2016;16:110. [PMC free article: PMC4837581] [PubMed: 27095106]

21.

Hatfield D, McCulloughL, FrantzSH, KriegerK. Do we know when our clients get worse? An investigation of therapists’ ability to detect negative client change. Clin Psychol Psychother. 2010;17(1):25–32. [PubMed: 19916162]

22.

Wray LO, RitchieMJ, OslinDW, BeehlerGP. Enhancing implementation of measurement-based mental health care in primary care: A mixed-methods randomized effectiveness evaluation of implementation facilitation. BMC Health Serv Res. 2018;18(1):753. [PMC free article: PMC6171308] [PubMed: 30285718]

23.

Fortney JC, UnützerJ, WrennG, et al. A tipping point for measurement-based care. Psychiatric Services. 2017;68(2):179–188. [PubMed: 27582237]

24.

Landes SJ, CarlsonEB, RuzekJI, et al. Provider-driven development of a measurement feedback system to enhance measurement-based care in VA mental health. Cognitive and Behavioral Practice. 2015;22(1):87–100.

25.

Carlier IV, MeuldijkD, Van VlietIM, Van FenemaE, Van der WeeNJ, ZitmanFG. Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. J Eval Clin Pract. 2012;18(1):104–110. [PubMed: 20846319]

26.

Riemer M, Rosof-WilliamsJ, BickmanL. Theories related to changing clinician practice. Child Adolesc Psychiatr Clin N Am. 2005;14(2):241–254, viii. [PubMed: 15694784]

27.

Department of Defense, Department of Veterans Affairs, Department of Health and Human Services. Interagency task force on military and veterans mental health: 2013 annual report. In. Washington, DC2016.

28.

Department of Defense, Department of Veterans Affairs, Department of Health and Human Services. Interagency task force on military and veterans mental health: 2013 annual report. In. Washington, DC2013.

29.

Miller SD, DuncanBL, SorrellR, BrownGS. The partners for change outcome management system. J Clin Psychol. 2005;61(2):199–208. [PubMed: 15609362]

30.

Gillaspy J, MurphyJJ. Incorporating outcome and session rating scales in solution-focused brief therapy In: Solution-focused brief therapy: A handbook of evidence-based practice. Oxford: Oxford University Press; 2011.

31.

Duncan BL, ReeseRJ. The partners for change outcome management system (pcoms) revisiting the client’s frame of reference. Psychotherapy: Theory, Research, Practice, Training. 2015;52(4):391–401. [PubMed: 26641369]

32.

Lambert MJ, ShimokawaK. Collecting client feedback. Psychotherapy. 2011;48(1):72–79 [PubMed: 21401277]

33.

Miller SD, BargmannS. The outcome rating scale (ORS) and the session rating scale (SRS)Integrating Science and Practice. 2012;2(2):38–31.

34.

Lambert MJ, WhippleJL, SmartDW, VermeerschDA, NielsenSL, HawkinsEJ. The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced?Psychotherapy Research. 2001;11(1):49–68. [PubMed: 25849877]

35.

Lambert MJ, FinchAE. The outcome questionnaire. In: MaruishME, ed. The use of psychological testing for treatment planning and outcomes assessment. NJ: Erlbaum1999:831–869.

36.

Lambert MJ, WhippleJL, VermeerschDA, et al. Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clinical Psychology & Psychotherapy. 2002;9(2):91–103.

37.

Mavandadi S, BensonA, DiFilippoS, StreimJE, OslinD. A telephone-based program to provide symptom monitoring alone vs symptom monitoring plus care management for late-life depression and anxiety: A randomized clinical trial. JAMA psychiatry. 2015;72(12):1211–1218. [PubMed: 26558530]

38.

Bruce ML, Ten HaveTR, ReynoldsCF, 3rd, et al. Reducing suicidal ideation and depressive symptoms in depressed older primary care patients: A randomized controlled trial. Jama. 2004;291(9):1081–1091. [PubMed: 14996777]

39.

Unutzer J, KatonW, CallahanCM, et al. Collaborative care management of late-life depression in the primary care setting: A randomized controlled trial. Jama. 2002;288(22):2836–2845. [PubMed: 12472325]

40.

Excellence NIfHaC. Quality standards for the management of depression in adults. 2011; https://www​.nice.org.uk/guidance/qs8. Accessed October, 2018.

41.

Institute of Medicine. Treatment for posttraumatic stress disorder in military and veteran populations: Initial assessment.. Washington, DC: National Academies Press2012 [PubMed: 24830058]

42.

Resnick SH, Rani. The measurement based care (MBC) in mental health initiative in VHA. VA Psychology Leadership Conference2017; https://avapl​.org/conference​/pubs/2017%20Conference​%20Presentations​/Day%201/AVAPL​%20MBC%20Resnick%20Hoff.pdf. Accessed October 11th, 2018.

43.

The Joint Commission. New outcome measures standard https://www​.jointcommission​.org/accreditation​/bhc_new_outcome_measures_standard​.aspx. Accessed October 18th, 2018.

44.

Boswell JF, KrausDR, MillerSD, LambertMJ. Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research. 2015;25(1):6–19. [PubMed: 23885809]

45.

Krägeloh CU, CzubaKJ, BillingtonDR, KerstenP, SiegertRJ. Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatric Services. 2015;66(3):224–241. [PubMed: 25727110]

46.

Greenhalgh J. The applications of pros in clinical practice: What are they, do they work, and why?Qual Life Res. 2009;18(1):115–123. [PubMed: 19105048]

47.

Krageloh CU, CzubaKJ, BillingtonD, KerstenP, SiegertRJ. Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatric Services. 2015;66(3):224–241. [PubMed: 25727110]

48.

Oslin DW, HoffR, MignognaJ, ResnickSG. Provider attitudes and experience with measurement-based mental health care in the VA implementation project. Psychiatr Serv. 2018:appips201800228. [PubMed: 30373495]

49.

Callaly T, HylandM, CoombsT, TrauerT. Routine outcome measurement in public mental health: Results of a clinician survey. Aust Health Rev. 2006;30(2):164–173. [PubMed: 16646765]

50.

Leydon GM, DowrickCF, McBrideAS, et al. Questionnaire severity measures for depression: A threat to the doctor-patient relationship?The British journal of general practice: the journal of the Royal College of General Practitioners. 2011;61(583):117. [PMC free article: PMC3026150] [PubMed: 21276338]

51.

Dowrick C, LeydonGM, McBrideA, et al. Patients’ and doctors’ views on depression severity questionnaires incentivised in uk quality and outcomes framework: Qualitative study. BMJ. 2009;338. [PubMed: 19299474]

52.

Kendrick T, El-GoharyM, StuartB, et al. Routine use of patient reported outcome measures (proms) for improving treatment of common mental health disorders in adults. Cochrane Database of Systematic Reviews. 2016;7:CD011119. [PMC free article: PMC6472430] [PubMed: 27409972]

53.

Lambert MJ, WhippleJL, KleinstauberM. Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy. 2018;55(4):520–537. [PubMed: 30335463]

54.

Higgins JP, AltmanDG, GotzschePC, et al. The cochrane collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. [PMC free article: PMC3196245] [PubMed: 22008217]

55.

Berkman ND, LohrKN, AnsariM, et al. Grading the strength of a body of evidence when assessing health care interventions for the effective health care program of the agency for healthcare research and quality: An update methods guide for effectiveness and comparative effectiveness reviews. In. Rockville MD: Agency for Healthcare Research and Quality; 2013.

56.

Brodey BB, CuffelB, McCullochJ, et al. The acceptability and effectiveness of patient-reported assessments and feedback in a managed behavioral healthcare setting. The American journal of managed care. 2005;11(12):774. [PubMed: 16336061]

57.

Chamberlin CW. The effective professional: The adoption and use of a feedback system in psychotherapy. Dissertation Abstracts International: Section B: The Sciences and Engineering. 2016;77(6-B(E)):No Pagination Specified.

58.

Cheyne A, KinnS. A pilot study for a randomised controlled trial of the use of the schedule for the evaluation of individual quality of life (seiqol) in an alcohol counselling setting. Addiction Research & Theory, 2001, Vol9(2), p165-178. 2001;9(2):165–178.

59.

Connolly Gibbons MB, KurtzJE, ThompsonDL, et al. The effectiveness of clinician feedback in the treatment of depression in the community mental health system. Journal of Consulting and Clinical Psychology. 2015;83(4):748–759. [PMC free article: PMC4516646] [PubMed: 26052874]

60.

Davidson KM, RankinML, BegleyA, et al. Assessing patient progress in psychological therapy through feedback in supervision: The memos* randomized controlled trial (*measuring and monitoring clinical outcomes in supervision: Memos). Behavioural and Cognitive Psychotherapy. 2017;45(3):209–224. [PubMed: 28166849]

61.

De Jong K, Van SluisP, NugterMA, HeiserWJ, SpinhovenP. Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research. 2012;22(4):464–474. [PubMed: 22468992]

62.

Errazuriz P, Zilcha-ManoS. In psychotherapy with severe patients discouraging news may be worse than no news: The impact of providing feedback to therapists on psychotherapy outcome, session attendance, and the alliance. Journal of Consulting and Clinical Psychology. 2018;86(2):125–139. [PubMed: 29265833]

63.

Guo T, XiangY-T, XiaoL, et al. Measurement-based care versus standard care for major depression: A randomized controlled trial with blind raters. American Journal of Psychiatry. 2015;172(10):1004–1013. [PubMed: 26315978]

64.

Hansson H, RundbergJ, ÖsterlingA, ÖjehagenA, BerglundM. Intervention with feedback using outcome questionnaire 45 (OQ-45) in a swedish psychiatric outpatient population. A randomized controlled trial. Nordic Journal of Psychiatry. 2013;67(4):274–281. [PubMed: 23157691]

65.

Lutz W, RubelJ, SchiefeleA-K, ZimmermannD, BohnkeJR, WittmannWW. Feedback and therapist effects in the context of treatment outcome and treatment length. Psychotherapy Research. 2015;25(6):647–660. [PubMed: 26218788]

66.

Probst T, LambertMJ, DahlbenderRW, LoewTH, TrittK. Providing patient progress feedback and clinical support tools to therapists: Is the therapeutic process of patients on-track to recovery enhanced in psychosomatic in-patient therapy under the conditions of routine practice?Journal of Psychosomatic Research. 2014;76(6). [PubMed: 24840143]

67.

Probst T, LambertMJ, LoewTH, DahlbenderRW, GöllnerR, TrittK. Feedback on patient progress and clinical support tools for therapists: Improved outcome for patients at risk of treatment failure in psychosomatic in-patient therapy under the conditions of routine practice. Journal of Psychosomatic Research. 2013;75(3):255–261. [PubMed: 23972415]

68.

Puschner B, SchöferD, KnaupC, BeckerT. Outcome management in in patient psychiatric care. Acta Psychiatrica Scandinavica. 2009;120(4):308–319. [PubMed: 19485963]

69.

Schuman DL, SloneNC, ReeseRJ, DuncanB. Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment. Psychother Res. 2015;25(4):396–407. [PubMed: 24708386]

70.

She Z, DuncanBL, ReeseRJ, et al. Client feedback in china: A randomized clinical trial in a college counseling center. Journal of Counseling Psychology. 2018:No Pagination Specified. [PubMed: 30058829]

71.

Slade M, McCroneP, KuipersE, et al. Use of standardised outcome measures in adult mental health services: Randomised controlled trial. The British journal of psychiatry: the journal of mental science. 2006;189:330. [PubMed: 17012656]

72.

Trudeau LS. Effects of a clinical feedback system on client and therapist outcomes in a rural community mental health center. In: Digital Repository @ Iowa State University; 2000.

73.

Guise JM, ChangC, ViswanathanM, et al. Agency for healthcare research and quality evidence-based practice center methods for systematically reviewing complex multicomponent health care interventions. J Clin Epidemiol. 2014;67(11):1181–1191. [PubMed: 25438663]

74.

Goldstein L, Connolly GibbonsM, ThompsonS, et al. Outcome assessment via handheld computer in community mental health: Consumer satisfaction and reliability. The Journal of Behavioral Health Services & Research. 2011;38(3):414–423. [PMC free article: PMC3299491] [PubMed: 21107916]

Search Strategies

List of Excluded Studies

Exclude reasons: 1=Ineligible population (eg, patients receiving mental health care not separately evaluated), 2=Ineligible intervention (eg, not patient reported outcome measures, MBC as part of a more intensive collaborative care/care management/integrated care approach), 3=Ineligible comparator (eg, not shared decision making or usual care without an MBC component), 4=Ineligible outcome (eg, patient preferences or implementation experiences), 5=Ineligible setting, 6=Ineligible study design (eg, case report), 7=Ineligible publication type (eg, editorial, narrative review), 8=Outdated or ineligible systematic review, 9=non-English language, S=non-RCT meeting other criteria

Evidence Tables

Ongoing Studies

Peer Review

References

1.

Aardoom JJ, DingemansAE, van GinkelJR, SpinhovenP, Van FurthEF, Van den Akker-van MarleME. Cost-utility of an internet-based intervention with or without therapist support in comparison with a waiting list for individuals with eating disorder symptoms: A randomized controlled trial. International journal of eating disorders. 2016;49(12):1068–1076. [PubMed: 27441418]

2.

Amble I, GudeT, UlvenesP, StubdalS, WampoldBE. How and when feedback works in psychotherapy: Is it the signal?Psychotherapy Research. 2015;26(5):545–555. [PubMed: 26169948]

3.

Bargmann S. Achieving excellence through feedback-informed supervision. In: Feedback-informed treatment in clinical practice: Reaching for excellence. Washington, DC: American Psychological Association; US; 2017:79–100.

4.

Berking M, OrthU, LutzW. Wie effektiv sind systematische rückmeldungen des therapieverlaufs an den therapeuten? - how effective is systematic feedback of treatment progress to the therapist? An empirical study in a cognitive-behaviouralorieted impatient setting. Zeitschrift für Klinische Psychologie und Psychotherapie. 2006;35(1):21–29.

5.

Bickman L, KelleySD, BredaC, de AndradeAR, RiemerM. Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services. 2011;62(12):1423–1429. [PubMed: 22193788]

6.

Bilsker D, GoldnerEM. Routine outcome measurement by mental health-care providers: Is it worth doing?The Lancet. 2002;360(9346):1689–1690. [PubMed: 12457807]

7.

Black SW, OwenJ, ChapmanN, LavinK, DrinaneJM, KuoP. Feedback informed treatment: An empirically supported case study of psychodynamic treatment. Journal of Clinical Psychology. 2017;73(11):1499–1509. [PubMed: 29044599]

8.

Brodey BB, GonzalezNL, ElkinKA, SasielaWJ, BrodeyIS. Assessing the equivalence of paper, mobile phone, and tablet survey responses at a community mental health center using equivalent halves of a ‘gold-standard’ depression item bank. JMIR Mental Health. 2017;4(3):e36. [PMC free article: PMC5607438] [PubMed: 28877861]

9.

Brown GS, SimonA, CameronJ, MinamiT. A collaborative outcome resource network (acorn): Tools for increasing the value of psychotherapy. Psychotherapy. 2015;52(4):412–421. [PubMed: 26641371]

10.

Burlingame GM, WhitcombKE, WoodlandSC, OlsenJA, BeecherM, GleaveR. The effects of relationship and progress feedback in group psychotherapy using the group questionnaire and outcome questionnaire-45: A randomized clinical trial. Psychotherapy. 2018;55(2):116–131. [PubMed: 29863392]

11.

Chan AT, SunGY, TamWW, TsoiKK, WongSY. The effectiveness of group-based behavioral activation in the treatment of depression: An updated meta-analysis of randomized controlled trial. Journal of Affective Disorders. 2017;208:345–354. [PubMed: 27810717]

12.

Chang TE, JingY, YeungAS, et al. Effect of communicating depression severity on physician prescribing patterns: Findings from the clinical outcomes in measurement-based treatment (comet) trial. General Hospital Psychiatry. 2012;34(2):105–112. [PubMed: 22264654]

13.

Cross S, Mellor-ClarkJ, MacdonaldJ. Tracking responses to items in measures as a means of increasing therapeutic engagement in clients: A complementary clinical approach to tracking outcomes. Clinical Psychology & Psychotherapy. 2015;22(6):698–707. [PubMed: 25402715]

14.

Davidson K, PerryA, BellL. Would continuous feedback of patient’s clinical outcomes to practitioners improve nhs psychological therapy services? Critical analysis and assessment of quality of existing studies. Psychology and Psychotherapy: Theory, Research and Practice. 2015;88(1):21–37. [PubMed: 24850588]

15.

Delgadillo J, OverendK, LucockM, et al. Improving the efficiency of psychological treatment using outcome feedback technology. Behaviour Research & Therapy. 2017;99:89–97. [PubMed: 29024821]

16.

Drummond KL, PainterJT, CurranGM, et al. Hiv patient and provider feedback on a telehealth collaborative care for depression intervention. AIDS Care. 2017;29(3):290–298. [PubMed: 27871183]

17.

Duncan BL, ReeseRJ. The partners for change outcome management system (pcoms) revisiting the client’s frame of reference. Psychotherapy: Theory, Research, Practice, Training. 2015;52(4):391–401. [PubMed: 26641369]

18.

Dyer K, HookeGR, PageAC. Effects of providing domain specific progress monitoring and feedback to therapists and patients on outcome. Psychotherapy Research. 2014;26(3):297–306. [PubMed: 25506654]

19.

Eisen SV, DickeyB, SedererLI. A self-report symptom and problem rating scale to increase inpatients’ involvement in treatment. Psychiatric Services. 2000;51(3):349–353. [PubMed: 10686242]

20.

Faurholt-Jepsen M, FrostM, MartinyK, et al. Reducing the rate and duration of re-admissions among patients with unipolar disorder and bipolar disorder using smartphone-based monitoring and treatment - the radmis trials: Study protocol for two randomized controlled trials. Trials [Electronic Resource]. 2017;18(1):277. [PMC free article: PMC5472886] [PubMed: 28619114]

21.

Fihn SD, McDonellMB, DiehrP, et al. Effects of sustained audit/feedback on self-reported health status of primary care patients. The American Journal of Medicine. 2004;116(4):241–248. [PubMed: 14969652]

22.

Fortney JC, UnützerJ, WrennG, et al. A tipping point for measurement-based care. Psychiatric Services. 2017;68(2):179–188. [PubMed: 27582237]

23.

Fridberg DJ, CaoD, KingAC. Integrating alcohol response feedback in a brief intervention for young adult heavy drinkers who smoke: A pilot study. Drug and alcohol dependence. 2015;155:293–297. [PMC free article: PMC4582006] [PubMed: 26341847]

24.

Friedhoff LA. Question development by individuals in therapeutic assessment: Does it result in more positive outcomes?Dissertation Abstracts International: Section B: The Sciences and Engineering. 2013;75(8-B(E)):No Pagination Specified.

25.

Gondek D, Edbrooke-ChildsJ, FinkE, DeightonJ, WolpertM. Feedback from outcome measures and treatment effectiveness, treatment efficiency, and collaborative practice: A systematic review. Administration and Policy in Mental Health and Mental Health Services Research. 2016;43(3):325–343. [PMC free article: PMC4831994] [PubMed: 26744316]

26.

Haland AT, TildenT. Lessons learned from the implementation of a feedback system in couple and family therapy. In: Routine outcome monitoring in couple and family therapy: The empirically informed therapist. Cham, Switzerland: Springer International Publishing; Switzerland; 2017:211–224.

27.

Hamann J, ParchmannA, SassenbergN, et al. Training patients with schizophrenia to share decisions with their psychiatrists: A randomized-controlled trial. Social Psychiatry and Psychiatric Epidemiology. 2017;52(2):175–182. [PubMed: 28040825]

28.

Harmon SC, LambertMJ, SmartDM, et al. Enhancing outcome for potential treatment failures: Therapist–client feedback and clinical support tools. Psychotherapy Research. 2007;17(4):379–392.

29.

Hartmann JA, WichersM, Menne-LothmannC, et al. Experience sampling-based personalized feedback and positive affect: A randomized controlled trial in depressed patients. PLoS ONE. 2015;10(6):e0128095. [PMC free article: PMC4452775] [PubMed: 26034983]

30.

Hooke GR, SngAA, CunninghamNK, PageAC. Methods of delivering progress feedback to optimise patient outcomes: The value of expected treatment trajectories. Cognitive Therapy and Research. 2018;42(2):204–211.

31.

Janse PD, De JongK, Van DijkMK, HutschemaekersGJ, VerbraakMJ. Improving the efficiency of cognitive-behavioural therapy by using formal client feedback. Psychotherapy Research. 2017;27(5):525–538. [PubMed: 27013204]

32.

Jensen-Doss A, HaimesEM, SmithAM, et al. Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health and Mental Health Services Research. 2018;45(1):48–61. [PMC free article: PMC5495625] [PubMed: 27631610]

33.

Jolley S, OnwumereJ, BissoliS, et al. A pilot evaluation of therapist training in cognitive therapy for psychosis: Therapy quality and clinical outcomes. Behavioural and Cognitive Psychotherapy. 2015;43(4):478–489. [PubMed: 24360498]

34.

Kendrick T, El-GoharyM, StuartB, et al. Routine use of patient reported outcome measures (proms) for improving treatment of common mental health disorders in adults. Cochrane Database of Systematic Reviews. 2016;7:CD011119. [PMC free article: PMC6472430] [PubMed: 27409972]

35.

Khdour HY, AbushalbaqOM, MughrabiIT, et al. Generalized anxiety disorder and social anxiety disorder, but not panic anxiety disorder, are associated with higher sensitivity to learning from negative feedback: Behavioral and computational investigation. Frontiers in Integrative Neuroscience. 2016;10:20. [PMC free article: PMC4925696] [PubMed: 27445719]

36.

Klundt JS. Are therapists using outcome measures and does it matter? A naturalistic usage study. Dissertation Abstracts International: Section B: The Sciences and Engineering. 2015;76(3-B(E)):No Pagination Specified.

37.

Knaup C, KoestersM, SchoeferD, BeckerT, PuschnerB. Effect of feedback of treatment outcome in specialist mental healthcare: Meta-analysis. The British journal of psychiatry: the journal of mental science. 2009;195(1):15. [PubMed: 19567889]

38.

Koementas-de Vos MM, NugterM, EngelsbelF, De JongK. Does progress feedback enhance the outcome of group psychotherapy?Psychotherapy. 2018;55(2):151–163. [PubMed: 29863395]

39.

Krägeloh CU, CzubaKJ, BillingtonDR, KerstenP, SiegertRJ. Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatric Services. 2015;66(3):224–241. [PubMed: 25727110]

40.

Lambert MJ, WhippleJL, VermeerschDA, et al. Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clinical Psychology & Psychotherapy. 2002;9(2):91–103.

41.

Lambert MJ, ShimokawaK. Collecting client feedback. Psychotherapy. 2011;48(1):72–79 [PubMed: 21401277]

42.

Lambert MJ. Maximizing psychotherapy outcome beyond evidence-based medicine. Psychotherapy and Psychosomatics. 2017;86(2):80–89. [PubMed: 28183083]

43.

Lutz W, ZimmermannD, MullerV, DeisenhoferAK, RubelJA. Randomized controlled trial to evaluate the effects of personalized prediction and adaptation tools on treatment outcome in outpatient psychotherapy: Study protocol. BMC Psychiatry. 2017;17(1):306. [PMC free article: PMC5571503] [PubMed: 28836954]

44.

Maeschalck CL, BarfknechtLR. Using client feedback to inform treatment. In: Feedback-informed treatment in clinical practice: Reaching for excellence. Washington, DC: American Psychological Association; US; 2017:53–77.

45.

Mathias SD, FiferSK, MazonsonPD, LubeckDP, BueschingDP, PatrickDL. Necessary but not sufficient: The effect of screening and feedback on outcomes of primary care patients with untreated anxiety. Journal of general internal medicine. 1994;9(11):606. [PubMed: 7853069]

46.

Metz MJ, FranxGC, VeerbeekMA, de BeursE, van der Feltz-CornelisCM, BeekmanAT. Shared decision making in mental health care using routine outcome monitoring as a source of information: A cluster randomised controlled trial. BMC Psychiatry. 2015;15:313. [PMC free article: PMC4678650] [PubMed: 26666295]

47.

Mikeal CW, GillaspyJ, ScolesMT, MurphyJJ. A dismantling study of the partners for change outcome management system. Journal of Counseling Psychology. 2016;63(6):704–709. [PubMed: 27513088]

48.

Miller SD, BargmannS, ChowD, SeidelJ, MaeschalckC. Feedback-informed treatment (fit): Improving the outcome of psychotherapy one person at a time. In: Quality improvement in behavioral health. Cham, Switzerland: Springer International Publishing; Switzerland; 2016:247–262.

49.

Newnham EA, HookeGR, PageAC. Progress monitoring and feedback in psychiatric care reduces depressive symptoms. Journal of Affective Disorders. 2010;127(1):139–146. [PubMed: 20542574]

50.

Priebe S, McCabeR, BullenkampJ, et al. The impact of routine outcome measurement on treatment processes in community mental health care: Approach and methods of the mecca study. Epidemiologia e psichiatria sociale. 2002;11(3):198. [PubMed: 12451967]

51.

Rollman BL, HanusaBH, LoweHJ, GilbertT, KapoorWN, SchulbergHC. A randomized trial using computerized decision support to improve treatment of major depression in primary care. Journal of General Internal Medicine. 2002;17(7):493–503. [PMC free article: PMC1495078] [PubMed: 12133139]

52.

Schiepek G, EckertH, AasB, WallotS, WallotA. Integrative psychotherapy: A feedback-driven dynamic systems approach. Boston, MA: Hogrefe Publishing; US; 2015.

53.

Scott K, LewisCC. Using measurement-based care to enhance any treatment. Cognitive and Behavioral Practice. 2015;22(1):49–59. [PMC free article: PMC4910387] [PubMed: 27330267]

54.

Seitz J, Mee-LeeD. Feedback-informed treatment in an addiction treatment agency. In: Feedback-informed treatment in clinical practice: Reaching for excellence. Washington, DC: American Psychological Association; US; 2017:231–248.

55.

Shimokawa K, LambertMJ, SmartDW. Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology. 2010;78(3):298–311. [PubMed: 20515206]

56.

Slade K, LambertMJ, HarmonSC, SmartDW, BaileyR. Improving psychotherapy outcome: The use of immediate electronic feedback and revised clinical support tools. Clinical Psychology & Psychotherapy. 2008;15(5):287–303. [PubMed: 19115449]

57.

Stanley-Olson AR. Client feedback and group therapy outcomes for adults with cooccurring mental illness and substance abuse. Dissertation Abstracts International: Section B: The Sciences and Engineering. 2017;79(4-B(E)):No Pagination Specified.

58.

Trivedi MH, RushAJ, WisniewskiSR, et al. Evaluation of outcomes with citalopram for depression using measurement-based care in stard: Implications for clinical practice. American Journal of Psychiatry. 2006;163(1):28–40. [PubMed: 16390886]

59.

Waldrop J, McGuinnessTM. Measurement-based care in psychiatry. J Psychosoc Nurs Ment Health Serv. 2017;55(11):30–35. [PubMed: 28840934]

60.

Wampold BE. Routine outcome monitoring: Coming of age-with the usual developmental challenges. Psychotherapy. 2015;52(4):458–462. [PubMed: 26641376]

61.

Whipple JL, LambertMJ, VermeerschDA, SmartDW, NielsenSL, HawkinsEJ. Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. Journal of Counseling Psychology. 2003;50(1):59–68.

62.

Whittingham M, GrahamL. The impact of providing group performance feedback on a large mental health system. Psychotherapy. 2018;55(2):203–206. [PubMed: 29863402]

63.

Wise EA, StreinerD. Routine outcome monitoring and feedback in an intensive outpatient program. Practice Innovations. 2018;3(2):69–83.

Suggested citation:

Peterson K, Anderson J, Bourne, D. Evidence Brief: Suicide Prevention in Veterans. VA ESP Project #09-199; 2018.

This report is based on research conducted by the Evidence-based Synthesis Program (ESP) Coordinating Center located at the Portland VA Health Care System, Portland, OR, funded by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Quality Enhancement Research Initiative. The findings and conclusions in this document are those of the author(s) who are responsible for its contents; the findings and conclusions do not necessarily represent the views of the Department of Veterans Affairs or the United States government. Therefore, no statement in this article should be construed as an official position of the Department of Veterans Affairs. No investigators have any affiliations or financial involvement (eg, employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties) that conflict with material presented in the report.