Executive Summary

Scope and Key Questions

MTM is a complex intervention that could have different components depending on the goals and scope of the MTM program. This review seeks to catalog outpatient-based MTM intervention components, assess the overall effectiveness of outpatient-based MTM in comparison with usual care, examine the factors under which outpatient-based MTM is effective and optimally delivered, assess what types of patients are likely to benefit from outpatient-based MTM services, and clarify what types of patients may be at risk of harms from such programs. This review does not address (1) MTM services provided within inpatient settings or shortly after hospital discharge, (2) disease management services provided by pharmacists, or (3) interventions designed as a single episode of contact. The rationale for limiting the scope to exclude some types of MTM interventions is to ensure that included studies are reasonably comparable with respect to intended purpose and design of the MTM intervention.

The Key Questions (KQs) addressed in this review are—

KQ 1: What are the components and implementation features of MTM interventions?

KQ 2: In adults with one or more chronic diseases who are taking prescription medications, is MTM effective in improving the following:

1. Intermediate outcomes, including biometric and laboratory measures, drug therapy problems identified, drug therapy problems resolved, medication adherence, goals of therapy met, and patient engagement in medication management?
2. Patient-centered outcomes, such as disease-specific morbidity, disease-specific or all-cause mortality, adverse drug events, health-related quality of life, activities of daily living, patient satisfaction with health care, work or school absenteeism, and patient and caregiver participation in medical care and decisionmaking?
3. Resource utilization, such as prescription drug costs, other health care costs, and health care utilization?

KQ 3: Does the effectiveness of MTM differ by MTM components and implementation features?

KQ 4; Does the effectiveness of MTM differ by patient characteristics, including but not limited to patient demographics and numbers and types of conditions and medications?

KQ 5: Are there harms of MTM, and if so, what are they?

Analytic Framework

The KQs are placed in relation to one another and the populations, interventions, comparators, outcomes, timing, and setting (PICOTS) in the analytic framework (Figure A). Specific details regarding patient population, intervention components, and outcomes are provided in the next section.

Populations, Interventions, Comparators, Outcomes, Timing, and Setting

Table A lays out the PICOTS for this review. For this review, we took a broad perspective on the population and interventions evaluated; we did not require CMS Part D MTM eligibility criteria. Specifically, we did not require multiple chronic conditions or a minimum number or level of expenditures on prescription drugs. We included randomized and controlled clinical trials, systematic reviews, and prospective and retrospective cohort studies. We included observational studies because we anticipated, from our topic refinement work, that a review limited to trials alone would fail to yield evidence on our wide range of prespecified benefits and harms for MTM interventions as a whole and for studies evaluating the modifying effects of specific intervention and patient characteristics on outcomes of MTM interventions.

Topic Refinement and Review Protocol

The topic of this report and preliminary KQs arose through a nomination from the Pharmacy Quality Alliance. Key Informants representing several clinical and scientific disciplines provided input on the initial KQs; we revised them as needed. An initial draft of the revised KQs was posted for public comment from March 6, 2013, through April 2, 2013, on the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program Web site. We received comments from 23 professional organizations and individuals and further revised KQs as appropriate.

Literature Search and Identification Strategy

To identify articles relevant to each KQ, we began with a focused MEDLINE^® search for MTM interventions using a combination of medical subject headings and title and abstract keywords, and limiting the search to English-language and human-only studies (inception through January 9, 2014). We also searched the Cochrane Library (inception through January 10, 2014) and the International Pharmaceutical Abstracts database (inception through January 10, 2014) using analogous search terms. We selected these databases based on preliminary searches and consultation with content experts. We conducted quality checks to ensure that the searches identified known studies (i.e., studies identified during topic nomination and refinement). Based on these quality checks, we revised and ran additional searches (specifically, using keywords such as “drug therapy management,” “drug therapy problem,” and “medications management”) to avoid missing articles that might prove eligible for this systematic review.

In addition, we searched the gray literature for unpublished studies relevant to this review and included studies that met all the inclusion criteria and contained enough methodological information to assess risk of bias. Specifically, sources of gray literature included ClinicalTrials.gov, the World Health Organization's International Clinical Trials Registry Platform, Health Services Research Projects in Progress (HSRProj), the National Institutes of Health Research Portfolio Online Reporting Tools, the Database of Promoting Health Effectiveness Reviews, the New York Academy of Medicine Grey Literature Report, and CMS.gov. In addition, we reviewed the yield from AHRQ's request for Scientific Information Packets in the Federal Register, posted for 30 days from September 16, 2013 onward.

We reviewed our search strategy with an independent information specialist and the Technical Expert Panel, and supplemented it according to their recommendations. In addition, to avoid retrieval bias, we manually searched the reference lists of landmark studies and background articles on this topic to identify any relevant citations that our electronic searches might have missed.

Two trained members of the research team independently reviewed each of the titles and abstracts against the inclusion/exclusion criteria listed in Table A. We applied the same criteria to systematic reviews and primary studies. Each article that either or both reviewers chose to include based on the abstract review underwent full-text review. Two reviewers reviewed the full text for eligibility against our inclusion/exclusion criteria. During full-text review, if both reviewers agreed that a study did not meet the eligibility criteria (including designation of high risk of bias), we excluded the study. Reviewers resolved conflicts by discussion and consensus or by consulting a third member of the review team.

For studies that met our inclusion criteria, a trained reviewer abstracted information into structured evidence tables; a second senior member of the team reviewed all data abstractions for completeness and accuracy. Reviewers resolved conflicts by discussion and consensus or by consulting a third member of the review team.

Assessment of Risk of Bias of Individual Studies

To assess the risk of bias of individual studies, we used predefined criteria developed by AHRQ.¹⁸ For randomized controlled trials (RCTs), we relied on the risk-of-bias tool developed by the Cochrane Collaboration.¹⁹ We assessed the risk of bias of observational studies using an item bank developed by RTI International.²⁰

In general terms, results of a study with low risk of bias are considered valid. Studies marked low risk of bias did not have any major flaws in design or execution. A study with medium risk of bias is susceptible to some bias but probably not sufficient to invalidate its results. A study with high risk of bias has significant methodological flaws (e.g., stemming from serious errors in design or analysis) that may invalidate its results. Primary concerns for our review included selection bias, confounding, performance bias, detection bias, and attrition bias. Very high attrition rates, particularly when coupled with a failure to control for confounding or to conduct intention-to-treat analyses, resulted in a rating of high risk of bias for trials and prospective cohort studies. Likewise, we rated studies with an inherently high risk of confounding in design (e.g., observational studies comparing refusers vs. acceptors of MTM interventions) as high risk of bias if they failed to address confounding through design (e.g., matching) or analysis (e.g., regression). Specifically, we evaluated trials on the adequacy of randomization, allocation concealment, similarity of groups at baseline, masking, attrition, whether intention-to-treat analysis was used, method of handling dropouts and missing data, validity and reliability of outcome measures, and treatment fidelity. For observational studies, we did not assess adequacy of randomization or allocation concealment but assessed for confounding. We also evaluated trials for confounding due to randomization failure through biased selection or attrition. In other words, we evaluated trials with potential randomization failure for the same risks of bias as observational studies.

We excluded studies that we deemed at high risk of bias from our main data synthesis and main analyses. We included them for sensitivity analyses; in cases when we had no other available or credible evidence, we included in the report a brief synopsis of studies assessed as high risk of bias.

Data Synthesis

When we found three or more similar studies for a comparison of interest, we conducted meta-analysis of the data from those studies using Comprehensive Meta-Analysis software (Biostat, Inc, Englewood, NJ). For all analyses, we used random-effects models to estimate pooled or comparative effects. To determine whether quantitative analyses were appropriate, we assessed the clinical and methodological heterogeneity of the studies under consideration following established guidance;²¹ that is, we qualitatively assessed the PICOTS of the included studies, looking for similarities and differences. When we conducted quantitative syntheses (i.e., meta-analysis), we assessed statistical heterogeneity in effects between studies by calculating the chi-squared statistic and the I² statistic (the proportion of variation in study estimates attributable to heterogeneity). The importance of the observed value of I² depends on the magnitude and direction of effects and on the strength of evidence for heterogeneity (e.g., the p-value from the chi-squared test or a confidence interval for I²). Where relevant, we examined potential sources of heterogeneity using sensitivity analysis.

When quantitative analyses were not appropriate (e.g., because of heterogeneity, insufficient numbers of similar studies, or insufficiency or variation in outcome reporting), we synthesized the data qualitatively. Whenever possible, we computed confidence intervals for individual outcomes.

Numerous articles did not provide complete information about findings (e.g., 95% confidence intervals, statistical significance values, or between-group data). In many cases, therefore, we had to calculate odds ratios, mean differences or standardized mean differences, the relevant 95-percent confidence intervals, and p-values.

Grading Strength of Evidence for Individual Comparisons and Outcomes

We graded the strength of evidence based on the guidance established for the AHRQ Evidence-based Practice Center program.²² Developed to grade the overall strength of a body of evidence, this approach incorporates four key domains: study limitations (includes study design and aggregate quality), consistency (similar magnitude and direction of effect), directness (evidence links interventions directly to outcome of interest for the review), and precision of the evidence (degree of certainty surrounding an effect estimate based on sample size and number of events). In addition, the evidence may be rated as lower strength for bodies of evidence with suspected reporting bias from publication, selective outcome reporting, or selective analysis reporting. Regardless of the specific risk of bias of observational studies, this approach to grading the evidence assigns observational studies a grade of high for study limitations, which then leads to low strength of evidence. The strength of evidence from observational studies can be rated as higher for scenarios such as a strong dose-response association, plausible confounding that would decrease the observed effect, and a high strength of association (magnitude of effect). We evaluated optimal information size criteria to make judgments about precision based on guidance from Guyatt and colleagues²³ and based our grades on RCTs with low or medium risk of bias or on observational studies unless none were available, based on guidance from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group²⁴ and the AHRQ Evidence-based Practice Center program.²²

Table B describes the grades of evidence that can be assigned.²⁵ Grades reflect the strength of the body of evidence to answer the KQs on the overall effectiveness, comparative effectiveness, and harms of the interventions examined in this review. Two reviewers assessed each domain for each major outcome, and resolved any differences by consensus discussion or referral to a third, senior member of the team. We graded the strength of evidence for the outcomes deemed to be of greatest importance to decisionmakers and those commonly reported in the literature; we did not grade the strength of evidence for KQ 1 (on components and features of MTM services). The grades shown in Table B describe the state of evidence (which may demonstrate benefit, harm, or no effect) and the confidence in the stability of that state. An insufficient grade is not a statement about lack of efficacy or effectiveness; rather it is a statement about the lack of convincing evidence on benefit, harm, or lack of effect.

Assessing Applicability

We assessed applicability of the evidence following guidance from the “Methods Guide for Effectiveness and Comparative Effectiveness Reviews.”²⁶ We used the PICOTS framework to explore factors that affect applicability. Some factors identified a priori that may limit the applicability of evidence include the following: age and health status of enrolled populations, health insurance coverage and access to health care, and complexity and intensity of the MTM intervention.

Results of Literature Searches

Figure B presents our literature search results through January 9, 2014. We identified 2,516 unduplicated citations. In addition, we identified 233 publications through gray literature searches, suggestions from Technical Experts or public comments received during topic refinement, hand searches of included studies, and Scientific Information Packets. After applying our eligibility and exclusion criteria to titles and abstracts of all 2,749 identified citations, we obtained full-text copies of 419 published articles. We reapplied our inclusion criteria and excluded 358 of these articles from further review before doing the risk-of-bias assessment. The 61 articles included after full-text review represent 44 studies.

This evidence base consisted of 44 studies (21 RCTs, 4 controlled clinical trials, and 19 cohort studies) reported in 61 articles. Most RCTs compared an MTM intervention with usual care rather than with a different active intervention; all observational studies were cohort studies. Numerous studies had methods problems that led us to rate them as having a medium or high risk of bias; only a few studies were of low risk of bias. When possible (enough studies similar in intervention, populations, and outcomes measured), we conducted meta-analyses of data from RCTs or cohort studies separately; when relevant, we did two sets of meta-analyses, one with and one without the trials that had high risk of bias.

Because of the wide variation in types of interventions classified as MTM, we first cataloged intervention components and implementation features of MTM interventions (KQ 1). We then evaluated the effect of MTM on intermediate, patient-centered, and resource utilization outcomes (KQ 2). We also reviewed the evidence to identify how these effects might vary by specific intervention components and features (KQ 3) and patient characteristics (KQ 4). Finally, we reviewed the evidence on harms associated with MTM (KQ 5).

Below, we summarize the main findings and strength of evidence, where applicable. We then discuss the findings in relationship to what is already known, applicability of the findings, implications for decisionmaking, limitations, research gaps, and conclusions.

Key Findings and Strength of Evidence

Key Findings

We included 44 eligible studies (21 randomized controlled trials, 4 controlled clinical trials, and 19 cohort studies) reported in 61 articles, described in detail in the full report (KQ 1). Evidence was insufficient on the effect of MTM on most outcomes (KQ 2). In a few instances, described below, the evidence led us to conclude with a low strength of evidence either a benefit or lack of benefit. Specifically, we found evidence that MTM results in improvement when compared with usual care for some measures of medication adherence and appropriateness; medication dosing; health plan expenditures on medication costs; and, for patients with diabetes, the proportion and costs of hospitalization. Similarly, we conclude based on a low strength of evidence that MTM conferred no benefit for patient satisfaction and most measures of health-related quality of life.

We found evidence on five intervention components and intervention features (KQ 3): one study provided information on each feature and yielded insufficient evidence for most outcomes, with the two following exceptions. MTM programs with pharmacist access to brief clinical summaries from the medical record reduced the mean number of adverse drug events when compared with basic MTM programs without such access (low strength of evidence). Community pharmacists increased the generic dispensing ratio more than pharmacists based in call centers (low strength of evidence). We found no relevant studies on patient characteristics moderating the effect of MTM interventions (KQ 4). Similarly, the evidence on harms associated with MTM was limited to one study on inconvenience and was rated as insufficient (KQ 5).

Findings in Relation to What Is Already Known

Our findings contrast with conclusions that Chisholm-Burns and colleagues reached in a recent systematic review.³⁴ In that review, the authors concluded on page 923: “Pharmacist-provided direct patient care has favorable effects across various patient outcomes, health care settings, and disease states.”³⁴ Several differences between the Chisholm-Burns review and the current review may account for the discrepant conclusions. First, the Chisholm-Burns review included all studies that cited evidence of pharmacist involvement in direct patient care. The interventions examined included chronic disease management and prospective and retrospective drug utilization review; we excluded these types of efforts because of the clinical heterogeneity those interventions would have introduced into the review. Notably, the Chisholm-Burns review did not use the term “medication therapy management” to categorize the interventions in the articles they reviewed. Second, approximately 30 percent of the studies in the Chisholm-Burns review were conducted entirely in institutional settings. In contrast, we did not identify any studies within institutional settings that met our MTM intervention definition criteria. Third, the Chisholm-Burns review included a total of 298 articles and did not omit from the analyses studies with a high risk of bias; in contrast, we based our strength-of-evidence grades in this review on only those studies with no more than medium risk of bias. Thus, a direct comparison of findings between these two reviews would be ill advised.

The striking differences between the conclusions reached in these two reviews emphasize two important needs for efforts to systematically review MTM programs. The first is for researchers to specify the MTM intervention based on existing definitions, taxonomies, or service models. The second is to develop consensus guidelines for describing intervention features and fidelity of intervention delivery in publications reporting findings from evaluation studies. Progress on these two steps would enable systematic reviews to differentiate better between different types of services and avoid the problem of overgeneralizing review results.

Applicability of the Findings

This body of evidence has significant clinical and methodological heterogeneity, which limits the ability to make any universal statements about effectiveness. However, the range of study designs, which includes RCTs, nonrandomized trials, and cohort studies, enhances the applicability of findings for real-world settings.²⁶ Included studies ranged from relatively small interventions in single clinics provided by a single interventionist to evaluations of MTM services delivered on a large scale through integrated health systems or health plans as a Medicare Part D or other drug plan benefit. This diversity of studies enhanced the applicability of findings to a wide variety of settings, including outpatient clinics, community pharmacies, and centralized pharmacy call centers. A few studies conducted outside the United States included MTM as part of a home visits program; findings from this model may not be directly applicable within the United States.

The studies in this review are broadly applicable to a range of chronically ill adult patient populations. A majority of interventions were directed at populations with multiple and common chronic conditions, such as diabetes, chronic heart failure, and hypertension. Several specifically targeted adults age 65 years or older. Few studies reported sociodemographic characteristics beyond age and sex; thus, the applicability of findings to specific populations (e.g., rural, low socioeconomic status, cognitively impaired, uninsured) is unknown. The nature of the MTM intervention, which includes involving patients as active participants in the process, limits the extent to which findings can be generalized beyond patients who agree to participate in such interventions. Patients who agree to participate may be systematically different from those who decline to be in such a program. For that reason, the impact of such interventions at a population or health-plan level may be limited by the degree of uptake among interested patients.

The intervention used across most studies can be characterized as complex and moderately resource intensive. Components involve identifying applicable patients; initially assessing patients; providing counseling, education, and care coordination; and following patients over time. These services were provided per protocol in some studies and as needed or ad hoc in others. Most studies described intervention components in terms of “pharmaceutical care model” components or Medicare Part D MTM program criteria, but few provided detailed descriptions or measurement of implementation fidelity.

The comparator arm in all studies was usual medical or pharmacy care. This does not typically include distinct MTM services by health care providers other than prescribing providers (not common for the time period covered by most of the studies). Models of collaborative health care delivery are evolving, and the changing roles and training of pharmacists increase the potential applicability of MTM interventions in future models of health care.

The broad sets of outcomes evaluated across this body of evidence spanned a substantial range of both intermediate and health outcomes as well as outcomes related to resource use. Proximal and intermediate outcomes included number of drugs, identification of drug therapy problems, appropriateness of medication prescribing, and laboratory or biometric markers of disease control (e.g., hypertension, hemoglobin A1c, low-density lipoprotein cholesterol). Patient-centered outcomes focused on numerous measures of quality of life as well as adverse drug events. Many studies also reported outcomes involving health care resource use and expenditures (e.g., number and costs of hospitalizations, emergency department visits, outpatient visits).

Most studies did not, however, clearly indicate the expected, desired, or intended direction of effect on most resource use outcomes, making the applicability of using these interventions to reduce drug-related health care costs or expenditures difficult to assess. For example, it is not clear whether one should expect the number of medications prescribed for heart failure to increase or decrease under the careful scrutiny of an MTM intervention because the desired impact will be based on the goal of therapy for each individual.

The focus of outcome measurement in many studies was the short-term identification and characterization of drug therapy problems and their resolution; these endpoints are thought to be the outcomes most sensitive to change as a result of receiving MTM services. However, because identification of drug therapy problems is, by design, a part of the MTM intervention itself, differences between the nature of the intervention and that of the control programs mean that measuring these outcomes cannot be as rigorous in a usual-care comparison group as it is in the intervention group. In fact, many studies were able to measure changes in this outcome only in the intervention group. Hence, many studies failed to demonstrate a direct analytic link between the resolution of drug therapy problems as a result of MTM and impact on intermediate outcomes, patient-centered outcomes, and resource utilization. Thus, the applicability of studies that demonstrate an impact on the resolution of drug therapy problems is limited.

Implications for Clinical Practice and Policymakers

Although we found the evidence insufficient in general to draw definitive conclusions about the comparative effectiveness of MTM for most outcomes that we evaluated, our findings suggest some implications for practice and policy. MTM is already in widespread practice and is now shaped in the United States largely by Medicare Part D policy; this presents both challenges and opportunities. MTM programs sponsored and administered by Part D drug benefit plans are often centrally administered and delivered primarily by phone, and may be less integrated into routine health care than some of the interventions included in our review. MTM programs of the future have the potential to be more integrated into routine health care through participation in accountable care organizations or patient-centered medical home models. We were unable to answer definitively whether level of integration matters for effectiveness, but policymakers may need to consider expectations about the impact that MTM might have on patient-centered outcomes and resource use in the context of other health care delivery transformation activities or quality improvement initiatives that are also occurring. More integration of MTM services with other activities may be effective; however, the more integrated MTM becomes within routine medical care, the more difficult it becomes to isolate it as a discrete intervention for evaluation.

Policymakers could thus consider whether MTM services should be positioned as a contributor to overall improvement in processes of care, health status, and costs or positioned as an intervention to which effects can be discretely attributed. As noted earlier, improvements in medication appropriateness or drug therapy regimens may not always translate into improvements in health or costs, and even if they do, secular trends in related quality improvement (e.g., medication adherence interventions, regulatory requirements for medication reconciliation, meaningful use incentives for electronic health records) may make measuring outcomes attributable to MTM very challenging.

Future training of MTM providers would benefit from a better understanding of which MTM components really matter. At the moment, such information is lacking. Policymakers and funders who wish to understand the comparative effectiveness of different MTM components could encourage rigorous program evaluation designs that fit within the context of the real-world implementation of these programs. For example, positive deviance analyses³⁵ with rigorous measurement of implementation features or stepped wedge trial designs³⁶ may be useful approaches.

A typical approach for evaluating complex interventions is to identify the “core” components for standardization, while allowing for flexibility for peripheral components or variations in implementation. In complex practice-based innovations, such flexibility may reflect desirable (or unavoidable) adaptations to local circumstances. Policy governing MTM programs may warrant modifications to permit investigators to conduct rigorous and innovative evaluative designs to identify core components or effectiveness-enhancing modifications. As future research and evaluation elucidate these components or enhancements, policy will need to evolve to keep pace with best practices.

Finally, consideration of both patients' and prescribers' perspectives in future design and delivery of MTM services may be needed. In our current analytic framework, MTM interventions require a significant element of engagement by both patients and prescribers if the interventions are to have a reasonable likelihood of improving outcomes. Although “opt in” strategies may increase the reach of such interventions, keeping patients (and their prescribing providers) engaged in the intervention over a reasonable amount of time may be the key to translating the potential of MTM interventions into actual improvements. Further refinement of eligibility criteria based on evidence to provide interventions to those most at risk from drug-related problems, and therefore most likely to benefit, may also be warranted.

Limitations of the Comparative Effectiveness Review Process

The constraints for populations, interventions, and settings that we imposed on this systematic review may limit its applicability, as discussed above. During topic refinement and based on Technical Expert Panel inputs and public comment, we expanded the scope by removing an exclusion criterion that would have required MTM interventions to have been directed at a patient population with two or more chronic conditions. As a result, we included studies that focused on one chronic condition. Because of the prevalence of certain chronic conditions in the adult population, and particularly among Medicare beneficiaries, we think this decision was sensible and permitted us to examine a broader evidence base than would otherwise have been the case.

Although we tried to distinguish MTM from disease or case-management interventions, making this distinction was challenging. We created a threshold for the intervention components that were required to be present for this distinction. Specifically, we elected to emphasize whether the intervention entailed a comprehensive review of all medications; for that reason, we did not constrain studies of interest to those that targeted a single medication or drug regimen or that focused on a single condition such as diabetes or hypertension.

When we were unable to determine which medications the interventionist had reviewed, we wrote to the authors for additional information. We chose to pursue authors in an effort to permit us to use studies that had been designed as MTM but did not describe the comprehensive medication review component in detail.

Our approach may have been overly inclusive because it led us to include studies that addressed a single disease, as long as the pharmacist reviewed all medications. For example, 10 of the 44 studies were relatively narrowly focused; 2 of these addressed patients with chronic heart failure and 2 addressed patients with either hypertension or hypertension and diabetes. The remaining six studies focused on patients with diabetes, HIV, glucocorticoid-induced osteoporosis, or hemodialysis. The fact that we did not require patients to have more than one clinical condition resulted in an approach that was inclusive of these more narrowly focused (albeit often termed “MTM”) studies and may render our results less applicable to MTM interventions targeted to patients with a wide range of chronic conditions.

Also based on feedback during the process of setting out the scope of this review, we chose to include interventions that were broader than the Medicare Part D MTM-defined interventions. Put another way, we broadened our view of patient populations and intervention criteria, and we allowed studies not conducted in the United States into the evidence base. This decision led us to include interventions described as “pharmaceutical care,” which were generally based on the pharmaceutical care model principles;⁹ it also permitted us to examine investigations with elements of pharmaceutical care or MTM that did not specifically label the intervention as either MTM or pharmaceutical care. These studies were often described as “clinical pharmacist interventions.”

Furthermore, all the non-U.S. studies involved interventions within single-payer health systems. Hence, the interventions in this review constitute a more heterogeneous group than if we had allowed only those labeled as Medicare Part D MTM programs. This is both a limitation and a strength. Although our approach makes results more challenging to interpret, it enhances our ability not to miss interventions that include MTM components but lack the descriptor term “MTM.”

Studies did not often explicitly describe certain MTM components. In cases when we could not determine whether investigators had provided certain MTM components (such as patient education and counseling or coordination with other health care providers), we contacted the authors to gain additional information that would allow us to make an informed decision. We were fairly permissive in interpreting the presence of the MTM intervention components other than comprehensive medication review. The main reason is that we recognized that terms describing some components have evolved over time and may have been absent from the lexicon in earlier years or implicitly conveyed by authors by simply using the terms “MTM” or “pharmaceutical care” to describe their intervention.

Our approach to categorizing interventions for KQ 1 relied primarily on the short descriptions in published manuscripts and those we were able to obtain via email inquiries. Their similarities or differences substituted for any overarching taxonomy, because none that we considered seemed to fit our purpose. Thus, we have introduced intervention labels that, admittedly, do not fully describe or account for clinical heterogeneity among interventions. This approach limits our ability to make definitive statements about the effectiveness of various intervention components. We believe that the clusters and categorizations we used are useful heuristics, but some may regard them more as hypothesis generating than as reflecting settled principles of classification.

Finally, our search process was complicated by having to ensure coverage of all terms that could be used to describe MTM interventions over time. Adding to this challenge was our effort to examine the gray literature, where we thought we might find studies tilted toward effectiveness and real-world program evaluation. As it turned out, studies of these types of interventions were not indexed similarly; for that reason, we needed to rely heavily on hand searches of citation lists from key background articles to identify possibly relevant studies for inclusion. Thus, we may have missed some studies that might have qualified for inclusion. Given the considerable diversity in the evidence base we did have, however, we do not think that any potentially missed studies would have changed our conclusions in any material way. No meta-analyses included more than five studies; as a result, we did not examine included studies for publication bias quantitatively.

Limitations of the Evidence

As a body of evidence, the MTM literature evaluated in this review has measured numerous outcomes. As indicated in previous sections, very few outcomes, with the exception of harms, remain completely unexamined. Of the 44 studies in this review, we rated 28 as having medium, low, or mixed risk of bias. The 44 studies included 21 trials and 4 nonrandomized controlled studies. In other words, the literature on this topic is not marked by failure to consider important outcomes, universally high risk of bias, or pervasively weak designs.

Despite these advantages, we were unable to identify sufficient evidence on the majority of hypothesized outcomes of MTM. In several instances, our inability to rate evidence as higher than insufficient came from inconsistent and imprecise evidence or from a body of evidence with high study limitations. The choice of outcome measures in this body of evidence limited our ability to come to conclusions in some instances. For example, some studies did not focus on changes that proponents might expect MTM services to produce. Because effective MTM can either increase or decrease expenditures or use of services based on the needs of the patient, studies that did not prespecify the expected direction of change had no way to interpret their results as an appropriate change. Studies that demonstrated inconsistent results in direction of change (i.e., some showing an increase in resource use and others showing a decrease) may well have been consistent in terms of appropriate change, but because they generally failed to establish a priori the direction in which they expected to find an effect, we rated such evidence as indirect and inconsistent.

Similarly, studies often used nonstandardized or idiosyncratic measures for outcomes such as adverse events, adherence, and expenditures or costs; this tendency limited our ability to meta-analyze results. When studies focused on specific outcomes, they were often significantly underpowered to detect differences between groups (i.e., they did not meet optimal information size criteria). As a result, we rated several studies as imprecise.

MTM intervention studies are largely practice based and incorporate substantial heterogeneity in specific intervention elements and in patient populations targeted. Yet the evidence is sharply constrained in its ability to inform questions about the effectiveness of specific MTM components or intervention features (KQ 3 in our review) because study designs did not often capitalize on variants in MTM programs for a prospective evaluation of outcomes by those variants. Neither did they measure fidelity to intended MTM elements for post hoc evaluation. Similarly, the relatively untargeted nature of the MTM interventions meant that, in many studies, only small numbers of patients had any one specific condition, and most studies did not measure patient characteristics beyond age and sex, thus limiting our ability to address KQ 4 in our review. For this reason, the evidence we identified for this review was most relevant for KQ 2.

Research Gaps

In many bodies of research, questions regarding the comparative effectiveness of specific intervention components or implementation features are best answered after clear evidence of the overall effectiveness of the intervention relative to usual care has been established. Our review largely indicates insufficient evidence on the primary question of effectiveness relative to usual care. By definition, this limited what we could say about comparative effectiveness.

Nonetheless, the widespread implementation of MTM coexists with the urgent need for actionable information for policy, program policies, and training. This clinical and policy environment means that new research cannot afford to address causal claims relative to usual care first, followed by comparative effectiveness of the intervention elements in a relatively controlled environment, and finally, program evaluation of real-world implementation, all in sequential order.

In prioritizing among various research goals, therefore, funders may wish to consider the relative value of new evidence on overall effectiveness, effectiveness of implementation features, and program implementation and accountability. Trial research in narrow clinical settings can address questions of effectiveness but may lack applicability to real-world implementation. Likewise, evaluations of real-world programs with variable fidelity to interventions can answer questions about process and implementation, but they offer limited information on effectiveness. Research prioritization exercises will also need to account for already commissioned MTM intervention studies.

For new studies focusing on causal claims, a critical gap relates to the failure to specify the expected direction of effect. New research requires a strong theoretical foundation to help specify causal mechanisms and hypothesized effects. Without such an edifice, future research will continue to produce inconsistent and uninterpretable results.

Heightened attention to causal mechanisms will also help researchers convey their understanding of what outcomes these types of interventions are likely to influence. For instance, how should researchers wishing to establish direct causal links between MTM programs and outcomes evaluate distal outcomes such as patient-centered outcomes and resource utilization? This effort requires a better understanding of the relationship between proximal outcomes such as “drug therapy problems identified and resolved” and distal outcomes. For instance, MTM may reduce outpatient visits to address side effects. MTM may also result in the need for further testing and evaluation for some patients, which could, in turn, result in more rather than fewer outpatient visits. Unless the nature of change resulting from MTM is specified in relation to goals of drug therapy, studies cannot assert benefit or harm. Further, drug therapy problems are diverse and may not all have the same causal relationship to health, quality of life, patient satisfaction, or resource use outcomes. Furthermore, a causal model of these distal outcomes may need to take into account the competing or complementary contributions of MTM, new models of health care delivery (e.g., patient-centered medical homes), and other quality improvement interventions.

Investigators embarking on new studies focusing on causal links between MTM and outcomes may wish to consider the limitations of studies based on secondary data from existing MTM programs that use opt-in/opt-out patient enrollment mechanisms. Although these studies may provide invaluable information on process measures such as patient engagement, underlying issues of confounding severely limit the validity of causal claims from such studies.

Regardless of the goal of their future research, investigators should consider issues of sample size to ensure precision of their results. This issue is particularly relevant when evaluating outcomes likely to occur in smaller subgroups defined by patient risk, context, or highly adapted intervention features. Innovative designs (e.g., stepped wedge trials, statistical process control, time-series analysis, simulations, and factorial experiments) may permit both rigor and adequate sample size within the context of real-world implementation. With careful attention to fidelity, new studies may also inform questions about the effectiveness of intervention components and implementation features. Mixed-methods approaches may allow more information on variations in context and implementation. Such designs may also help inform our understanding of critical training elements for MTM service providers.

Regarding research gaps for specific outcomes such as patient satisfaction, measures specific to the types of services provided through MTM (e.g., patient education about medications) or to the proximal outcomes that MTM is intended to achieve (e.g., reduced medication side effects, improved disease control) may offer better insights into the effects of MTM. Similarly, a medication-related instrument may better measure patients' concerns that are directly related to medication use (e.g., experience of side effects, intrusiveness of the medication regimen) than generic tools do.

Conclusions

The evidence base offers low evidence of benefit for a limited number of intermediate and health utilization outcomes.We graded the evidence as insufficient for most outcomes because of inconsistency in direction, magnitude, and precision, rather than lack of evidence. Wide variations in populations and interventions, both within and across studies, likely explain these inconsistencies. Given the widespread implementation of MTM and urgent need for actionable information, optimal investments in new research require a process of research prioritization in which the value of information from each proposed study is carefully considered. Studies designed to identify causal relationships between MTM interventions and their outcomes require adequate controls for confounding but may offer limited information on which factors explain program success or failure. Studies designed to explore the reasons for program success or failure using qualitative or single-arm designs may offer hypothesis-generating rather than hypothesis-confirming insights on MTM effectiveness. New research, regardless of specific focus, will likely continue to find inconsistent results until underlying sources of heterogeneity are accounted for.