Background: Many questions in evidence-based medicine involve multiple outcomes. They can be approached with separate, independent meta-analyses, or they can be analyzed jointly, in a single model. We aimed to compare separate (univariate) with joint (multivariate) meta-analysis in real examples and in an illustrative simulation study.
Methods: We screened the whole Cochrane Library of Systematic Reviews (2012, first quarter) to identify sets of univariate meta-analyses of categorical outcomes that can also be analyzed jointly. Eligible were pairs or triplets of meta-analyses comparing the same interventions; having at least seven randomized controlled trials (RCTs) reporting all outcomes; and in which the numbers in the cross-classification of outcomes were exactly recoverable. Examples of outcomes with completely recoverable cross-classification include mutually exclusive outcomes, or sets of outcomes where the one is a subset of the other. We analyzed these data with univariate and multivariate meta-analysis. In an accompanying simulation study, we compared summary estimates and their standard errors with univariate and multivariate meta-analysis.
Results: We identified 45 pairs or triplets of binary meta-analyses corresponding to 1473 RCTs and 258,675 randomized patients. In 38 (of 45) topics the first outcome was a subset of the second outcome; in 5 topics pairs of outcomes were mutually exclusive, and in 2 topics triplets of outcomes had an is-subset-of relationship. The 45 topics pertained to various medical areas (e.g., cardiology, surgery, mental health). Overall, the summary effects for each outcome and the accompanying confidence/credible intervals were very similar with univariate and multivariate meta-analysis (both using the approximate and the discrete likelihood). However, univariate and multivariate approaches yield different confidence/credible intervals for the difference between the summary effects of distinct outcomes (e.g., the difference in the log odds ratio for the first outcome minus the log odds ratio for the second outcome). Depending on the estimated covariance between the compared effects, the multivariate methods can yield tighter or wider confidence intervals than univariate methods. Most likely, systematic review conclusions from the meta-analyses in the empirical sample would remain qualitatively the same with either method of analysis. The simulation analyses were congruent with the aforementioned observations from the empirical analyses.
Conclusions: In the empirical sample and the simulation study, the numerical difference in the summary effects and their confidence intervals between univariate and multivariate meta-analysis was almost always small. In practice, in many (if not most) cases, conclusions based on the main effects of each outcome are likely to remain similar with either method. In principle, multivariate meta-analysis utilizes more information through the correlations; therefore, when possible, it is commendable to use both univariate and multivariate approaches in a sensitivity analysis. Multivariate meta-analysis should be preferred over univariate meta-analysis for estimating differences between outcome-specific summary treatment effects.