We performed a comprehensive review of 760 medical test accuracy meta-analyses published over the last 25 years. This work provides a “snapshot” of the available literature and an overview of longitudinal trends in methods and reporting, with the aim of identifying where future reviews could be improved. Meta-analyses of test accuracy are increasingly being pursued: in recent years approximately 100 such reports have been published annually. Overall, the available literature appears to have several limitations (Box 1): most reviews do not appraise important quality items, statistical analyses use methods that may be suboptimal for test accuracy and direct comparisons of index tests are scarce. Our findings regarding the limitations of existing systematic reviews of test accuracy generally agree with previously published, smaller-scale surveys of reviews of test accuracy. We have summarized some of these previous empirical investigations in Table 9. Generally, previous assessments of systematic reviews of diagnostic tests have assessed much smaller numbers of studies or have been limited to a single clinical topic (e.g., oncology³¹). Furthermore, with the exception of a report focusing on the statistical methods used for meta-analysis,¹¹ no previous overview has included an adequate number of studies spread over several years that would allow the exploration of trends over time.

Box 1

Common limitations of existing systematic reviews of test accuracy. Comparative effectiveness of medical tests Most systematic reviews were focused on a single index test.

Table 9

Summary of selected previously published overviews of systematic review of test accuracy.

We also found that many aspects of the methods and reporting of systematic reviews of medical test accuracy have improved over time. Searching of multiple electronic databases without language restrictions has become more common; quality items such as verification bias, spectrum bias and blinding have been increasingly been considered in quality appraisal; and advanced statistical methods that simultaneously model sensitivity and specificity are beginning to be adopted. Empirical studies comparing the reporting of methodological quality items have documented an increase in the clarity of reporting of quality items in systematic reviews of therapeutic interventions³⁶ after the International Committee of Medical Journal Editors endorsed the Quality of Reporting of Meta-analyses (QUOROM) checklist, compared to before. Similar data exist on the impact of the Consolidated Standards of Reporting Trials (CONSORT) statement^37,38 for reporting of randomized trials.^39,40 We observed that the QUADAS tool^19,21 and quality items developed based on the STARD reporting checklist^22,23 were used more often by recent systematic reviews; we hypothesize that their use may have had a similar influence on the reporting of meta-analyses of test accuracy.

A recent focused empirical assessment¹¹ of meta-analyses of medical tests concluded that the increased use of bivariate random effects statistical models for sensitivity and specificity coincided with the development of easy-to-use routines for performing such analyses (e.g., the metandi and midas commands in Stata or scripts for SAS programming).^41,42 We observed the same pattern. Further, we observed that, at the same time, the use of simpler, but less appropriate methods such as the fixed effects SROC model of Moses and Littenberg^13,27 (which accounts for only part of the uncertainty in the bivariate probability model), has decreased. Guidance within the AHRQ Evidence-based Practice Center Program^c and the Cochrane Collaboration^d supports the use of hierarchical modeling methods in meta-analyses of test accuracy. Although such models are more theoretically motivated compared to separate univariate analyses of sensitivity and specificity for the binary classification case, their judicious application requires an understanding of the underlying model assumptions.⁴³

We found substantial differences in methods and reporting of test accuracy studies across different types of medical tests. These differences may reflect either heterogeneous diffusion of methodological advances between research groups focusing on specific test types, or the reviewers' assessment that specific methodological approaches are not applicable to specific test types. In contrast, we found few differences in comparisons across different medical fields. Differences in some reporting or methodological characteristics of reviews correlated with the impact factor of journals where they were published and the number of citations they accrued over time. Interpretation of these differences is challenging, given the large number of comparisons performed and the possibility that journal editorial policies and journal readability could confound many of the observed associations.

Our work has several limitations that need to be considered when interpreting our results. First, we relied on searches using methodological filters for identifying reports of meta-analyses of medical test accuracy studies and we only considered English language publications.⁴⁴ Also, we relied exclusively on MEDLINE searches, supplemented by screening of the reference lists of eligible studies and those of relevant review articles, to identify eligible reviews. More comprehensive searches would have required the examination of a much larger number of abstracts with little expected incremental yield. Second, we focused on meta-analyses using aggregate published data and excluded individual patient data analyses. Although the latter represent a minority of all published meta-analyses they provide additional flexibility in exploring between-study heterogeneity due to patient level factors.⁴⁵ Third, we did not perform double extraction for all eligible studies. However, we implemented several procedures for standardizing the definition of the extracted variables during data extraction and performed extensive quality control of the final dataset. Further, a substantial proportion of eligible articles were extracted in duplicate.

This comprehensive overview of meta-analyses of test accuracy highlights the current status and the temporal evolution of a complex research field. Available meta-analyses of medical tests have several limitations in regards to methodological approaches and reporting characteristics; however, over time reviews have increasingly performed more comprehensive assessments of study quality and have used more appropriate statistical methods addressing the particular challenges relevant to reviews of test accuracy. Based on our review of the literature, and observations from this current empirical assessment, we identify some cross-cutting methodological issues relevant to meta-analytic practice in Box 2. Areas for potential future methodological research include the assessment of publication and reporting bias in reviews of test accuracy, the collection of empirical evidence on how study-level characteristics can influence the results of systematic reviews, quantitative methods for the comparative assessment of multiple alternative index tests, and the evaluation of modeling approaches for contextualizing the findings of reviews of test accuracy.

Box 2

Cross-cutting methodological issues relevant to meta-analytic practice. Comparative effectiveness reviews of medical tests Often many alternative tests are applicable to a given testing scenario; systematic reviews may want to consider all relevant index (more...)

The large and rapidly expanding number of available meta-analyses identified by this overview reflects the growing interest in “evidence-based diagnosis”.^46,47 Increasing use of quality checklists is expected to facilitate further improvements in the quality assessment of primary studies included in meta-analyses of medical tests. Similarly, increasing diffusion of methodological advances, availability of software to perform advanced statistical analyses and clear guidelines for the conduct and reporting of meta-analyses of test accuracy will hopefully lead to further improvements in the practice of systematic reviews of medical tests.