Table 5Strengths and Limitations of Network Meta-analysis using ISPOR checklist5

StrengthsLimitations
Fan 20188
Introduction
  • The rationale and objective of the review was clearly stated.
Methods
  • A comprehensive and systematic search of the literature was conducted in accordance with the PRISMA statement and PRISMA extension statement for NMAs.
  • The search strategy and the study selection criteria were clearly stated.
  • Data extraction and study quality assessment was done independently by two reviewers; inconsistencies were resolved by a third reviewer. The Cochrane collaboration risk of bias tool and the Newcastle-Ottawa scale were used to appraise the methodological quality of RCTs and non-RCTs, respectively. The quality of the included studies varied, with low, medium, and high risk of bias found depending on the trial.
  • The outcome measures were selected appropriately and clearly described.
  • The choice of fixed or random effect was justified and based on I2 value of <50% as a cutpoint for statistical heterogeneity.
  • Both direct pairwise and indirect comparisons were performed to assess the consistency assumption, as well as the results from consistency and inconsistency models of NMA.
  • Assessment of publication bias was done visually using Funnel plots and quantitatively using the Begg’s test and Egger’s test. Other than the pooled PFS and discontinuation rate, publication bias was unlikely for other outcomes.
  • The methods for indirect comparison (Bayesian network meta-analyses) and ranking treatment probabilities for each line of therapy were appropriate.
  • The methods for pooling Kaplan-Meier curves and other binary outcome measures were appropriate.
Results
  • Identification and selection of full-text studies for the NMA were well reported, as well as presented in a PRISMA flowchart. Additionally, a network diagram was provided.
  • Although not in great detail, a table with study and patient characteristics was provided; and summary effect estimates from each included trial was available.
  • Convergence of all models using the MCMC method was shown with a PSRF value of 1.00.
Discussion
  • A description of the main findings was presented that highlighted the potential limitations of the results as well as possible explanations for discrepancies across studies.
  • The authors did not provide a discussion on the generalizability of findings; however, given the included studies combined a high number of patients, all available ALK-inhibitors, and trials conducted across the world in various settings, the generalizability of the results is not likely to be a concern.
Methods
  • The models were conducted without covariate adjustment for patient or study characteristics, and hence control of potential bias could not be assessed.
  • There was no information on prior distributions for model parameters, and whether priors were informative or non-informative.
  • No sensitivity analyses were performed to assess the effect of different covariate distributions or model assumptions.
Discussion:
  • Between-study heterogeneity was high for a number of pairwise comparisons, due primarily to differences in doses of treatment, baseline parameters of enrolled patients, and follow-up duration.
  • Overall, the original trials used to pool results for direct and indirect comparisons included a mix of ALK inhibitor-naive and pretreated patients, therefore the combined results should be interpreted with caution. However, this was not a limitation of the NMA per se, rather limited the usability of the results given the research questions in this Rapid Response report.

ISPOR = International Society for Pharmacoeconomics and Outcomes Research; MCMC = Markov chain Monte Carlo; NMA = network meta-analyses; PRISMA = preferred reporting items for systematic reviews and meta-analysis; PSRF = potential scale reduction factor; RCT = randomized controlled trial

From: Anaplastic Lymphoma Kinase Inhibitors for Genetically Rearranged Non-Small Cell Lung Cancer: A Review of the Clinical Effectiveness

Cover of Anaplastic Lymphoma Kinase Inhibitors for Genetically Rearranged Non-Small Cell Lung Cancer: A Review of the Clinical Effectiveness
Anaplastic Lymphoma Kinase Inhibitors for Genetically Rearranged Non-Small Cell Lung Cancer: A Review of the Clinical Effectiveness [Internet].
Islam S, Argáez C.
Copyright © 2018 Canadian Agency for Drugs and Technologies in Health.

The copyright and other intellectual property rights in this document are owned by CADTH and its licensors. These rights are protected by the Canadian Copyright Act and other national and international laws and agreements. Users are permitted to make copies of this document for non-commercial purposes only, provided it is not modified when reproduced and appropriate credit is given to CADTH and its licensors.

Except where otherwise noted, this work is distributed under the terms of a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International licence (CC BY-NC-ND), a copy of which is available at http://creativecommons.org/licenses/by-nc-nd/4.0/

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.