U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

McDonagh MS, Selover D, Santa J, et al. Drug Class Review on Agents for Overactive Bladder: Final Report [Internet]. Portland (OR): Oregon Health & Science University; 2005 Dec.

Cover
  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Drug Class Review on Agents for Overactive Bladder

Drug Class Review on Agents for Overactive Bladder: Final Report [Internet].

Show details

Appendix B. Quality Assessment Methods for Drug Class Reviews for the Drug Effectiveness Review Project

The purpose of this document is to outline the methods used by the Oregon Evidence-based Practice Center (EPC), based at Oregon Health & Science University, and any subcontracting EPCs, in producing drug class reviews for the Drug Effectiveness Review Project.

The methods outlined in this document ensure that the products created in this process are methodologically sound, scientifically defensible, reproducible, and well-documented. This document has been adapted from the Procedure Manual developed by the Methods Work Group of the United States Preventive Services Task Force (version 1.9, September 2001), with additional material from the NHS Centre for Reviews and Dissemination (CRD) report on Undertaking Systematic Reviews of Research on Effectiveness: CRD's Guidance for Carrying Out or Commissioning Reviews (2nd edition, 2001) and "The Database of Abstracts of Reviews of Effects (DARE)" in Effectiveness Matters, vol. 6, issue 2, December 2002, published by the CRD. To ensure scientific rigor and relevance of the work, the Oregon EPC develops key questions and criteria for admissible evidence, and uses these to create a literature search strategy that best captures the appropriate evidence. To consider papers identified by the searches, the teams use the criteria for admissible evidence (explicit inclusion and exclusion criteria) to select papers that provide information to help answer the key questions. They abstract key data from these selected papers. The team's use established criteria to assess the internal validity of the evidence in each paper, as well as the total internal validity, external validity, and coherence of the evidence for each key question.

Key Questions and Inclusion/Exclusion Criteria

Key questions are essential in focusing the literature review on a manageable and clinically relevant topic. All key questions are reviewed and approved by the topic team in the process of assessing and refining the topic before the detailed literature review. The EPC teams work with the subcommittee members of the Oregon Health Resources Commission assigned to a particular drug class to finalize the key questions for that drug class.

We clearly document the criteria by which the team chooses to admit evidence on a given key question. Such criteria might include, for example, study design (e.g., randomized controlled trials, cohort studies), setting, sample size, population studied, language(s) of publication, and year(s) of publication.

No generic criteria for admissible evidence have been established. Rather, the criteria are determined on a topic-by-topic and key question-by-key question basis, depending on the questions involved and the amount and quality of evidence available. All inclusion/exclusion criteria are reviewed and approved by the entire topic team.

Databases to Be Searched and Documenting Search Terms

At a minimum, all topics include a review of the English-language literature in MEDLINE and EMBASE bibliographic databases and the Cochrane Controlled Trials Register. Other databases (e.g., nursing or psychology databases) are searched as deemed necessary by the topic team. Evidence reviews document the databases used.

Search terms used for each key question, along with the yield associated with each term, are documented in a table or set of tables; these appear in the final evidence review.

Database of Abstracts

The EPC, for each review, establishes a database of all abstracts (i.e., both those included and those eventually excluded from the final set of full-text articles reviewed). Information captured in the database includes the key question(s) associated with each included abstract and reason for exclusion if the abstract does not meet inclusion criteria.

Abstraction Forms

Although the EPC has no standard or generic abstraction form, the following broad categories are always abstracted from included articles: study design, study participant description, quality information, and outcomes. Each team uses these (and, if indicated, other) general categories to develop an abstraction form specific to the topic at hand.

Double Abstraction of Included Articles

The EPC teams abstract only those articles that, after review of the entire article, meet criteria for both quality and focus on the key question at hand. Key articles are always read and checked by more than one team member. All reviewers are trained in the topic, the analytic framework and key questions, and the use of the abstraction instrument. Initial reliability checks are done for quality control.

Quality Criteria

Assessment of Internal Validity

To assess the internal validity of individual studies, the EPC adopted criteria for assessing the internal validity of individual studies from the US Preventive Services Task Force and the NHS Centre for Reviews and Dissemination.

For Controlled Trials

Assessment of Internal Validity

  1. Was the assignment to the treatment groups really random?
    • Adequate approaches to sequence generation:
      • Computer-generated random numbers
      • Random numbers tables
    • Inferior approaches to sequence generation:
      • Use of alternation, case record numbers, birth dates or week days
    • Not reported
  2. Was the treatment allocation concealed?
    • Adequate approaches to concealment of randomization:
      • Centralized or pharmacy-controlled randomization
      • Serially-numbered identical containers
      • On-site computer based system with a randomization sequence that is not readable until allocation
      • Other approaches sequence to clinicians and patients
    • Inferior approaches to concealment of randomization:
      • Use of alternation, case record numbers, birth dates or week days
      • Open random numbers lists
      • Serially numbered envelopes (even sealed opaque envelopes can be subject to manipulation)
    • Not reported
  3. Were the groups similar at baseline in terms of prognostic factors?
  4. Were the eligibility criteria specified?
  5. Were outcome assessors blinded to the treatment allocation?
  6. Was the care provider blinded?
  7. Was the patient kept unaware of the treatment received?
  8. Did the article include an intention-to-treat analysis, or provide the data needed to calculate it (i.e., number assigned to each group, number of subjects who finished in each group, and their results)?
  9. Did the study maintain comparable groups?
  10. Did the article report attrition, crossovers, adherence, and contamination?
  11. Is there important differential loss to followup or overall high loss to followup? (give numbers in each group)

Assessment of External Validity (Generalizability)

  1. How similar is the population to the population to whom the intervention would be applied?
  2. How many patients were recruited?
  3. What were the exclusion criteria for recruitment? (Give numbers excluded at each step)
  4. What was the funding source and role of funder in the study?
  5. Did the control group receive the standard of care?
  6. What was the length of followup? (Give numbers at each stage of attrition.)

For Reports of Complications/Adverse Effects

Assessment of Internal Validity

  1. Was the selection of patients for inclusion non-biased (Was any group of patients systematically excluded)?
  2. Is there important differential loss to followup or overall high loss to followup? (Give numbers in each group.)
  3. Were the events investigated specified and defined?
  4. Was there a clear description of the techniques used to identify the events?
  5. Was there non-biased and accurate ascertainment of events (independent ascertainer; validation of ascertainment technique)?
  6. Were potential confounding variables and risk factors identified and examined using acceptable statistical techniques?
  7. Did the duration of followup correlate to reasonable timing for investigated events? (Does it meet the stated threshold?)

Assessment of External Validity

  1. Was the description of the population adequate?
  2. How similar is the population to the population to whom the intervention would be applied?
  3. How many patients were recruited?
  4. What were the exclusion criteria for recruitment? (Give numbers excluded at each step)
  5. What was the funding source and role of funder in the study?

Economic Studies

Assessment of Internal Validity

Framing
  1. Was a well-defined question posed in answerable form?
  2. Was a comprehensive description of the competing alternatives given?
  3. Are the interventions and populations compared appropriate?
  4. Is the study conducted from the societal perspective?
  5. Is the time horizon clinically appropriate and relevant to the study question?
Effects
  1. Are all important drivers of effectiveness included?
  2. Are key harms included?
  3. Is the best available evidence used to estimate effectiveness?
  4. Are long-term outcomes used?
  5. Do effect measures capture preferences or utilities?
Costs
  1. Are costs and outcomes measured accurately?
  2. Are costs and outcomes valued credibly?
  3. Are costs and outcomes adjusted for differential timing?
  4. Are all appropriate downstream medical costs included?
  5. Are charges converted to costs appropriately?
  6. Are the best available data used to estimate costs? (like first question)
  7. Are all important and relevant costs and outcomes for each alternative identified?
Results
  1. Are incremental cost-effectiveness ratios presented?
  2. Are appropriate sensitivity analyses performed?
  3. How far do study results include all issues of concern to users?

Assessment of External Validity

Are the results generalizable to the setting of interest in the review?

Systematic Reviews

  1. Is the systematic review recent and relevant?
  2. Is the review comprehensive in considering sources and in searching databases to find all relevant research?
  3. Are inclusion/exclusion criteria reported relating to the primary studies that address the review question? If so, are they explicit and relevant?
  4. Are the primary studies summarized appropriately?
  5. Is sufficient detail of the primary studies presented?
  6. Is there standard appraisal of the primary studies?
  7. Is the validity of primary studies adequately assessed?
  8. Are there valid conclusions in the systematic review?
Copyright & 2005, Oregon Health & Science University, Portland, Oregon.
Bookshelf ID: NBK10417
Contents

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (1.4M)

In this Page

Other titles in this collection

Recent Activity

ClearTurn OffTurn On

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...