The ideal study design for examining interventions has traditionally been high-quality RCTs. The randomization of study participants to treatment and comparator groups, when allocation is concealed, minimizes selection bias.¹⁴ Prospective assignment of participants to study arms prior to outcomes occurring helps avoid selection bias that may arise in retrospective NRSIs if the selection of participants into the study is conditional on an outcome related to the intervention being evaluated (e.g., if only survivors are included). When properly executed, the randomization process helps ensure that the study groups are comparable with respect to known and unknown baseline prognostic factors (i.e., confounders).¹⁵ RCTs particularly larger, better conducted studies, generally have registered and/or published protocols and may be required to report deviations from protocols. Such requirements may reduce the potential for bias arising from deviations from intended interventions and selective outcome reporting¹³ and may help promote consistency of treatments and measurements that may be challenging in NRSIs, particularly retrospective NRSIs.

Despite the above advantages, RCTs bear some important limitations. First, RCTs may be underpowered to detect differences between comparators in harms. RCTs may be of limited value in the assessment of harms of interventions because RCTs are frequently small and/or of too short duration for uncommon harms or longer-term harms to be detected. Given the relatively small sample sizes and short duration of most RCTs, they are frequently underpowered to detect differences on several measures of effectiveness prioritized for a given SR. Second, in the situation of rare diseases, RCTs may have to draw from a very small population of interest, which may make enrollment very challenging. Third, it may be unethical to perform an RCT due to the absence of clinical equipoise or for other reasons. Fourth, it may be infeasible or highly resource-intensive to conduct an RCT, for example to examine very long-term outcomes, which may be more important to patients. Depending on the topic, RCTs may focus on interim (or surrogate) outcomes instead of clinically important or patient-important outcomes. Finally, results of some RCTs may not be broadly applicable due to their narrow eligibility criteria for participants, tightly controlled implementation of interventions and comparators, smaller sample size, shorter duration, and focus on short-term, surrogate, and/or composite outcomes. However, applicability may be less of a concern for some RCT designs (e.g., large simple RCTs, pragmatic RCTs).