7.1.1. Search procedure

The search in bibliographic databases, trial registries as well as guideline databases and websites of guideline providers consists of the following steps:

1. if necessary, specification of the research question posed
2. modification of the research question to a searchable research question
3. formulation of a search concept
4. selection of databases
5. identification of search terms
6. formulation of the search strategies
7. quality assurance (in the case of a bibliographic search)
8. performance of the search
9. storage of the search results in text files and import into a reference management software programme (if a standardized export is possible)
10. documentation of the search

Relevant publications identified in the preliminary search are usually drawn upon to identify search terms and formulate the search strategy for bibliographic databases. As a quality assurance step, it is tested whether the search strategy developed in this way identifies known relevant primary publications (test set) with sufficient certainty. The test set is generated by using previous publications by other working groups (systematic reviews on the topic of interest). In addition, a formal internal quality assurance is performed taking the review by Sampson into account [478,479].

7.1.2. Bibliographic databases

7.1.3. Search in trial registries

The systematic search should identify not only published but also unpublished studies. In this context, “unpublished” means that the studies (or individual data sets) have not been published (or only partly) in scientific journals. Study publications are generally identified by means of a search in bibliographic databases such as MEDLINE or EMBASE. Trial registries can be drawn upon in the search for unpublished studies or data [352].

As a rule, the Institute's benefit assessments involve a search in large general trial registries, as well as meta-registries thereof. In particular these include the trial registry ClinicalTrials.gov of the US National Institute of Health (NIH), the WHO's meta-registry “International Clinical Trials Registry Platform Search Portal” (ICTRP), as well as the EU Clinical Trials Register of the European Medicines Agency (EMA). In the benefit assessment of drugs, two German meta-registries (“Pharmnet.BUND Klinische Prüfung” and “Arzneimittel-Informationssystem”, AMIS) as well as trial registries of the pharmaceutical industry (individual companies and meta-registries) are additionally screened. Searches in disease-specific trial registries are only performed in exceptional cases.

In addition to information on the existence of a study, some registries are also increasingly including study results. This applies, for example, to ClinicalTrials.gov and trial registries of the pharmaceutical industry. Providing the study in question is in principle relevant to the assessment, results from trial registries can be considered in the Institute's reports.

7.1.4. Clinical practice guideline databases and providers

If the aim of the search is to identify CPGs, it is primarily conducted in guideline databases (e.g. Guidelines International Network [G-I-N], the AWMF, or the National Guideline Clearinghouse [NGC]), and may be followed by a search on the websites of providers of specialist and multi-disciplinary guidelines. Whether a supplementary search for guidelines is performed in bibliographic databases depends on the type of report to be prepared.

For the search in guideline databases or websites of guideline providers, the search strategy to be applied is targeted towards the structure and options of the particular websites. Only a few websites allow a search with key words, so that generally the complete list of a website's published guidelines is screened. In addition, for the search in guideline databases or websites of guideline providers, a standardized export is often not possible. For this reason, the search and number of hits are documented in a standardized search protocol. The potentially relevant hits are documented in a literature management programme. Otherwise, the procedure is followed as described in Section 7.1.1.

Within a benefit assessment, guidelines are not categorically excluded as a source of information. However, a systematic search for guidelines is not usually conducted.

7.1.5. Requests to manufacturers

Within the framework of the Institute's benefit assessments, the manufacturers of the technologies to be assessed are usually asked to provide previously unpublished information. The aim of this request is to identify all studies and other information relevant to the benefit assessment, independent of their publication status. For drug assessments this request is usually made in 2 steps. In the first, the Institute asks the manufacturer to supply a complete overview of all studies conducted by the manufacturer on the drug to be assessed. If appropriate, the Institute defines the project-specific inclusion criteria for this overview. In the second, the Institute identifies studies relevant to the benefit assessment from the overview, and requests detailed information on these studies. This may refer to a request for unpublished studies, or for supplementary, previously unpublished information on published studies. Previously unpublished information considered in the benefit assessment will also be published in the Institute's reports in order to ensure transparency. The basis for the incorporation of previously unpublished information into the benefit assessment is the conclusion of an agreement on the transfer and publication of study information. This agreement is made between the Institute and the manufacturer involved before the submission of data (see sample contract [293]). It specifies the procedure, the requirements for the documents to be submitted, as well as their confidential and non-confidential components. If the manufacturer concerned does not agree to this contract and therefore does not agree in particular to the complete transfer of all information requested by the Institute, or does not completely transfer the information requested despite conclusion of the agreement, no further requests to the manufacturer will be made. This is to prevent biased results due to selective provision of information.

7.1.6. Other data sources for the search

7.1.7. Selection of relevant publications

Due to the primarily sensitive approach, the literature search in bibliographic databases results in a large number of citations that are not relevant to the assessment. The selection of relevant publications is made in several steps:

▪

Exclusion of definitely irrelevant publications (i.e. publications not fulfilling the inclusion or exclusion criteria of the report plan or project outline) through perusal of the titles, and, if available, the abstracts. This step can be divided into 2 in order to distinguish completely irrelevant publications from topic-related ones which, however, do not fulfil the inclusion or exclusion criteria. “Topic-related” refers, for example, to studies investigating the topic of interest but with a different study design or duration from that specified in the report plan or project outline.

▪

The full texts of the remaining potentially relevant publications are obtained. The decision on the inclusion of the study in the assessment concerned is then made on the basis of these documents.

▪

Depending on the research question, a supplementary third step is performed for the search in (clinical practice) guideline databases and on websites of guideline providers, where it is examined whether a methodological approach was adopted in the development and formulation of the guideline. This usually refers to the evidence base of the guideline (see Section 5.2). When preparing the report plan the Institute specifies a priori whether on the basis of the research question only evidence-based guidelines are to be considered in the particular report.

All selection steps are performed by 2 persons independently of each other. Discrepancies are resolved by discussion. In the first selection step, if doubts exist as to the relevance of a study, the corresponding full text is obtained and assessed. In this step, completely irrelevant publications may also be distinguished from topic-related ones.

The languages of publication are usually restricted to those of Western Europe. However, other foreign-language publications may also be included if the available information on these publications indicates that additional and relevant information for answering the research question is to be expected.

In the search for guidelines, the steps for the full-text screening (from the second screening onwards) are performed by 2 persons independently of each other. The quality assurance of the first screening step is conducted with the help of a standardized search protocol.

7.1.8. Documentation of the search

All steps in the search in bibliographic databases are documented. This especially includes

▪

the search strategy for the databases selected

▪

the search date

▪

the user interface

▪

the number of hits

▪

after perusal of all hits: documentation of the publications judged relevant to the research question posed (citations)

▪

after perusal of the full texts: documentation of the citations not judged relevant; alternatively, documentation of the topic-related publications that were, however, irrelevant for the report (in each case providing a reason for exclusion)

All other steps in the information retrieval procedure are also documented (e.g. correspondence with authors, queries to manufacturers).

7.1.9. Benefit assessments based on systematic reviews – supplementary search

In most cases a supplementary search for current primary studies is required; this search covers the period between the last date of the search conducted in the systematic review and the date of the search conducted by IQWiG in the preparation of its report. In benefit assessments based on systematic reviews, a supplementary update search for primary literature can only be dispensed with in justified exceptional cases. This applies, for example, if it is sufficiently certain that the result of the assessment would not be considerably changed by the inclusion of additional information or studies (robustness).

In addition, it may be necessary to conduct supplementary searches for primary literature for specific research questions not addressed in the systematic review.