Monitoring and evaluation of Xpert MTB/RIF implementation is necessary to ensure the effective and efficient use of resources and also to measure the impact of Xpert MTB/RIF in order to guide and justify further scale-up.
8.1. Routine monitoring
At the site level, monitoring the use of Xpert MTB/RIF ensures that established diagnostic algorithms are being followed, detects whether a particular instrument module is functioning suboptimally or whether any users require additional training, and allows supplies to be effectively managed. Site-level information should be shared with the supervising regional or national reference laboratory; this will allow the relevant laboratory to provide guidance on any actions that need to be undertaken to improve effectiveness, efficiency or user performance, and to strengthen the supply-management process to prevent stock-outs or cartridges from expiring by exchanging cartridges among sites.
The key data that are recommended to be collected monthly or quarterly fall into three main categories.
Monitoring the groups of patient tested and the test results:
the number of Xpert MTB/RIF tests performed, disaggregated by reason for testing (that is, by the group of either TB patients or individuals suspected of having TB);
the number of tests with MTB DETECTED, Rif resistance NOT DETECTED;
the number of tests with MTB DETECTED, Rif resistance DETECTED;
the number of tests with MTB DETECTED, Rif resistance INDETERMINATE;
the number of tests with MTB NOT DETECTED;
the number of tests that had invalid results, no results or other errors.
Monitoring the operation of the GeneXpert platform and the performance of users:
the number and types of various errors. Identifying the most frequent types of errors can help troubleshoot the process, given that certain errors may be associated with the technique used to process specimens; other errors may be related to mechanical problems with the instrument's modules or other issues, such as room temperature;
the number of errors occurring by instrument module. If a particular module produces more errors over time compared with other modules, it may require repair;
the number of errors occurring by user. If a particular user has an unusually high number of errors, further investigation of the specific error types is warranted, since some errors may be caused by the technique used to process specimens;
the number of tests lost due to power outages or surges;
the number, duration, and causes of routine interruptions in the Xpert MTB/RIF testing service. Common causes of service interruptions include cartridge stock-outs, expired cartridges, no staff available, instrument breakdown, and computer breakdown;
the number of instrument modules not functioning and the duration (in days) of module failure during the reporting period;
the number of instrument modules overdue for calibration at the end of the reporting period.
Remote monitoring tools that automatically send results via the Internet to a central country focal point greatly facilitate performance monitoring. Cepheid expects to roll out its remote monitoring platform in June 2014. In early 2014, other tools that serve this purpose include those developed by Abt Associates (GXAlert)
41 and Interactive Research and Development (XpertSMS)
42.
Monitoring supply management:
the number of cartridges in stock at the beginning of the reporting period;
the number of cartridges received during the reporting period;
the number of cartridges used during the reporting period;
the number of cartridges that were lost or damaged;
the number of cartridges in stock at the end of the reporting period ;
whether there were any stock-outs during the reporting period, the duration of stock-out (in days);
Number of cartridges that expired before being used.
8.2. Measuring the impact
In order to understand the impact of Xpert MTB/RIF on case detection, the management of patients, and other laboratory processes, additional data need to be collected from other sources in the laboratory or at the district level or treatment-facility level. Because impact can be assessed only using a comparator, baseline data from a year before Xpert MTB/RIF was introduced are needed.
Sites introducing Xpert MTB/RIF usually observe a significant increase in the number of bacteriologically confirmed TB cases. In order to measure this increase, the following data should be collected:
the number and proportion of incident cases (both new and relapsed) confirmed by microscopy, culture, or Xpert MTB/RIF, or a combination of these, during the reporting period after the introduction of Xpert MTB/RIF;
the number and proportion of incident cases (both new and relapsed) confirmed by microscopy or culture, or both, during an analogous reporting period before the introduction of Xpert MTB/RIF.
Some sites and countries have reported that the introduction of Xpert MTB/RIF has not resulted in an overall increase in TB case notification. This is usually found in settings where a large number of patients have been diagnosed based on a clinical assessment. Diagnosing TB based on the clinical evidence alone can result in patients being falsely diagnosed with TB and receiving unnecessary treatment. If an increase in the proportion of bacteriologically confirmed cases is observed, it can provide assurance that the risk of misdiagnosis and unnecessary treatment has been reduced.
Unless culture and phenotypic DST or LPA were already widely in use, sites introducing Xpert MTB/RIF will observe increases in the number of rifampicin-resistant cases detected. To evaluate the impact of Xpert MTB/RIF on the diagnosis of rifampicin-resistant cases and MDR-TB cases, and to ensure that patients with detected resistance receive appropriate follow-up testing and treatment, the following data should be collected and monitored:
the number and proportion of individuals found to have rifampicin-resistant TB by any method during the reporting period, and the number and proportion found to have rifampicin-resistant TB during an analogous reporting period before Xpert MTB/RIF was introduced, disaggregated by patient group;
the number and proportion of rifampicin-resistant cases detected by Xpert MTB/RIF that received further phenotypic DST during the reporting period;
the number and proportion of rifampicin-resistant cases detected by Xpert MTB/RIF that were initiated on a WHO-recommended treatment regimen for MDR during the reporting period.
In order to understand the impact of introducing Xpert MTB/RIF on the number of other diagnostic tests being performed, the following data should be collected, as applicable, and compared with baseline data from an analogous period:
the number of smear-microscopy tests performed for diagnosis and for treatment follow-up;
the numbers of culture tests performed for diagnosis and for treatment follow-up;
the number of DST performed.
Other aspects of implementation – in particular data on cost-effectiveness and the impact on diagnostic delays and time to treatment initiation – are best collected by operational research studies rather than as part of the routine processes for monitoring and evaluation.
