Challenges to estimating vaccine impact using hospitalization data

Vaccine. 2017 Jan 3;35(1):118-124. doi: 10.1016/j.vaccine.2016.11.030. Epub 2016 Nov 26.

Abstract

Because the real-world impact of new vaccines cannot be known before they are implemented in national programs, post-implementation studies at the population level are critical. Studies based on analysis of hospitalization rates of vaccine-preventable outcomes are typically used for this purpose. However, estimates of vaccine impact based on hospitalization data are particularly prone to confounding, as hospitalization rates are tightly linked to changes in the quality, access and use of the healthcare system, which often occur simultaneously with introduction of new vaccines. Here we illustrate how changes in healthcare delivery coincident with vaccine introduction can influence estimates of vaccine impact, using as an example reductions in infant pneumonia hospitalizations after introduction of the 10-valent pneumococcal conjugate vaccine (PCV10) in Brazil. To this end, we explore the effect of changes in several metrics of quality and access to public healthcare on trends in hospitalization rates before (2008-09) and after (2011-12) PCV10 introduction in 2010. Changes in infant pneumonia hospitalization rates following vaccine introduction were significantly associated with concomitant changes in hospital capacity and the fraction of the population using public hospitals. Importantly, reduction of pneumonia hospitalization rates after PCV10 were also associated with the expansion of outpatient services in several Brazilian states, falling more sharply where primary care coverage and the number of health units offering basic and emergency care increased more. We show that adjustments for unrelated (non-vaccine) trends commonly employed by impact studies, such as use of single control outcomes, are not always sufficient for accurate impact assessment. We discuss several ways to identify and overcome such biases, including sensitivity analyses using different denominators to calculate hospitalizations rates and methods that track changes in the outpatient setting. Employing these practices can improve the accuracy of vaccine impact estimates, particularly in evolving healthcare settings typical of low- and middle-income countries.

Keywords: Bias; Brazil; Confounding factors; Delivery of health care; Health impact assessment; Hospitalization; Latin America; Observational studies; Pneumococcal conjugate vaccines; Pneumococcal vaccines; Pneumococcus; Pneumonia; Public health; Vaccines.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Biostatistics
  • Brazil / epidemiology
  • Female
  • Hospitalization*
  • Humans
  • Infant
  • Male
  • Pneumococcal Vaccines / administration & dosage*
  • Pneumococcal Vaccines / immunology*
  • Pneumonia, Pneumococcal / epidemiology*
  • Pneumonia, Pneumococcal / prevention & control*
  • Treatment Outcome

Substances

  • 10-valent pneumococcal conjugate vaccine
  • Pneumococcal Vaccines