Decision making is not always easy, especially under complex circumstances. Deciding which vaccines—or which of any sort of health care products or services—that one should invest in requires a complex assessment of alternatives. And the planning process, which can consume massive amounts of resources, generally must contend with a variety of uncertainties and sometimes also biases and bits of “conventional wisdom” that may actually be incorrect.

New vaccine development is a demanding process. It is long and often arduous, and few appreciate the amount of work and resources that go toward producing and delivering what may seem to many like a trivial matter—say, half a milliliter of fluid contained in a vaccine vial or ampoule. The process typically consumes hundreds of millions of dollars, and its success relies on the co-evolution of scientific understanding, regulatory environment and requirements, production technologies, public health needs, human resource management, and often an understanding of the culture of the intended recipients of vaccination (Rappuoli et al., 2011).

The process for developing new vaccines has changed significantly over the last three decades (see Figure 1-1). In the 1980s the major obstacles for new vaccines were on the discovery side, while development was relatively easy, and the licensing process required only several hundreds of subjects evaluated in clinical trials (Rappuoli and Alderem, 2011).

FIGURE 1-1

Change in the time and resources required for vaccine discovery and development from 1980 to 2010. In the 1980s vaccine discovery required a long time, while development periods were relatively short. More recently, the time involved in new vaccine discovery (more...)

During the 1990s many new promising technologies, including recombinant DNA, conjugation, and genomics, emerged and aided vaccine discovery (Bagnoli et al., 2011). However, during the same period the timelines and budgets required for the development of vaccines soared (Milstien and Candries, 2002). The burgeoning research and development budgets coupled with increasing regulatory complexity and the increased time required for the development of a new vaccine made the decision process very challenging and necessitated the use of sophisticated models to predict the returns on the investment.

Moreover, while high-income countries face greatly increased lengths of time and various financial and scientific challenges when developing new vaccines, as illustrated in Figure 1-1, developing vaccines for use only in low-income countries is perhaps even more challenging, as there are fewer mechanisms in place to develop those vaccines (Batson, 2005; Rappuoli and Alderem, 2011). It may take a number of additional years after a vaccine is commercially available in high-income countries to introduce the same vaccine in low-income countries. And even if a vaccine is available in high-income countries, it may be the case that models are unable to justify the investment required for the development of the same vaccine in low-income countries, where they may not be a profitable market; this is a particular challenge with innovative vaccines. An example is a conjugate vaccine against meningococcus A that was developed specifically for a low-income region of sub-Saharan Africa. The development effort was made possible only through the work of a vaccine manufacturer from a low-income country with the support of the Bill & Melinda Gates Foundation—even though these meningococcal conjugates were already developed and licensed in high-income countries (Bishai et al., 2011). This inequity between high- and low-income countries needs to be captured within decision-support tools in order to emphasize economic and health returns.

Another example that illustrates the need for decision-support tools concerns the introduction of improved vaccines. Older vaccines continue to be used because the business models are not able to justify the investment necessary to improve those products. An example is the pertussis vaccine. Because of reactions associated with the older whole-cell pertussis vaccines developed and licensed in the late 1940s, new vaccine development became a high priority for research funding agencies, regulatory bodies, and industry in the 1980s. New, more highly purified acellular vaccines were developed and licensed in the mid-1990s and were shown to have an excellent safety profile. After that success, interest in the science of pertussis decreased, and little effort was made to improve the vaccine further. The situation has recently changed with the finding that the immunity created by the acellular vaccines appears to be not as long-lasting as the immunity from the whole cell vaccine. Now, with the shortcomings of the pertussis vaccines apparent, funding agencies are being asked to support research in the biology of pertussis, and regulatory agencies are being requested to find innovative ways to license new pertussis vaccines in the absence of efficacy trials. However, private industry has little incentive to invest in this work because a company cannot justify investing in a new full-fledged development program without proof of concept, a clear regulatory strategy, a price point advantage or an authoritative use recommendation that will generate a return on its investment.

Yet another example illustrating the need for a comprehensive prioritization model concerns discounting, which typically puts vaccination at a disadvantage to therapeutic interventions in a company's financial calculations. In most calculations the benefit of an intervention is captured in full for the first year and then discounted in following years. This is not an issue for a therapeutic intervention, where the cost occurs very close to the benefit. However, it is an issue for vaccines because the benefits occur many years after vaccination. Therefore, applying similar discounting methods to both vaccines and therapeutics—which is typical—can have a strong influence on the outcome of models that are based solely on cost-effectiveness and thus can profoundly affect the resulting decisions (Bloom et al., 2005).

Finally, there are some features that are unique to vaccination and that make impact assessment even more complicated. One of these is the concept of herd immunity, which refers to the fact that vaccines protect not only the vaccinated subjects but also unvaccinated people by reducing the circulation of a pathogen (Drummond et al., 2007). This benefit is realized several years after implementation of the primary intervention and is often not included in most cost-effectiveness models. Furthermore, if herd immunity is included in the model, it is discounted, thus reducing the calculated true value of the intervention. Yet herd immunity can have major benefits. An extreme example is the eradication of the pathogen that causes a disease. For example, smallpox has been eradicated, and polio is on the verge of eradication (Brilliant and Foege, 2013; Tomori, 2011). Thus, it is important that the impact of herd immunity be adequately captured in decision models.

Today, decision-support frameworks provide guidance in planning and prioritizing many of the above-mentioned scenarios. However, those involved in such assessments plan and prioritize development and implementation processes in their own ways, which are sometimes proprietary. Decision making related to the development and implementation of vaccines is complex and involves many stakeholders, including vaccine manufacturers, public and private funding agencies, nongovernmental organizations, regulators, and purchasers. Each of these partners needs tools or mechanisms to compare the relative benefits of different vaccines in their portfolio, of new vaccines that may become available, and of vaccines weighed against other interventions. Decision making involves understanding the existing and emerging landscape of vaccine development, the real benefits that vaccination brings to society, and the limitations of the decision models available today. Sound decision making sometimes also involves persuading others; for example, a minister of health may need to convince the minister of finance about the value of a given vaccine (or a vaccination program) and why it should be prioritized versus other interventions.

Decision makers in different areas look at different factors in making their decisions. Industrial executives, for instance, may need to first evaluate the technical feasibility and the projected efficacy of a new vaccine and then decide whether the investment in a particular vaccine provides better return to the investors than an investment in other options, such as therapeutic drugs, where profit margins are usually higher. Funders of vaccine research, development, and implementation prioritize different vaccines in different countries for different reasons. A tool that facilitates an assessment of the decision-making process of all the diverse, independent, and sometimes conflicting stakeholders would greatly improve the quality of the discussions and decisions related to individual and public health priorities. This is reinforced by the fact that all stakeholders operate under conditions of limited resources and must choose among alternatives.

The need for a better and more comprehensive tool that can support different entities is also underscored by the abundance of varying perspectives within the vaccine enterprise. The stakeholders involved in vaccine development range from government entities to public and private funding organizations, vaccine manufacturers, and vaccine program implementation leaders. It was in order to accommodate the many different scenarios and even more viewpoints that the Institute of Medicine (IOM) Committee on Identifying and Prioritizing New Vaccines for Development designed SMART Vaccines. This software has been developed, keeping various stakeholders in mind, to provide a more consistent method for informing decisions and to offer an analytical base for reaching individual or collective decisions.

Study Context and Scope

A critical development in the realm of vaccine policy was the release of the 2010 National Vaccine Plan by the U.S. Department of Health and Human Services' National Vaccine Program Office (NVPO) (HHS, 2010). The plan's various goals and priorities make it compellingly clear that all strata of the vaccine enterprise must work toward the development of safe, effective vaccines that are important to global public health. The first goal of the plan is to “develop new and improved vaccines,” with one of the corresponding implementation priorities relating to the development of a catalogue of vaccine targets that are domestically and globally important (highlighted as bold text in Box 1-1).

BOX 1-1

The 2010 National Vaccine Plan U.S. Department of Health and Human Services. Develop new and improved vaccines. Enhance the vaccine safety system.

In order to achieve the first goal of the National Vaccine Plan 2010, the NVPO requested that the IOM conduct a study to create a framework for prioritizing vaccines. This work has been carried out in two phases, whose places in the overall plan are shown in Figure 1-2, which outlines the tasks required to reach the ultimate vision of creating a catalogue of priority vaccines for both domestic and international importance. The Phase I committee developed a multi-attribute utility framework and a blueprint for software named the Strategic Multi-Attribute Ranking Tool for Vaccines—or SMART Vaccines Beta.

FIGURE 1-2

Steps needed to achieve the first goal of the National Vaccine Plan, according to the National Vaccine Program Office of the Department of Health and Human Services. Phase I in Step 1 resulted in the Institute of Medicine's Ranking Vaccines: A Prioritization (more...)

The Phase II committee continued the Phase I committee's work by refining the model underpinning SMART Vaccines Beta. (See Box S-1 for the Statement of Task.) The current enhanced version of the software, SMART Vaccines 1.0, is a product of continuous stakeholder feedback coupled with the committee's deliberations and has been made available for public use. The committee has also expanded the datasets available for use with the software and evaluated three additional vaccine candidates for the United States and South Africa. The combined group of vaccine candidates consists of vaccines for influenza, tuberculosis, group B streptococcus, human papillomavirus, pneumococcal infection, and rotavirus.

Study Process and Feedback from Stakeholders

In the summer of 2012, immediately after the release of the Phase I report Ranking Vaccines: A Prioritization Framework (IOM, 2012), an 18–member committee was formed that contained some members who had served on the Phase I committee plus some new members. (Appendix F contains the biographical information of the members.) To accomplish its task, the committee held three committee meetings as well as several ad hoc subgroup committee meetings held via teleconference. The committee worked with eight consultants, one of whom assisted with modeling and programming, while the others helped to evaluate an early prototype version of SMART Vaccines 1.0.

As with any software application, the development of SMART Vaccines followed an iterative process. The committee went through multiple versions of the software, each of which took into account feedback and suggested refinements from stakeholders. To gather feedback on the model, software, and data, the committee organized and conducted several feedback-gathering sessions with interested stakeholders and the public.

As part of the efforts to gather public feedback, the committee members used a variety of formats for demonstrating the concept and utility of SMART Vaccines. Webinars, teleconferences, plenary talks, group discussions, and presentations were offered for a variety of audiences that included representatives from federal advisory groups, professional societies, policy groups, international governmental agencies, private industry, and philanthropic and trade organizations. The committee also organized an international stakeholder workshop to obtain additional feedback for use in improving the functionality of SMART Vaccines. (See Appendix E for a listing of speakers.)

The questions posed to the speakers fell into five main categories: usefulness, preliminary usability, data needs, application development, and possibilities for outreach (described in Box 1-2). In the course of numerous public presentations about SMART Vaccines (based on the Phase I report), the committee members received many comments and questions. Table 1-1 contains a listing of the most common questions and comments from stakeholders, along with the committee's response and commentary.

BOX 1-2

Framing Questions for Stakeholders' Feedback. Do you think SMART Vaccines could be useful to you as you make decisions regarding new vaccine development and prioritization? Please elaborate on how you might use it and for what purposes. How does this (more...)

TABLE 1-1

Frequently Asked Questions and the Committee's Responses.

The committee took the gathered feedback into account in its deliberations on refining the model, on informing the data collection for additional vaccine candidates, and on redesigning the software interface. Those efforts are detailed in Chapter 2.