Introduction: New Vaccines and SMART Vaccines

Guruprasad Madhavan; Kinpritma Sangha; Charles Phelps; Dennis Fryback; Rino Rappuoli; Rose Marie Martinez; Lonnie King; Committee on Identifying and Prioritizing New Preventive Vaccines for Development, Phase II; Board on Population Health and Public Health Practice; Board on Global Health; Institute of Medicine

1Introduction: New Vaccines and SMART Vaccines

Decision making is not always easy, especially under complex circumstances. Deciding which vaccines—or which of any sort of health care products or services—that one should invest in requires a complex assessment of alternatives. And the planning process, which can consume massive amounts of resources, generally must contend with a variety of uncertainties and sometimes also biases and bits of “conventional wisdom” that may actually be incorrect.

New vaccine development is a demanding process. It is long and often arduous, and few appreciate the amount of work and resources that go toward producing and delivering what may seem to many like a trivial matter—say, half a milliliter of fluid contained in a vaccine vial or ampoule. The process typically consumes hundreds of millions of dollars, and its success relies on the co-evolution of scientific understanding, regulatory environment and requirements, production technologies, public health needs, human resource management, and often an understanding of the culture of the intended recipients of vaccination (Rappuoli et al., 2011).

The process for developing new vaccines has changed significantly over the last three decades (see Figure 1-1). In the 1980s the major obstacles for new vaccines were on the discovery side, while development was relatively easy, and the licensing process required only several hundreds of subjects evaluated in clinical trials (Rappuoli and Alderem, 2011).

FIGURE 1-1

During the 1990s many new promising technologies, including recombinant DNA, conjugation, and genomics, emerged and aided vaccine discovery (Bagnoli et al., 2011). However, during the same period the timelines and budgets required for the development of vaccines soared (Milstien and Candries, 2002). The burgeoning research and development budgets coupled with increasing regulatory complexity and the increased time required for the development of a new vaccine made the decision process very challenging and necessitated the use of sophisticated models to predict the returns on the investment.

Moreover, while high-income countries face greatly increased lengths of time and various financial and scientific challenges when developing new vaccines, as illustrated in Figure 1-1, developing vaccines for use only in low-income countries is perhaps even more challenging, as there are fewer mechanisms in place to develop those vaccines (Batson, 2005; Rappuoli and Alderem, 2011). It may take a number of additional years after a vaccine is commercially available in high-income countries to introduce the same vaccine in low-income countries. And even if a vaccine is available in high-income countries, it may be the case that models are unable to justify the investment required for the development of the same vaccine in low-income countries, where they may not be a profitable market; this is a particular challenge with innovative vaccines. An example is a conjugate vaccine against meningococcus A that was developed specifically for a low-income region of sub-Saharan Africa. The development effort was made possible only through the work of a vaccine manufacturer from a low-income country with the support of the Bill & Melinda Gates Foundation—even though these meningococcal conjugates were already developed and licensed in high-income countries (Bishai et al., 2011). This inequity between high- and low-income countries needs to be captured within decision-support tools in order to emphasize economic and health returns.

Another example that illustrates the need for decision-support tools concerns the introduction of improved vaccines. Older vaccines continue to be used because the business models are not able to justify the investment necessary to improve those products. An example is the pertussis vaccine. Because of reactions associated with the older whole-cell pertussis vaccines developed and licensed in the late 1940s, new vaccine development became a high priority for research funding agencies, regulatory bodies, and industry in the 1980s. New, more highly purified acellular vaccines were developed and licensed in the mid-1990s and were shown to have an excellent safety profile. After that success, interest in the science of pertussis decreased, and little effort was made to improve the vaccine further. The situation has recently changed with the finding that the immunity created by the acellular vaccines appears to be not as long-lasting as the immunity from the whole cell vaccine. Now, with the shortcomings of the pertussis vaccines apparent, funding agencies are being asked to support research in the biology of pertussis, and regulatory agencies are being requested to find innovative ways to license new pertussis vaccines in the absence of efficacy trials. However, private industry has little incentive to invest in this work because a company cannot justify investing in a new full-fledged development program without proof of concept, a clear regulatory strategy, a price point advantage or an authoritative use recommendation that will generate a return on its investment.

Yet another example illustrating the need for a comprehensive prioritization model concerns discounting, which typically puts vaccination at a disadvantage to therapeutic interventions in a company's financial calculations. In most calculations the benefit of an intervention is captured in full for the first year and then discounted in following years. This is not an issue for a therapeutic intervention, where the cost occurs very close to the benefit. However, it is an issue for vaccines because the benefits occur many years after vaccination. Therefore, applying similar discounting methods to both vaccines and therapeutics—which is typical—can have a strong influence on the outcome of models that are based solely on cost-effectiveness and thus can profoundly affect the resulting decisions (Bloom et al., 2005).

Finally, there are some features that are unique to vaccination and that make impact assessment even more complicated. One of these is the concept of herd immunity, which refers to the fact that vaccines protect not only the vaccinated subjects but also unvaccinated people by reducing the circulation of a pathogen (Drummond et al., 2007). This benefit is realized several years after implementation of the primary intervention and is often not included in most cost-effectiveness models. Furthermore, if herd immunity is included in the model, it is discounted, thus reducing the calculated true value of the intervention. Yet herd immunity can have major benefits. An extreme example is the eradication of the pathogen that causes a disease. For example, smallpox has been eradicated, and polio is on the verge of eradication (Brilliant and Foege, 2013; Tomori, 2011). Thus, it is important that the impact of herd immunity be adequately captured in decision models.

Today, decision-support frameworks provide guidance in planning and prioritizing many of the above-mentioned scenarios. However, those involved in such assessments plan and prioritize development and implementation processes in their own ways, which are sometimes proprietary. Decision making related to the development and implementation of vaccines is complex and involves many stakeholders, including vaccine manufacturers, public and private funding agencies, nongovernmental organizations, regulators, and purchasers. Each of these partners needs tools or mechanisms to compare the relative benefits of different vaccines in their portfolio, of new vaccines that may become available, and of vaccines weighed against other interventions. Decision making involves understanding the existing and emerging landscape of vaccine development, the real benefits that vaccination brings to society, and the limitations of the decision models available today. Sound decision making sometimes also involves persuading others; for example, a minister of health may need to convince the minister of finance about the value of a given vaccine (or a vaccination program) and why it should be prioritized versus other interventions.

Decision makers in different areas look at different factors in making their decisions. Industrial executives, for instance, may need to first evaluate the technical feasibility and the projected efficacy of a new vaccine and then decide whether the investment in a particular vaccine provides better return to the investors than an investment in other options, such as therapeutic drugs, where profit margins are usually higher. Funders of vaccine research, development, and implementation prioritize different vaccines in different countries for different reasons. A tool that facilitates an assessment of the decision-making process of all the diverse, independent, and sometimes conflicting stakeholders would greatly improve the quality of the discussions and decisions related to individual and public health priorities. This is reinforced by the fact that all stakeholders operate under conditions of limited resources and must choose among alternatives.

The need for a better and more comprehensive tool that can support different entities is also underscored by the abundance of varying perspectives within the vaccine enterprise. The stakeholders involved in vaccine development range from government entities to public and private funding organizations, vaccine manufacturers, and vaccine program implementation leaders. It was in order to accommodate the many different scenarios and even more viewpoints that the Institute of Medicine (IOM) Committee on Identifying and Prioritizing New Vaccines for Development designed SMART Vaccines. This software has been developed, keeping various stakeholders in mind, to provide a more consistent method for informing decisions and to offer an analytical base for reaching individual or collective decisions.

Study Context and Scope

A critical development in the realm of vaccine policy was the release of the 2010 National Vaccine Plan by the U.S. Department of Health and Human Services' National Vaccine Program Office (NVPO) (HHS, 2010). The plan's various goals and priorities make it compellingly clear that all strata of the vaccine enterprise must work toward the development of safe, effective vaccines that are important to global public health. The first goal of the plan is to “develop new and improved vaccines,” with one of the corresponding implementation priorities relating to the development of a catalogue of vaccine targets that are domestically and globally important (highlighted as bold text in Box 1-1).

BOX 1-1

The 2010 National Vaccine Plan U.S. Department of Health and Human Services. Develop new and improved vaccines. Enhance the vaccine safety system.

In order to achieve the first goal of the National Vaccine Plan 2010, the NVPO requested that the IOM conduct a study to create a framework for prioritizing vaccines. This work has been carried out in two phases, whose places in the overall plan are shown in Figure 1-2, which outlines the tasks required to reach the ultimate vision of creating a catalogue of priority vaccines for both domestic and international importance. The Phase I committee developed a multi-attribute utility framework and a blueprint for software named the Strategic Multi-Attribute Ranking Tool for Vaccines—or SMART Vaccines Beta.

FIGURE 1-2

The Phase II committee continued the Phase I committee's work by refining the model underpinning SMART Vaccines Beta. (See Box S-1 for the Statement of Task.) The current enhanced version of the software, SMART Vaccines 1.0, is a product of continuous stakeholder feedback coupled with the committee's deliberations and has been made available for public use. The committee has also expanded the datasets available for use with the software and evaluated three additional vaccine candidates for the United States and South Africa. The combined group of vaccine candidates consists of vaccines for influenza, tuberculosis, group B streptococcus, human papillomavirus, pneumococcal infection, and rotavirus.

Study Process and Feedback from Stakeholders

In the summer of 2012, immediately after the release of the Phase I report Ranking Vaccines: A Prioritization Framework (IOM, 2012), an 18–member committee was formed that contained some members who had served on the Phase I committee plus some new members. (Appendix F contains the biographical information of the members.) To accomplish its task, the committee held three committee meetings as well as several ad hoc subgroup committee meetings held via teleconference. The committee worked with eight consultants, one of whom assisted with modeling and programming, while the others helped to evaluate an early prototype version of SMART Vaccines 1.0.

As with any software application, the development of SMART Vaccines followed an iterative process. The committee went through multiple versions of the software, each of which took into account feedback and suggested refinements from stakeholders. To gather feedback on the model, software, and data, the committee organized and conducted several feedback-gathering sessions with interested stakeholders and the public.

As part of the efforts to gather public feedback, the committee members used a variety of formats for demonstrating the concept and utility of SMART Vaccines. Webinars, teleconferences, plenary talks, group discussions, and presentations were offered for a variety of audiences that included representatives from federal advisory groups, professional societies, policy groups, international governmental agencies, private industry, and philanthropic and trade organizations. The committee also organized an international stakeholder workshop to obtain additional feedback for use in improving the functionality of SMART Vaccines. (See Appendix E for a listing of speakers.)

The questions posed to the speakers fell into five main categories: usefulness, preliminary usability, data needs, application development, and possibilities for outreach (described in Box 1-2). In the course of numerous public presentations about SMART Vaccines (based on the Phase I report), the committee members received many comments and questions. Table 1-1 contains a listing of the most common questions and comments from stakeholders, along with the committee's response and commentary.

BOX 1-2

Framing Questions for Stakeholders' Feedback. Do you think SMART Vaccines could be useful to you as you make decisions regarding new vaccine development and prioritization? Please elaborate on how you might use it and for what purposes. How does this (more...)

TABLE 1-1

Frequently Asked Questions and the Committee's Responses.

The committee took the gathered feedback into account in its deliberations on refining the model, on informing the data collection for additional vaccine candidates, and on redesigning the software interface. Those efforts are detailed in Chapter 2.

Publication Details

Copyright

Publisher

National Academies Press (US), Washington (DC)

NLM Citation

Committee on Identifying and Prioritizing New Preventive Vaccines for Development, Phase II; Board on Population Health and Public Health Practice; Board on Global Health; Institute of Medicine; Madhavan G, Sangha K, Phelps C, et al., editors. Ranking Vaccines: A Prioritization Software Tool: Phase II: Prototype of a Decision-Support System. Washington (DC): National Academies Press (US); 2013 Oct 17. 1, Introduction: New Vaccines and SMART Vaccines.

FIGURE 1-1

Change in the time and resources required for vaccine discovery and development from 1980 to 2010. In the 1980s vaccine discovery required a long time, while development periods were relatively short. More recently, the time involved in new vaccine discovery has shortened dramatically due to the availability of various new technologies. However, in the meantime the regulatory requirements have lengthened the development timelines substantially. This results in longer times for vaccine licensure and significant increases in development costs.

SOURCE: Adapted from Rappuoli and Alderem, 2011.

BOX 1-1The 2010 National Vaccine Plan U.S. Department of Health and Human Services

Goals

1.: Develop new and improved vaccines.

2.: Enhance the vaccine safety system.

3.: Support communications to enhance informed vaccine decision making.

4.: Ensure a stable supply of, access to, and better use of recommended vaccines in the United States.

Priorities

A.: Develop a catalogue of priority vaccine targets of domestic and global health importance.

B.: Strengthen the science base for the development and licensure of new vaccines.

C.: Enhance timely detection and verification of vaccine safety signals and develop a vaccine safety scientific agenda.

D.: Increase awareness of vaccines, vaccine-preventable diseases, and the benefits/risks of immunization among the public, providers, and other stakeholders.

E.: Use evidence-based science to enhance vaccine-preventable disease surveillance, measurement of vaccine coverage, and measurement of vaccine effectiveness.

F.: Eliminate financial barriers for providers and consumers to facilitate access to routinely recommended vaccines.

G.: Create an adequate and stable supply of routinely recommended vaccines and vaccines for public health preparedness.

H.: Increase and improve the use of interoperable health information technology and electronic health records.

I.: Improve global surveillance for vaccine-preventable diseases and strengthen global health information systems to monitor vaccine coverage, effectiveness, and safety.

J.: Support global introduction and availability of new and under-utilized vaccines to prevent diseases of public health importance.

FIGURE 1-2

Steps needed to achieve the first goal of the National Vaccine Plan, according to the National Vaccine Program Office of the Department of Health and Human Services. Phase I in Step 1 resulted in the Institute of Medicine's Ranking Vaccines: A Prioritization Framework as well as the blueprint of the software SMART Vaccines Beta (IOM, 2012). Phase II in Step 1 (highlighted in yellow) relates to this report, Ranking Vaccines: A Prioritization Software Tool, and SMART Vaccines 1.0. Step 2 (Phase III) involves the data architecture and software usability studies, with Step 3 efforts ultimately resulting in a catalogue of domestically and globally significant vaccine candidates.

BOX 1-2Framing Questions for Stakeholders' Feedback

Usefulness

Do you think SMART Vaccines could be useful to you as you make decisions regarding new vaccine development and prioritization? Please elaborate on how you might use it and for what purposes. How does this approach complement and/or differ from your current decision-making approach?

Usability

Does SMART Vaccines cover the most relevant issues related to vaccine development and prioritization? Is the current software version user-friendly? Please comment on the ease of under-standing how to use it and the demands on the user.

Data Library

How should the committee address the intensive needs for data inputs into the model? How should the user groups think about data requirements and resources for data collection and standardization?

Application Development

In what ways can SMART Vaccines be enhanced?

Outreach

What advice can you give regarding how best to engage various user groups and decision makers to use—and further develop—SMART Vaccines?

TABLE 1-1Frequently Asked Questions and the Committee's Responses

Stakeholder Question	Committee's Response
Is there not a risk that the multi-attribute utility model underlying SMART Vaccines can be “gamed” so that users get the rankings they wanted in the first place?	Technically yes, but SMART Vaccines makes explicit what has previously remained hidden from view. The committee anticipates and hopes that when various users begin to discuss the rankings they have produced using SMART Vaccines, others will insist that each user make clear the levels of attributes they have assigned to various vaccine candidates (including cost, efficacy, coverage, side effects) and the multi-attribute utility value weights. With these data available for open discussion, various parties can compare their inputs and results and reach an understanding on what drives each user's results.
Should the most important variable in the system be life-years saved? Why bother with anything else?	Previous ranked lists, including the 1985–1986 and 2000 reports from the Institute of Medicine, used a single attribute for vaccine prioritization: The 1985–1986 reports used a metric similar to life-years saved, and the 2000 report used an efficiency measure of cost-effectiveness measured as cost per quality-adjusted life year (IOM, 1985, 1986, 2000). But both studies stated clearly in their reports that many other issues would guide final decision making on vaccine priorities. SMART Vaccines seeks to make explicit exactly how these “other issues” affect the decisions. There will still remain issues and attributes not taken into account by SMART Vaccines, but the committee believes that making these considerations explicit will improve decision making and communication among affected and interested parties.
SMART Vaccines is of limited use without much better data, is it not?	Yes. The committee not only agrees with this, but hopes that the creation of SMART Vaccines will accelerate the production of necessary data. In the absence of such data, decisions continue to be made, and the committee believes that decision making about vaccine priority ranking will improve with the production of better data and the use of a carefully structured model such as SMART Vaccines. This report concludes with some strategic steps that the committee believes will greatly enhance the production of high-quality datasets for use in SMART Vaccines.
It is unusual to place “corporate profits” into a social welfare function such as created by SMART Vaccines, is it not?	SMART Vaccines does not create a classic social welfare function. Users can do such by choosing attributes in the multi-attribute utility model and weights attached to those attributes that are consistent with traditional economic models of social welfare maximization. But it is not limited to that use. For example, vaccine manufacturers can also use SMART Vaccines to measure value from their own viewpoint (including, presumably, corporate profitability) and also to help them understand the values and resultant rankings of their potential customers.
SMART Vaccines creates a large data burden on users, does it not?	To some extent, yes, but if one carefully assesses the data needed to analyze the related issues intelligently, it becomes apparent that the data needs are driven by the intrinsic issue at hand, not the software. The committee has sought to make the best possible use of extant databases that will help SMART Vaccine users simplify the data burden, including, for example, population data (from the World Health Organization) and other data on burden of disease and related issues.
Would not the rankings from SMART Vaccines become useless if, for example, some new treatment emerges for a disease for which a new vaccine is under development?	Yes, but that remains true whether people have used SMART Vaccines or not. It cannot predict the emergence of disruptive technologies. It can readily re-estimate the priority scores in the presence of new information, and all rankings should be re-calculated when conditions surrounding any vaccine's potential use change.
How can you expect decision makers to deal with the complexity of this software program?	In general, the committee believes that high-level decision makers will not in fact have to deal with many facets of the software's complexity. More likely, specialized assistants to decision makers will create or import relevant data and possibly even carry out preliminary analyses using weights specified by the decision maker. The current version of SMART Vaccines provides entry points into the software at appropriate points for each possible type of user, ranging from technical data specialists to final decision makers.
How can I interpret what the scores from SMART Vaccines mean?	Each user's particular set of values and weights helps define the scale for the final priority score, so users cannot compare scores from one user to another unless they use the same attributes and endpoints. This is a standard feature of multi-attribute utility models. Some users have found it useful to think about the priority scores in the same way that we think about reports of temperature. In a Fahrenheit scale the difference between 50°F and 70°F (20 degrees) has the same meaning as the difference between 20°F and 40°F. However, in the Fahrenheit scale 40°F is not twice as hot as 20°F. Similarly, on a Celsius scale the difference between 20°C and 30°C (10 degrees) has the same meaning as the difference between 10°C and 20°C, but 20°C is not twice as hot as 10°C. Furthermore, 20°C and 20°F do not have the same meaning. These differences do not make thermometers useless, but they do require an “anchor” to interpret them. With thermometers, we can use standard reference points to help understand what 20°F and 20°C mean. We know that water freezes at 0°C and boils at 100°C, and similarly that water freezes at 32°F and boils at 212°F. Knowing these two pairs of values allows us to make direct comparisons between Fahrenheit and Celsius values, and we can calculate that they have the same meaning at only one temperature—that is, minus 40°C has the same value as minus 40°F.
Who is expected to use SMART Vaccines and why?	Potential users of SMART Vaccines (individually or collaboratively) include decision makers in a wide range of constituencies: federal and private research groups, funders, vaccine manufacturers, purchasers of vaccines, regulators, and nongovernmental groups. SMART Vaccines offers a new framework that could help provide a new standard for decision making among various stakeholders in many circumstances such as decision making under opacity; prioritizing under constrained resources, complexities associated with globalization, economies, and health. Furthermore, changing realities need decision models to be refreshed, which is what this tool offers—a dynamic, living decision-support framework that can be updated as new data, diseases and potential vaccine candidates emerge.