| Is there not a risk that the multi-attribute utility model underlying SMART Vaccines can be “gamed” so that users get the rankings they wanted in the first place? | Technically yes, but SMART Vaccines makes explicit what has previously remained hidden from view. The committee anticipates and hopes that when various users begin to discuss the rankings they have produced using SMART Vaccines, others will insist that each user make clear the levels of attributes they have assigned to various vaccine candidates (including cost, efficacy, coverage, side effects) and the multi-attribute utility value weights. With these data available for open discussion, various parties can compare their inputs and results and reach an understanding on what drives each user's results. |
| Should the most important variable in the system be life-years saved? Why bother with anything else? | Previous ranked lists, including the 1985–1986 and 2000 reports from the Institute of Medicine, used a single attribute for vaccine prioritization: The 1985–1986 reports used a metric similar to life-years saved, and the 2000 report used an efficiency measure of cost-effectiveness measured as cost per quality-adjusted life year (IOM, 1985, 1986, 2000). But both studies stated clearly in their reports that many other issues would guide final decision making on vaccine priorities. SMART Vaccines seeks to make explicit exactly how these “other issues” affect the decisions. There will still remain issues and attributes not taken into account by SMART Vaccines, but the committee believes that making these considerations explicit will improve decision making and communication among affected and interested parties. |
| SMART Vaccines is of limited use without much better data, is it not? | Yes. The committee not only agrees with this, but hopes that the creation of SMART Vaccines will accelerate the production of necessary data. In the absence of such data, decisions continue to be made, and the committee believes that decision making about vaccine priority ranking will improve with the production of better data and the use of a carefully structured model such as SMART Vaccines. This report concludes with some strategic steps that the committee believes will greatly enhance the production of high-quality datasets for use in SMART Vaccines. |
| It is unusual to place “corporate profits” into a social welfare function such as created by SMART Vaccines, is it not? | SMART Vaccines does not create a classic social welfare function. Users can do such by choosing attributes in the multi-attribute utility model and weights attached to those attributes that are consistent with traditional economic models of social welfare maximization. But it is not limited to that use. For example, vaccine manufacturers can also use SMART Vaccines to measure value from their own viewpoint (including, presumably, corporate profitability) and also to help them understand the values and resultant rankings of their potential customers. |
| SMART Vaccines creates a large data burden on users, does it not? | To some extent, yes, but if one carefully assesses the data needed to analyze the related issues intelligently, it becomes apparent that the data needs are driven by the intrinsic issue at hand, not the software. The committee has sought to make the best possible use of extant databases that will help SMART Vaccine users simplify the data burden, including, for example, population data (from the World Health Organization) and other data on burden of disease and related issues. |
| Would not the rankings from SMART Vaccines become useless if, for example, some new treatment emerges for a disease for which a new vaccine is under development? | Yes, but that remains true whether people have used SMART Vaccines or not. It cannot predict the emergence of disruptive technologies. It can readily re-estimate the priority scores in the presence of new information, and all rankings should be re-calculated when conditions surrounding any vaccine's potential use change. |
| How can you expect decision makers to deal with the complexity of this software program? | In general, the committee believes that high-level decision makers will not in fact have to deal with many facets of the software's complexity. More likely, specialized assistants to decision makers will create or import relevant data and possibly even carry out preliminary analyses using weights specified by the decision maker. The current version of SMART Vaccines provides entry points into the software at appropriate points for each possible type of user, ranging from technical data specialists to final decision makers. |
| How can I interpret what the scores from SMART Vaccines mean? | Each user's particular set of values and weights helps define the scale for the final priority score, so users cannot compare scores from one user to another unless they use the same attributes and endpoints. This is a standard feature of multi-attribute utility models.
Some users have found it useful to think about the priority scores in the same way that we think about reports of temperature. In a Fahrenheit scale the difference between 50°F and 70°F (20 degrees) has the same meaning as the difference between 20°F and 40°F. However, in the Fahrenheit scale 40°F is not twice as hot as 20°F. Similarly, on a Celsius scale the difference between 20°C and 30°C (10 degrees) has the same meaning as the difference between 10°C and 20°C, but 20°C is not twice as hot as 10°C. Furthermore, 20°C and 20°F do not have the same meaning. These differences do not make thermometers useless, but they do require an “anchor” to interpret them. With thermometers, we can use standard reference points to help understand what 20°F and 20°C mean. We know that water freezes at 0°C and boils at 100°C, and similarly that water freezes at 32°F and boils at 212°F. Knowing these two pairs of values allows us to make direct comparisons between Fahrenheit and Celsius values, and we can calculate that they have the same meaning at only one temperature—that is, minus 40°C has the same value as minus 40°F. |
| Who is expected to use SMART Vaccines and why? | Potential users of SMART Vaccines (individually or collaboratively) include decision makers in a wide range of constituencies: federal and private research groups, funders, vaccine manufacturers, purchasers of vaccines, regulators, and nongovernmental groups. SMART Vaccines offers a new framework that could help provide a new standard for decision making among various stakeholders in many circumstances such as decision making under opacity; prioritizing under constrained resources, complexities associated with globalization, economies, and health. Furthermore, changing realities need decision models to be refreshed, which is what this tool offers—a dynamic, living decision-support framework that can be updated as new data, diseases and potential vaccine candidates emerge. |
