INTRODUCTION

Research with biological select agents and toxins (BSAT) requires compliance with a number of policies and procedures designed to maintain the security of these materials. The issues related to personnel training, screening, and monitoring are discussed in Chapter 4. This chapter addresses other aspects of the program identified by the committee as important, along with the committee’s conclusions and recommendations.

The issues addressed here relate to stakeholder engagement in the Select Agent Program, the list of select agents and toxins, the need for accountability, security based on risk assessment, the role of evaluation in moving forward, improvement of the laboratory inspection system and inspector training, and mechanisms needed to fund security and compliance.

FACILITATING STAKEHOLDER INPUT BY FORMATION OF A BSAT ADVISORY COMMITTEE

One of the frequent themes that emerged from the public consultations held by the Executive Order (EO) Working Group on Strengthening the Biosecurity of the United States and the National Science Advisory Board for Biosecurity (NSABB) and in the committee’s own public sessions and site visits was the need for increased and more systematic communication among those agencies funding BSAT research, those agencies administering the Select Agent Program, and those entities conducting BSAT research. The Centers for Disease Control and Prevention (CDC) and the Animal and Plant Health Inspection Service (APHIS), through their training programs and outreach at professional meetings, are already providing information and guidance to the regulated community. The creation of the National Select Agent Registry as a single point of contact for agents regulated by both CDC and APHIS has been almost universally applauded for simplifying the regulatory environment and providing coordinated guidance. But because BSAT research is carried out and supported by several federal agencies, not just the Department of Health and Human Services (HHS) and the U.S. Department of Agriculture (USDA), the committee believes a more formal structure is needed to engage the community of stakeholders in the operation of the program.

To provide a locus for both feedback from the research community and discussion of issues of common interest, the committee recommends the establishment of a Biological Select Agents and Toxins Advisory Committee (BSATAC), whose membership would be drawn from the BSAT research community. This committee would provide an ongoing conduit for discussion of the implementation of the select agent regulations and would be made up of microbiologists and other infectious disease researchers (including select agent researchers) at various career stages, responsible officials, and those with experience in biosecurity, animal care and use, compliance, biosafety, operation of BSAT facilities as well as viewpoints from the public health, risk assessment, and legal communities. Representatives from the federal agencies with a responsibility for funding, conducting, or overseeing select agent research would serve in an ex officio capacity.¹ The committee would operate under the provisions of the Federal Advisory Committee Act, which mandates public meetings, adequate advance notice, and public accessibility of records—all of which would serve to broaden the reach of the BSATAC.

Such an advisory committee should have several specific responsibilities. Based on results of the consultations and site visits, it is clear that an important responsibility for the BSATAC would be the promulgation of guidance on the implementation of the Select Agent Program. A survey of the principal investigators (PIs) and co-PIs of the National Institutes of Health (NIH) Regional Centers of Excellence (RCEs) for Biodefense and Emerging Infectious Diseases found that more than 90 percent believed that select agents should be regulated, but also that a majority were concerned about whether or not they were in compliance with the rules and that they lacked a source to go to with their questions (Sutton 2009). In addition to providing formal guidance, the Advisory Committee would facilitate exchange of information about the program—such as aggregate data on laboratory-acquired infections—and promote the sharing of successful practices across institutions and sectors. Both of these actions would substantially assist in the implementation of the program. At present, they do not occur on a regular basis, to the detriment of efficiency and uniform compliance. To carry out these responsibilities, the Advisory Committee would convene regular public meetings of key constituency groups. All of these actions would contribute to another important potential Advisory Committee responsibility: promoting harmonization of regulatory policies and practices.

In addition to these responsibilities, the Advisory Committee should fulfill two other important functions. Below, we recommend that the Select Agent Program carry out regular evaluation of the relative security benefits it provides and of their consequences, both intended and unintended, that their implementation is having on BSAT research. We believe these evaluations are necessary, whether or not the recommended Advisory Committee is created, but it would be a natural body to provide oversight for the conduct of such evaluations and for determining how the results are used by the Select Agent Program.

We also believe the Advisory Committee should provide advice on the composition and/or stratification of the list of select agents and toxins, as discussed below. The BSATAC would not be a substitute for the rulemaking required by statute, but it could be a source of important input to decision-making and provide a mechanism for ongoing community engagement.

RECOMMENDATION 2: To provide continued engagement of stakeholders in oversight of the Select Agent Program, a Biological Select Agents and Toxins Advisory Committee (BSATAC) should be established. The members, who should be drawn from academic/research institutions and the private sector, should include microbiologists and other infectious disease researchers (including select agent researchers), directors of BSAT laboratories, and those with experience in biosecurity, animal care and use, compliance, biosafety, and operations. Representatives from the federal agencies with a responsibility for funding, conducting, or overseeing select agent research would serve in an ex officio capacity. Among the responsibilities of this advisory committee should be the following:

• Promulgate guidance on the implementation of the Select Agent Program;
• Facilitate exchange of information across institutions and sectors;
• Promote sharing of successful practices across institutions and sectors;
• Provide oversight for evaluation of the Select Agent Program;
• Provide advice on composition/stratification of the list of select agents and toxins;
• Convene regular meetings of key constituency groups; and
• Promote harmonization of regulatory policies and practices.

This committee is not recommending a particular site for the Advisory Committee within the federal government, although there appear to be two natural options. One choice would be to make BSATAC a joint enterprise for the Secretaries of HHS and USDA, since these two departments have the formal statutory authority for the Select Agent Program. The secretariat might then reside within one or the other agency, perhaps with the Select Agent Program office in CDC. This would have the advantage of locating the Advisory Committee with the program to which it is providing advice, but might overemphasize the role of the regulatory agencies with responsibility for the program and make it more difficult to consider the other federal agencies that are essential to effective functioning of the program.

Another option would be to make the committee a project of the Office of Science and Technology Policy (OSTP) within the Executive Office of the President, perhaps with a formal link to the National Security Council. In this case, the Director of OSTP would appoint the membership of the committee. This would give the Advisory Committee—and BSAT research more generally—greater visibility and allow OSTP to perform a coordinating function for science and technology policy that is one of its normal tasks.

Wherever the Advisory Committee is located, it will need a small secretariat and professional and administrative staff to support its work as well as dedicated funding. Creating the Advisory Committee without providing it with the necessary resources to carry out its responsibilities, or forcing its host agency to absorb the costs, would not be appropriate and thereby substantially reduce its chances of success.

A number of similar committees exist throughout the government to provide a mechanism for the formal engagement of key external stakeholders. For example, the Secretary’s Advisory Committee on Human Research Protections (SACHRP) provides expert advice and recommendations to the HHS Secretary and Assistant Secretary on issues related to human research subjects and reviews selected work and activities within HHS.² Similar to the proposal for BSATAC, SACHRP is composed of experts from outside the government along with nonvoting ex officio members from the relevant federal agencies.

STRATIFICATION OF THE LIST OF SELECT AGENTS AND TOXINS

The current list of select agents and toxins contains more than 80 entries.³ The list represents a diversity of pathogenic microorganisms and biological toxins with a range of potential for use as biothreat agents. This prompted the committee to ask if a single list of select agents and toxins, all of which are subject to the same security procedures, represents an optimal solution.

A very few items on this list have actually been used as a biothreat agent, and there is legitimate reason to be concerned about these items (see reviews by Carus 2001 and Wheelis et al. 2006). There is also reason to have security concerns about particularly deadly pathogens that may be genetically engineered to become more dangerous or resistant to treatment or those that can be effectively disseminated. But do all of the materials on the list prompt the same level of concern and risk? Does including a microorganism or toxin that poses a minimal security risk justify the potential lost capacity for research on that agent? Is a single undifferentiated list of select agents and toxins the safest and most effective way to ensure the security of the American public?

In addition to the biosafety classifications discussed in Chapter 2, several stratifications of the select agent list already exist (see Box 5-1). For example, CDC has defined Category A, B, and C bioterrorism agents, with Category A agents explicitly recognized as “organisms that pose a risk to national security.” The NIH Guidelines for Research Involving Recombinant DNA Molecules define four risk groups to catalogue biohazardous agents, including select agents. The Department of Homeland Security (DHS) conducts a bioterrorism risk assessment (BTRA) that encompasses a number of factors including the agents, routes of acquisition, methods of production and weaponization, targets, modes of dissemination, public health consequences and economic cost (Pesenti 2009).⁴ A Blue Ribbon Panel convened by OSTP prioritized a list of animal pathogens of the greatest concern (Kelly et al. 2004). In addition, other national priorities, such as the list of agents against which we develop countermeasures or stockpile vaccines provides some indication of the agents that prompt the most concern. Finally, a graded approach to security has been ongoing for some time in work with nuclear materials, so there is precedent for implementing different security procedures based on risk.⁵

BOX 5-1

Selected Existing Classifications of Biological Agents. Centers for Disease Control and Prevention (reproduced from <http://www.bt.cdc.gov/agent/agentlist-category.asp>) Category A: High-priority agents include organisms that pose a threat (more...)

The committee concluded that the present all-encompassing model for the list of select agents and toxins does not address appropriately the range of risks and vulnerabilities presented by these agents. Moreover, a list of more than 80 agents of varying risk dilutes attention from those that pose the greatest degree of concern, which may, in the process, render the nation less secure. It would be more effective to focus the highest scrutiny on those agents that are, indeed, of greatest concern and on those facilities with the equipment that enables weaponizing biological agents—and to offer a graded series of security procedures and policies for agents that pose less risk. For these reasons, the committee recommends a reconsideration of the purpose and composition of the list of select agents and toxins to reflect actual security concerns that merit inclusion on the list.

Although consideration of which specific agents and toxins should be on such a list is beyond the charge of the committee, the committee believes that stratification of the list of select agents and toxins is both warranted and necessary. Stratification should be consistent with the original purpose of creating the list, namely to catalogue those agents that pose a risk for use as a significant biothreat agent. If the purpose of the Select Agent Program were to protect the public from any infectious organism representing a threat to public health, then such a select agent list would be both unwieldy and highly disruptive to biomedical research—in fact, all infectious disease research would then be subject to inclusion on the list. If the purpose of the Select Agent Program is to secure and protect the public against microorganisms and toxins that might be used as biothreat agents where the consequences cannot be easily managed, then the list is too long and should not include microorganisms with little or no potential for use as a biothreat agent or those whose impact can be effectively managed in other ways.

A team composed of subject matter experts from within the federal government—the Intragovernmental Select Agents and Toxins Technical Advisory Committee (ISATTAC)—currently advises the Select Agent Program on proposals to add or delete agents or toxins from the list.⁶ At present there is no mechanism beyond the formal rulemaking process through Federal Register notices for the BSAT research community and the broader community of infectious disease researchers to provide suggestions, comments, or advice to ISATTAC or to the Select Agent Program.

We believe that it is important to develop mechanisms for adding or removing agents from the list without unwarranted delay, to ensure that the list remains reflective of legitimate concerns. A procedure is needed to assess the threat risk posed by a biological agent that would initiate a formal process to add it to the list—or, equally important, to determine that an earlier estimation of threat has diminished and an agent should be taken off the list. For example, development of an effective treatment or vaccine would diminish the value of a particular pathogen as a biothreat agent, thereby downgrading its risk profile. Critical in consideration of adding or removing an agent from the list is the inclusion of significant information and input from external stakeholders, beyond the usual formal commenting process to government officials.

RECOMMENDATION 3: The list of select agents and toxins should be stratified in risk groups according to the potential use of the material as a biothreat agent, with regulatory requirements and procedures calibrated against such stratification. Importantly, mechanisms for timely inclusion or removal of an agent or toxin from the list are necessary and should be developed.

As described in Recommendation 2 above, an external group of stakeholders—the Biological Select Agents and Toxins Advisory Committee—should be charged with advising on the composition of the list and facilitating wider engagement of the research and security communities. This Advisory Committee should also advise the Select Agent Program on the implications that stratification of the list of select agents and toxins has on implementation of personnel screening, physical security requirements, and other procedures.

It should be noted that addition of an agent or toxin to the list will have a significant impact on the conduct of research on that agent or toxin because the research will go from no special security requirements to the full complement of the select agent regulations. In effect, long-standing research programs will immediately be at risk if the institutions where the research is conducted are unable to take on the additional responsibilities for select agent research, if secure laboratory facilities cannot be obtained, or if researchers are unable to be cleared through the Security Risk Assessment (SRA) screening process. Even with a time for phasing in of implementation, how will the security enhancements be funded? What will happen if a researcher does not pass the SRA because of a past offense or other flag in the criminal, immigration, and terrorist database screen? What will be the impact of that exclusion on the researcher’s career and on the progress of science?

Similarly, removal of a select agent or toxin from the list will immediately decrease the security requirements and restrictions. What will happen with unnecessary and excessively secure laboratory capacity and an expensive security apparatus that may no longer be needed?

ACCOUNTING FOR MATERIALS

It is prudent and appropriate for entities with the responsibility for BSAT laboratories to know what types of select agents and toxins are present in their facilities. In addition to maintaining records of materials in a facility for security purposes, such listings serve an important safety function in detailing materials of concern for laboratory personnel, as well as for first responders in emergencies.

Several sections of the select agent regulations discuss inventory control and recordkeeping associated with select agents and toxins. The section on records contains the most detailed guidance and is included in Box 5-2. In addition, the Select Agent Program provides further guidance and examples of the type of record-keeping that may be expected:

BOX 5-2

Select Agent Regulations—Records Section (42 CFR 73.17). § 73.17 Records. An individual or entity required to register under this part must maintain complete records related to the activities covered by this part. Such records must include: (more...)

Inventory Control: Each entity is required to keep a current and up-to-date inventory. How that inventory is conducted and maintained must be documented in the entity’s security plan and must be consistent with the requirements found in Section 17 [see Box 5-2]. The select agents and toxins in the entity inventory must be labeled and identified in a way that leaves no question that what is in stock is accurately reflected in the inventory records…. All inventory records must be safeguarded to prevent alterations and be retained for 3 years…. (CDC/APHIS 2007)

Entities or individuals required to register under the Select Agent Program must develop and implement a written security plan that describes procedures for inventory control, the reporting of loss or theft of select agents and toxins, or the alteration of inventory records, among other elements. The security regulations also require that individuals with clearance to work with select agents are mandated to report the loss or theft of such agents or any sign that inventory or use records have been altered or otherwise compromised.

These regulations provide highly specific guidance with respect to information to be collected. While the committee believes that it is useful and important to know which agents are present and where they are located, we question the value of measuring the quantity for living microorganisms, except for the amounts when acquired by a facility or transferred out to another facility.

Unlike nuclear materials, biological organisms have the ability to replicate (see Box 5-3).⁷ Because a new culture can be prepared with as little as a single microorganism, an individual would need only a miniscule—and undetectable—amount from a single vial to establish a new culture and grow up large volumes of the agent in a matter of hours or a day. Therefore, determining that the number of vials is the same from one moment to another provides no guarantee that agents have not been removed from the laboratory since the original number of vials or tubes could remain the same while the agent itself has been removed. Also unlike nuclear materials, it is possible to completely inactivate BSAT materials: microorganisms can be autoclaved and toxins denatured so that they no longer pose a risk. As convenient as it might be to count vials, volumes, or number of organisms, it is not a biologically relevant means of inventory.

BOX 5-3

Comparing Nuclear and Biological Materials. Nuclear Materials Do not exist in nature in readily concentrated form appropriate for weapons

For these reasons, the committee concluded that undue reliance on accounting practices, including counting vials, leads to false security and is counter-productive.

So if counting vials or quantities is not a viable strategy for inventorying materials, what is? The committee strongly endorses a focus on accountability: specifically, what BSAT material is present, when it was obtained, where it is located, who has access, and when. It is also prudent that accountability not be limited to archived stocks but be extended to working materials as well, whether they are present in vials, Petri dishes, or laboratory animals. Accounting for the containers in which the cultures are maintained is neither feasible nor worthwhile. Accounting for access to select agents and toxins would be far more reliable and practical.

RECOMMENDATION 4: Because biological agents have an ability to replicate, accountability is best achieved by controlling access to archived stocks and working materials. Requirements for counting the number of vials or other such measures of the quantity of biological select agents (other than when an agent is transported from one laboratory site to another) should not be employed because they are both unreliable and counter-productive, yielding a false sense of security. A registered entity should record the identity of all biological select agents and toxins within that entity, where such agents are stored, who has access and when that access is available, and the intended use(s) of the materials.

It should be noted that Recommendation 4 makes a distinction between select agents—which have the capacity to replicate—and toxins—which do not. This recommendation, therefore, does not change the requirement to keep records on the amount of a toxin but does recommend that inventories for both select agents and toxins should include information about who has access to these materials, when, and for what intended purpose.

SECURITY BASED ON RISK ASSESSMENT

Physical security is required of all facilities registered with the Select Agent Program.¹⁰ Each facility must develop and implement a written security plan, which is reviewed by either CDC or APHIS as part of the initial and ongoing facility registration process. Because each facility is different in design, different physical security methods are required to address site-specific security requirements. Determination of which physical security measures to include in a site-specific plan is made based on “a site-specific risk assessment and must provided graded protection in accordance with the risk of the select agent or toxin, given its intended use” (HHS 2005:13306). In more detail, the security plan must:

1. Describe procedures for physical security, inventory control, and information systems control;
2. Contain provisions for the control of access to select agents and toxins;
3. Contain provisions for routine cleaning, maintenance, and repairs;
4. Establish procedures for removing unauthorized or suspicious persons;
5. Describe procedures for addressing loss or compromise of keys, passwords, combinations, etc. and protocols for changing access numbers or locks following staff changes;
6. Contain procedures for reporting unauthorized or suspicious persons or activities, loss or theft of select agents or toxins, release of select agents or toxins, or alteration of inventory records; and
7. Contain provisions for ensuring that all individuals with access approval from the HHS Secretary or APHIS Administrator understand and comply with the security procedures.

These regulations provide overall guidelines for the content of site-specific security plans; however, they are sufficiently broad to allow for variation in their implementation. While this variation has benefits, it also creates inconsistencies and confusion as facility operators, contractors, and inspectors attempt to determine whether specific security measures at individual facilities sufficiently adhere to these guidelines.

Variations in existing governing regulations create additional challenges for those designing and implementing site-specific, risk-based security plans. The long list of regulations applied to physical security includes the following:

• ARS 242.1-ARS Facility Design Standards (from the Agricultural Research Service)
• APHIS Security Design Standards
• Biological Safety in Microbiological and Biomedical Laboratories (CDC/NIH 2007)
• United Facilities Criteria (UFC) July 31, 2002
• UL 972 (12) Burglary Resistant Glazing Material-Standard Test Procedures-Smash and Grab Resistance
• UL 572 Standard Test Procedures for Bullet-Resistant Glazing
• ASTM 1233 (11) Burglary Resistant Glazing Material-Standard Test Procedures
• ASTM F 588 Resistance of Window Assemblies to Forced Entries
• ASTM F 476 Security of Swinging Door Assemblies
• ASTM F 842 Measurement of Forced Entry of Horizontal Sliding Door Assemblies
• ASTM F 1642 (21) Standard Test Method for Glazing and Glazing Systems Subject to Airblast Loadings
• NIH Draft Physical Security Design Guidelines for NIAID NBLs and RBLs, December 19, 2003
• Interagency Security Committee Security Design Criteria, for New Federal Office Buildings and Major Modernization Projects, May 28, 2001
• NIH Design Policy and Guidelines (spring 2003)
• Uniform Federal Accessibility Standards (UFAS) and/or the Americans with Disabilities Act (ADA)
• International Building Code, latest edition
• National Fire and Life Safety Codes, latest editions
• HHS (NIOSH) Publication No. 2002-139, Guidance for Protecting Building Environments from Airborne Chemical, Biological, or Radiological Attacks
• NIH Security Device Application Guideline #22230, latest edition
• Laboratory Security and Emergency Response Guidance for Laboratories Working with Select Agents (CDC 2002)
• U.S. Department of Energy, A Manual for the Prediction of Blast and Fragment Loadings on Structures
• 10 CFR 20.1801 and the Nuclear Regulatory Commission regulations on security of radioactive materials
• 42 CFR 73, including pathogens and toxins regulated by both HHS and USDA and non-overlap select agents of HHS
• Memorandum from the Secretary of the Department of Health and Human Services dated March 6, 2002, with 12 “Requirements for Securing Select Agents,” Attachment 5
• AR-XX Military Police-Biological Agent Security Program
• UFC 4-010-01 DOD Anti-terrorism Standards

These varied regulations and guidelines lead to inconsistencies in application for a variety of reasons, reflective of the fact that facilities and regulations differ. On the facility side, Threat Risk Assessments, which are generally conducted by contractors, produce results that are unique to the facility and are influenced by the specific contractor. Therefore, the contractors recommend different solutions for similar situations. New and old facilities have different operational and physical infrastructures. Regulations themselves also vary from those that are prescriptive (e.g., doors must use a certain kind of lock) to those that have performance-based requirements (e.g., doors must be locked). They lack consistent fundamentals, and no detailed requirements for operational security needs are provided. This leaves facility owners and operators to decide which approaches are most appropriate, considering the likelihood that various inspectors might reach different conclusions or have different interpretations of the regulations.¹¹ Given these inconsistencies, facility designers and operators often err on the side of caution and implement physical security approaches that far exceed those warranted in a particular situation and are, therefore, inefficient. Addressing these inconsistencies and the problems they create would be beneficial from a security and cost-benefit perspective.

In order to address these inconsistencies, a minimum set of physical/technical standards should be established that interfaces with operational practices applicable to all relevant agencies. One means of doing this may be through a tiered approach to security based on mission. Further, a review process that verifies physical/technical security requirements would be helpful, particularly if combined with the establishment of an incident database to validate performance of established principles. Finally, updating the Threat Risk Assessments periodically (perhaps every 2–3 years and ideally in line with the inspection cycle to allow time for appropriate adjustments) would allow facility operators to adjust security measures accordingly, ensuring that security measures are continually implemented based on the most recent assessment of risk. This would allow for more cost-effective and consistent compliance with security needs and regulations.¹²

In conclusion, currently little guidance is provided by the Select Agent Program as to the definition and interpretation of minimum standards for physical security, leading to a significant difference in the design and associated cost of BSAT facilities. This often presents subsequent inspection and compliance challenges and does not necessarily ensure greater security.

RECOMMENDATION 6: The Select Agent Program should define minimum cross-agency physical security requirements, which recognize that facilities have unique risk-based security needs and associated design components, to assist facilities in meeting their regulatory obligations.

This recommendation is not intended to move away from the performance-based standards promulgated by the Select Agent Program but to provide additional guidance to the community on minimal standards. Moreover, any stratification of the select agent list, as recommended above, may have implications for stratification of the physical security requirements as well. The Select Agent Program can further assist institutions in interpreting physical security requirements by establishing a hotline or other mechanism for rapid response in answering questions about interpretation of the standards.

EVALUATION

The committee believes that it is both appropriate and necessary to apply rigorous analytical methods to assess the mix of policies that promote both high-quality science and appropriate security. It is exceptionally difficult, however, to assess whether the policies are, in fact, achieving this optimal mix. Designing policies to prevent future terrorist activities and, later, understanding how and whether a program or programs prevented those activities and behaviors presents a particular evaluation challenge. After all, if the policies are successful, nothing bad will happen. Does this mean that the policies “worked,” that is, that they prevented terrorists from gaining access to BSAT materials? Or does it mean that the threat of bioterrorism has been exaggerated, the policies were not needed, and funds were needlessly redirected from research? Or does it mean that we are worrying about the wrong aspects of threat?¹³ Following from the difficulty in assessing the effectiveness of programs that will be successful if there is no obvious effect—other than the absence of another action—it is likewise difficult to assess whether the various costs associated with the program are appropriate.

Independent evaluation can provide useful information on how the Select Agent Program is implemented and identify important intended or unintended consequences of the program upon the research enterprise. Fortunately, there are several existing tools and techniques that could be applied in evaluating the program. The committee believes that any new policies intended to improve physical security and personnel reliability should be carefully evaluated, along with operation of the program overall. Relying on “dueling anecdotes” in not acceptable for establishing policy. We note that the National Research Council (NRC) report Biotechnology Research in an Age of Terrorism suggested a similar process for addressing another aspect of the efforts to reduce the risk of bioterrorism, that is the potential misuse of the knowledge, tools, and techniques that would result from BSAT (and other life sciences) research:

The substantial expansion of funding for research in biodefense now in progress and anticipated suggests that it will be vital to assess how these new resources affect the conduct of research and to be ready to make timely adjustments. The monitoring should be done with the goal of suggesting ways to improve the [committee’s proposed] system’s operation and efficiency. But it should also include the possibility of proposing that parts of the system be overhauled or even eliminated if they prove ineffective or an impediment to important scientific research. (NRC 2004a:120)

The committee emphasizes that formal evaluation of the Select Agent Program is more than accumulation of metrics and demographic data. It is difficult to obtain quantitative measures of the impact of complex policies on difficult problems. Furthermore, only a portion of the information needed to gain a detailed understanding of the Select Agent Program will be quantifiable. While it is not within the scope of this committee to elaborate in detail or to design a specific program evaluation plan, several practices widely employed by governments at all levels, by the international development community, and by various types of private organizations provide a constructive framework applicable to the Select Agent Program. See, for example, discussion of the range of program evaluations methods in Wholey et al. (2004) and World Bank (2004).

Among key aspects of program evaluation, regardless of the specific evaluation methods selected, are identification of program goals and objectives, recognition of the need for both quantitative and qualitative data, and development of a specific evaluation design. Given that the Select Agent Program has evolved since its inception, data (when and where they exist) may be imperfect and not necessarily suited to addressing questions about the goals and objectives of the program. However, there are specific measures that can be taken to enhance the ability to understand the impact of the Select Agent Program on the promotion of high-quality science and maintenance of appropriate security.

As with many similar programs, the Select Agent Program has multiple goals and objectives. Therefore, it follows that evaluation of the program may be designed to understand its effectiveness and the impact of one or more of these goals and not others; or an aggregate analysis can be designed to understand the impact of the program as a whole. The first step in designing program evaluation is identifying which specific question or questions are to be addressed. Evaluation questions relevant to the BSAT research may include: What is the relative impact/effectiveness of various and/or combined physical security measures? What is the relative impact/effectiveness of the SRA process as a whole and in part? What are the costs and benefits of these measures and processes? What impact does the Select Agent Program have on the ability of laboratories to recruit new researchers to perform research on select agents and toxins and to retain expertise within the select agent community? How effective are the relevant management practices employed at BSAT laboratories? Answers to each of these questions may lead to different evaluation designs, but posing the questions to be addressed is a necessary first step.

Any program evaluation, regardless of the specific question(s) of interest, will require analysis of information, usually a combination of both quantitative and qualitative data. Since the inception of the Select Agent Program, the collection and retention of important data have increased, including gathering data on applicants for SRA clearance, SRA processing times, and the number of facilities registered as BSAT laboratories. Despite this progress, relatively little data have been collected on the Select Agent Program as a whole. It is still difficult to know with any degree of certainty how much the different aspects of the program cost or how many researchers may or may not choose to work with select agents and toxins because of concerns about the SRA process. Going forward, more and varied types of data will be required to understand—either in part or in full—the impact of the Select Agent Program on the research community, on the type and quality of research undertaken, and ultimately on the safety and security of those working with select agents and toxins and of the surrounding community. A baseline survey may be an effective means of gathering initial data that would establish a more clear understanding of key aspects of the Select Agent Program and inform future evaluation efforts.

There are numerous means by which program evaluation experts can and do collect relevant data. Specific decisions regarding data collection are made based on cost, available personnel resources, ease and/or difficulty of data collection, and specific demands of the chosen program evaluation design. However, certain methodologies are common to many designs, such as surveys, interviews, and the addition of new questions to existing forms and/or reporting mechanisms. Most program evaluations employ a combination of these techniques to acquire the relevant and necessary data. In the BSAT context, program evaluation will require all segments of the research and regulatory community to participate in data collection, as there is no central location that has access to all of the relevant perspectives. Universities are in a position to collect data on the research and career decisions of graduate students, the regulatory and implementing agencies are in a position to collect data on certain costs and processing times, private biosecurity companies and facilities are in a position to collect data on physical security costs, and program evaluation experts are in a position to bring their experience with evaluating complex systems to bear on the task of collecting qualitative data. Moving beyond anecdotal information to data collection and analysis will require coordinated effort and participation.

Some of the most important studies to undertake may also be among the most difficult. For example, what can we learn about the research that is not conducted as a result of some aspect of the Select Agent Program? What are the missed opportunities and discoveries that are delayed or not achieved because of disincentives to select agent research? How do the financial, personnel, and security limitations impact which experiments are conducted and by whom? While efforts to evaluate the BSAT program, particularly in the initial stages, will present challenges to evaluation experts and many in the BSAT research community, there is a broad and growing body of expertise upon which to draw. Significant progress has been made in recent years and decades to improve evaluation and the results it can offer to stakeholders. A broad array of evaluation designs and techniques now exists to provide decisionmakers with the information and analyses necessary to make appropriate policy and legislative decisions that can improve the effectiveness of programs. Given the evolution of the Select Agent Program, and the genuine concerns of lawmakers, scientists, and citizens alike, it would be beneficial to undertake a series of evaluation studies to address those questions deemed relevant and appropriate.

RECOMMENDATION 7: Independent evaluation of the Select Agent Program should be undertaken to assess the relative benefits for achieving security, to consider the consequences of the program (intended and unintended) on the research enterprise, and to provide useful data about the Select Agent Program. Such evaluation, which may be coordinated through the BSAT Advisory Committee, should be provided with dedicated funding.

The committee believes that it would be effective for evaluation to be coordinated through the BSATAC, since it will provide an important advisory role across the entire Select Agent Program, including the connections between regulatory agencies, funding agencies, and entities performing BSAT research. In addition, if BSATAC is housed within OSTP, the results of evaluation would be directly available to those within and beyond the Select Agent Program who are able most effectively to restructure programs and implement interagency coordination.

TRAINING OF INSPECTORS

All select agent laboratories undergo regular inspections by CDC or APHIS, whether academic, commercial, or government and whether for research or public health. Routine inspections occur every three years with additional inspections undertaken when there is a significant change in an entity’s registration or in response to concerns (see Chapter 2 for more information). These inspections involve both extensive review of records and a multi-day visit to the facility during which inspectors verify the accuracy of records, inspect the security and safety components of the facility, and interview personnel.

In addition to the inspections by agencies with statutory responsibility for the Select Agent Program, many funding agencies—including DHS and the Department of Defense—conduct their own inspections on research and facilities they support. Other federal agencies also have responsibility for overseeing aspects of the facility and may conduct inspections. Finally, some state and local authorities inspect facilities within their jurisdiction.

Close coordination between CDC and APHIS in the Select Agent Program has served the research community well and should be expanded to include other government agencies with an involvement in BSAT research. Specifically, the committee encourages coordination and consolidation so that entities with select agent research sponsored and/or regulated by different federal agencies are not subject to very different and possibly conflicting guidance and regulations or to duplicative inspections. The committee is hopeful that the conversation begun through the interagency EO Working Group may have tangible benefits in the coordination of regulations and practices among the various U.S. government agencies that have some responsibility for BSAT research.

The current statutory authority of agencies with responsibility for the Select Agent Program will likely make it difficult to achieve a perfectly streamlined system of oversight and inspection. For example, the committee recognizes that it will be difficult for one agency to defer completely to the inspection conducted by a different agency because of Congressionally mandated responsibilities and different areas of focus. However, there are steps that can be taken to streamline different inspections, such as by having them coordinated in time and operating from a common set of fundamental principles and policies. In addition, it is critical to ensure that the requirements of multiple agencies are not contradictory. At the present time, the benefit of inspections is compromised by confusion caused by overlapping and sometimes conflicting guidance, as well as inconclusive findings from a variety of agencies and bodies.

The multiplicity of oversight and regulatory functions leads to significant expenditure of time and resources by the select agent entity in its preparing for, participating in, and responding to the inspection. The intent is that inspections provide an opportunity to consider important aspects of security and compliance issues. In fact, the committee heard from some members of the community that inspections conducted by well-trained and experienced inspectors were quite helpful and educational.

Nevertheless, complaints about the nature of some inspections have arisen (e.g., Box 5-4). Members of the community have cited the increasingly bureaucratic nature of some inspections, with expanding focus on the technical letter of the regulation without regard to the spirit of the regulation and its intended objective. For example, one entity cited an example of an inspector who was unwilling to accept an “Emergency Response Plan” when an “Incident Response Plan” was listed in the inspection checklist; simply changing the title on the document satisfied the inspector.

BOX 5-4

The Challenge of Compliance. The Southwest Foundation for Biomedical Research has been working with select agents since 1996 and housed the only operational academic BSL-4 laboratory from 2000 until the opening of the laboratory at the University of Texas (more...)

Concerns have been raised that inspectors have not had the technical knowledge needed to understand the specific nature of various risks and have been reluctant to seek guidance from knowledgeable colleagues at CDC or APHIS. While the Select Agent Program is structured to provide several levels of support when necessary, it appears that not all inspectors make use of these opportunities, resulting in inspection reports that are not fully informed by the scientific issues impacting security and safety of the laboratory.

Much of the concern may stem from the fact that some inspectors are not sufficiently familiar with the nature of BSAT research. Although the Select Agent Program seeks to hire inspectors with scientific experience, including work in select agent laboratories, there are others hired who come from a biosafety or regulatory background but without an understanding of the select agent laboratory environment. These challenges are even more severe for those government agencies that do not focus on select agents, such as the community of state and local health officials who have rarely encountered a select agent facility but may have a responsibility for inspecting them.

Because of the critical role that inspections play in monitoring the safety and security of select agent facilities and identifying areas for improvement, it is necessary that those individuals who conduct the inspections have the proper background, training and support. Currently, not all inspectors have this preparation. It is also important that inspectors from the many agencies that oversee select agent research or facilities receive similar training to facilitate harmonization in the application of select agent regulations and inspection guidelines.

RECOMMENDATION 8: Inspectors of select agent laboratories should have scientific and laboratory knowledge and experience, as well as appropriate training in conducting inspections specific to BSAT research. Inspector training and practice should be harmonized across federal, state, local, and other agencies.

Moreover, appropriately trained and experienced inspectors should be supported by their agencies in exercising informed discretion. Inspectors should be encouraged to utilize their experience and training to apply consistent regulations to variable environments and to solicit input from others where appropriate.

Although the committee does not make a specific recommendation in this regard, there may be a role for a certification process that will identify those inspectors who have the appropriate training and experience for inspecting select agent laboratories.

FUNDING SECURITY AND COMPLIANCE COSTS

Security and compliance procedures called for under the Select Agent Program can be significant, with costs substantially higher than for similar laboratory facilities. Security guards, cameras, access card readers, biometric identification technologies, alarms, lockable freezers and incubators, and other security measures all add to the cost of operating a select agent laboratory. Specialized equipment installed and maintained for biosafety purposes, such as air filtration systems, biosafety cabinets, decontamination showers, changing rooms, and the like have high initial and maintenance costs.

Construction of secure laboratories where select agent research will be conducted is often funded by grants specific for that purpose. For example, the National Institute of Allergy and Infectious Diseases (NIAID) is currently in the process of building a series of national and regional biocontainment laboratories (NBLs and RBLs). With tens to hundreds of millions of dollars invested in each of the two NBLs and 13 RBLs operational or under construction, as well as a network of 10 RCEs for Biodefense and Emerging Infectious Diseases, NIAID has made a substantial contribution to the nation’s biosecurity infrastructure.

But it is not only construction costs that make select agent facilities expensive. They have significant ongoing security and safety sustainment costs that far exceed the indirect costs that grantee institutions receive to cover the cost of facilities, maintenance, and operations. Although the committee did not have an opportunity to collect detailed information on security and compliance costs (see Box 5-5 for an example), comments on implementation of the select agent rules referenced in the Federal Register provide some indication. For example, annual operations and maintenance costs that were cited range from $100,000 to $700,000, with startup costs in the range of $1–4 million (HHS 2005). One university reported $300,000 in security improvements for electronic card access, alarm systems, and security cameras—not including additional recordkeeping and personnel requirements. Another institution cited a figure of $400,000 for a single BSL-3 laboratory. This is in line with reported cost of $130,000 to increase security at Louisiana State University and $150,000 to inventory and secure pathogens at Northern Arizona University (Wilkie 2004). The final Regulatory Impact Analysis to inform the select agent regulations, released in 2005, cited an annualized cost of the select agent regulations of $16 million, with annualized costs per facility of $15,300-$170,000 (HHS 2005). USDA also evaluated the cost of compliance. At the Ames, Iowa, National Veterinary Services Laboratory operated by USDA, for example, the cost of security upgrades implemented in 2002 was over $550,000 (USDA 2005). Although the needs and expenses will depend upon the current status of the facility and the site-specific risk assessment—which itself can cost $20,000 or more to conduct—the expenses can be a significant burden.

BOX 5-5

The Challenge of Funding the Operation of a Select Agent Laboratory. One of the committee’s site visits was to the National Center for Biodefense and Infectious Diseases at the Prince William Campus of George Mason University (GMU), which is currently (more...)

In addition to the financial cost of security requirements, personnel affiliated with select agent research facilities spend significant amounts of time ensuring compliance. This includes both additional time dedicated to screening, recordkeeping, and reporting as well as time spent preparing for, participating in, and responding to the required inspections. This time will be even greater for facilities that conduct research overseen by more than one agency. A Stimson Center survey found that researchers reported spending approximately four hours per week on select agent compliance issues, more than the time spent on regulatory demands for other issues (Fischer 2006).

Regardless of the specific resources dedicated to compliance and reporting, there is little dispute that the amount is significant: tens of thousands of dollars per year at even the smallest facilities. To be sure, the costs of security compliance pale in comparison to the astronomical costs of an incident and the secondary effects on research if security were to be compromised (Ekboir 1999; HHS 2005), but these operational and maintenance costs are real and need to be funded. How will these additional security and compliance bills be paid on an ongoing basis? If the institution will be relying, in part, on indirect costs from research conducted at the facility, what will happen if those research grants are not available?

Some select agent facilities provide all of the necessary support for research, personnel, and materials through infrastructure grants. For example, the New England RCE at Harvard Medical School, which the committee visited, is operating almost entirely on the RCE grant from NIAID. The center does not charge investigators for work conducted in its BSL-3 lab, including costs for the facility and its staff. Other laboratories operate as a fee for service in that investigators’ grants and other parties are charged for the direct costs of the research including laboratory staffing, perhaps even including a select agent surcharge. Other facilities may be limited to internally supported research with all of the security and compliance costs dependent upon indirect cost recovery and additional institutional support.

While the challenges of sustainable funding for scientific research go far beyond select agent research and this report, the implications are more troubling in the case of select agent research. It is not acceptable, either for the institution or for safety and security, to diminish appropriate and necessary risk-based security procedures and resources, regardless of the availability of funding for the facility. Host institutions, having to provide the difference, may choose to reduce their cost by understaffing the facility, hiring external contractors where a third party takes responsibility for key functions, or diverting funds planned to support scientific research to pay for security responsibilities. These are not sustainable solutions and raise risks.

The committee, therefore, urges federal agencies that fund BSAT research to establish dedicated funding for ongoing security and compliance responsibilities associated with this type of research. This is an essential obligation, and no facility should operate without appropriate security measures in place. Funds to support security and compliance should be from a separate source to avoid diminishing the already limited support for research and should be available on a continuing and competitive basis for the life of the facility.

RECOMMENDATION 9: Because of considerable security and compliance costs associated with research on biological select agents and toxins, federal agencies funding BSAT research should establish a separate category of funding to ensure sustained support for facilities where such research is conducted.

It is expected that these costs will be site specific and subject to change as security standards, risks, and successful practices evolve. The funding arrangements should also include a mechanism for supporting facility upgrades and implementing evolving standards and practices. In all cases, it will be important that these costs not be short-changed. In addition, the specific mechanism for providing such support will depend upon the nature of the laboratory and the funding source; for example, it may make sense to provide one mechanism for supporting the continued operation of federally funded facilities such as NBLs, RBLs, and RCEs and a different mechanism for investigator-initiated research grants.

Although this type of funding structure may be unusual for biomedical research laboratories, it is not uncommon for funding those areas of science where central infrastructure plays an important role. Primate research centers, telescopes, and the academic research fleet all have funding models in which operating costs are broken out as a separate direct expense, often from a separate account so that operations do not compete directly with science funding. The U.S. Academic Research Fleet, for example, divides the total operating expenses by the number of days the ship is at sea and charges this rate through ship operating proposals submitting to granting agencies (see Box 5-6). In this way, the granting agency pays for operating expenses directly and subject to the needs of the research projects but without relying upon research grants.

BOX 5-6

Funding Model for the Academic Research Fleet. The U.S. Academic Research Fleet provides an interesting model by which a federal agency takes responsibility for not only helping support the initial construction of a key element of research infrastructure, (more...)

Footnotes

1

Among the agencies it would be appropriate to have represented are the Office of Science and Technology Policy; the National Security Council/Homeland Security Council; the Office of Management and Budget; the Department of Health and Human Services; the Department of Agriculture; the Department of Justice; the Department of Homeland Security; the Department of Defense; the Department of Energy; the Department of Transportation; the Department of Commerce; the Department of State; the Environmental Protection Agency; the National Science Foundation; and the Office of the Director of National Intelligence.

2

See <http://www​.hhs.gov/ohrp/sachrp/> for more information about SACHRP.

3

In addition, there are pending proposals from the Department of Health and Human Services to add the SARS-associated coronavirus and Chapare virus to the list (HHS 2009ab).

4

See NRC (2008a) for a recent review of the BTRA.

5

Box 5-3 provides additional comparisons between nuclear and biological materials, including the security implications.

6

The members of the ISATTAC are federal government employees from CDC, APHIS, NIH, the Food and Drug Administration, USDA/Agricultural Research Service (ARS), USDA/CVB (Center for Veterinary Biologics), and the Department of Defense (DOD). The group’s three functions are to: (1) review requests for the exclusion of attenuated strains, (2) review requests to conduct restricted experiments, and (3) review requests for addition or deletion of agents or toxins to the list of select agents and toxins (Select Agent Program website: <http://www​.selectagents​.gov/FAQ_General.html#sec3q1>).

7

Toxins do not generally present the same difficulty as they cannot, themselves, replicate.

8

The concern expressed by the BSL-4 laboratory directors in this regard was that:

“Typically, the daily flow of work in a biocontainment laboratory involves ≥2 persons working near each other. Although the primary role of each person is to perform his or her work and not to monitor their coworker, there would still be ample opportunity to render aid if needed or to observe untoward activities. The risk issues of the 2-person rule arise when the normal activity in the laboratory is insufficient to satisfy the rule in every active area of the BSL-4 laboratory. These issues will occur frequently outside normal working hours, i.e., evenings, weekends, or holidays, and also during regular working shifts as project-specific tasks are completed asynchronously. The key implementation issue for a 2-person rule is when staff members are required to enter or extend time in containment solely to fulfill the security requirement. This personnel-centric approach has some serious shortcomings and in many circumstances may increase the safety risk for laboratory personnel. Effectively, the presence of an observer for the sole purpose of achieving a 2-person requirement would contribute to perceived time pressures, stress, distractions, and interruptions, all of which are factors identified by human performance management as error precursors” (LeDuc et al. 2009).

9

“No lone individual shall have access to a nuclear weapon. During any operation that may require access to nuclear weapons, there is a minimum of two authorized persons, each capable of detecting incorrect or unauthorized procedures with respect to the task to be performed and familiar with applicable safety and security requirements. Two authorized personnel are physically positioned where they can detect incorrect or unauthorized procedures with respect to the task or operation being performed. When application of the two-person policy is required, it is enforced by the persons who constitute the team during the entire period they are accomplishing the task or operation assigned and until they leave the area within which the two-person policy is required” (Blaisdell 2001).

10

Department of Health and Human Services, 42 CFR 72 and 73; 42 CFR 1003 provide the final rule on the Possession, Use and Transfer of Select Agents and Toxins. The specific section addressing physical security is 42 CFR 73.11 (HHS 2005).

11

See below for a discussion of inspections and the need for inspector training to promote harmonization.

12

Recommendation 9 suggests a mechanism for funding security upgrades.

13

For further discussion of these issues from different perspectives, see Danzig (2003), Altman et al. (2005), Leitenberg (2005), and Ostfield (2009).