STANDARDS FOR STANDARDS

At a minimum, standards should address an organization's level of performance in specific areas and, some would argue, not just what the organization is capable of doing but what it actually does (JCAHO, 2000). In theory, standards should set forth maximum achievable performance expectations for activities that affect the protection of human research participants. Perhaps most importantly, they should be based on widely accepted ethical principles that form the norms for research behavior.

In the United States, the principles embodied in The Belmont Report have served as the foundation for the ethical requirements in human research (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979). The three basic ethical principles in The Belmont Report are (1) respect for persons, (2) beneficence, and (3) justice. The first principle, respect for persons, encompasses two ethical concepts: first, “individuals should be treated as autonomous agents” and their decisions respected; and second, “persons with diminished autonomy are entitled to protection” (p. 4). The second principle, beneficence, incorporates the rules of “do no harm” and “maximize possible benefits and minimize possible harms” (p. 4). The third principle, justice, refers to a fair and equitable distribution of benefits and burdens, fair selection of participants, assurance that participants receive what is deserved or due, and ascertainment that equals are treated equally (p. 5). In the United States, these principles strongly influenced the development of federal regulations—in particular, regulations governing research sponsored by the federal government or regulated by the FDA—via the Federal Policy for the Protection of Human Subjects (45 CFR 46, subpart A, also known as the “Common Rule”) or parallel FDA regulations (21 CFR 50, 56; international studies of devices are covered by 21 CFR 312.120).

The ethical principles found in The Belmont Report are also found in many international documents, including the Declaration of Helsinki and guidelines promulgated by the Council for International Organizations of Medical Sciences, a source on the ethics of international research involving human subjects (CIOMS, 1993; World Medical Association, 2000).

The ethical principles should be accompanied by procedural requirements, which then form the basis of the standards. Thus, standards should have an explicit rationale that is consistent with the goal of protecting individuals or populations that participate in research. The committee's “standards for standards” are contained in two recommendations.

Recommendation 3: Articulate Sound Goals Within Acreditation Standards

The goals of accreditation standards should be to ensure

1. that the proposed research promises to contribute knowledge sufficient to justify research involving human participants;
2. independent review of research by a board knowledgeable about protection standards and the fields of research being reviewed;
3. that the perspectives of participants are represented on institutional review boards (IRBs), on research monitoring bodies, and throughout the research oversight system;
4. that IRB members do not review protocols with which they have financial or nonfinancial conflicts of interest;¹
5. that investigator and institutional conflicts of interest, both financial and nonfinancial, are disclosed to IRBs and participants and are managed responsibly by research institutions;
6. a review process that balances risks and potential benefits, keeps risks to the minimum necessary, and monitors research on a continuing basis;
7. that an effective process for obtaining voluntary informed consent of participants is in place;
8. that policies and procedures to assess the quality of HRPPP operations, enhance accountability, and improve performance are in place;
9. there is fairness in the recruitment and selection of participants;
10. that the privacy and confidentiality of research participants are protected; and
11. that the HRPPP is transparent so that participants can judge the research process to be trustworthy.

Recommendation 4: Establish Flexible, Ethics-Based, and Meaningful Standards

Accreditation standards should meet the following minimal criteria:

1. They should be based on sound and widely accepted ethical principles.²
2. They should be flexible and adapted to different kinds of research and different research institutions.
3. They should encourage accredited organizations to shift from a culture that relies on external compliance checks to a culture that puts safety and voluntary participation foremost.
4. They should facilitate compliance with federal regulations but should aim to move an organization toward having stronger protection of human research participants.
5. To the extent possible, they should focus on the use of meaningful measures of how well the rights and interests of research participants are being protected rather than simple determination of whether informed-consent statements have been signed or IRB meetings were duly constituted.

Measurement of an organization's compliance with the procedural requirements set forth by standards serves as a proxy for ascertainment of the organization's level of compliance with the ethical principles that underlie the standards.

In its early discussions, the committee noted that beyond the primary aspiration of protecting those who participate in research, institutions seeking accreditation will be motivated by other aims as well, for example, enhancing the qualities and reputations of their research programs (and, as a result, potentially improving their financial status or prestige), attracting faculty and students to their graduate research training programs, and facilitating the recruitment of individuals as research participants. A successful system of accreditation must offer incentives for participation, such as enhancing the likelihood that a program in compliance with the standards will attract these resources. In addition, a successful accreditation system must have realistic and enforceable mechanisms by which to deter noncompliance with the standards (e.g., suspension from the program or loss of accreditation).

DEVELOPING MEASURES TO ACCOMPANY STANDARDS

Standards must be developed with consideration of the measures that will be used to evaluate an organization's level of compliance. The processes of developing standards and designing a set of tools that can be used to measure compliance (i.e., accreditation) cannot generally be uncoupled. The measures must address areas in which performance is likely to have a significant impact on the protection of human research populations. In addition, they must be precisely defined and specified, that is, standardized with explicit predefined requirements for data collection and for calculation of the value of the measure or the score for the measure. Furthermore, for the purpose of accreditation, there must be documentation for the measure that includes defined data elements, corresponding data sources, and allowable values. Such measures must be reliable—that is, the measurement should be able to identify consistently the events that it was designed to identify across multiple HRPPPs over time—and they must be valid, that is, they must capture what they were intended to measure.

The tools used to measure compliance with standards should be easily interpreted by those who use the resulting data, including accreditors, research participants, and those conducting or overseeing the research. Finally, determinations of the levels of compliance with the standards must be based on data. HRPPPs seeking accreditation will be required to provide evidence of compliance. This evidence must be supported by a reasonable data collection effort. In the development of standards, accreditation bodies must be mindful of the availability and accessibility of the required data elements and the effort and cost of abstracting and collecting data.

In general, standards should help HRPPPs and accreditation bodies identify exemplary performance and best practices, thus serving as a benchmarking service for the organizations seeking accreditation. In addition, ideal standards would provide the content for publicly available comparative reports on the performance of the accredited organization.

This view of accreditation standards is reflected in Understanding Accreditation (Young et al., 1983), which notes four trends in the accreditation process: (1) it has moved from a more quantitative to a more qualitative system of assessment, with more general rather than specific standards; (2) it has placed less emphasis on making institutions look alike and more emphasis on a stance of recognizing and encouraging individuality; (3) it has evolved from a system based more on external review to a system of self-evaluation and self-regulation; and (4) it has moved from a focus on the institution to a focus on encouraging and assisting the organization in its efforts to improve quality.

In this light, standards should describe important functions related to the protection of research participants, and they should be framed as performance objectives that are unlikely to change substantially over time. Because standards aim to improve outcomes, they should place minimal emphasis on how to achieve these objectives. In addition any set of standards should make clear which standards are cores, that is, those that must be applied across programs and that are essential to an HRPPP. Some standards, such as those that directly relate to the protection of human research participants, should carry more weight than others. It is especially important that clear measurement tools be available for core standards and that guidance on how the measurement will be interpreted is available.

NEED FOR STANDARDS TO ENCOMPASS MULTIPLE RESEARCH SETTINGS AND METHODS

Recommendation 5: Accommodate Distinct Research Methods and Models Within Accreditation Programs

The accreditation process should accommodate other research organizations in addition to the tradtional models provided by academic health centers and VA facilities. The accreditation process should also cover research other than clinical research.

Standards must accommodate the distinct natures of several types of organizations, including research institutions, educational institutions, independent IRBs, academic medical centers, nongovernmental organizations, and private interests. A set of standards can make clear the scope of institutions to which they apply in several ways: (1) state explicitly in the preamble the intended focus of the standards; (2) include flexible language, such as “where applicable” or “as appropriate” to certain standards so that institutions not engaged in particular activities (e.g., nonmedical, low-risk research) could be exempt from certain standards (e.g., reporting of adverse events); or (3) organize the standards so that institutions and accreditation bodies can quickly ascertain which sections apply to them and which ones do not.

If standards were structured in a manner that requires the existence of a single entity with exclusive authority over all parties involved in the research process, then the three requirements listed above would not apply. It must be recognized, however, that certain organizations, such as independent IRBs and some private sponsors of research, would then not be eligible for accreditation. This would be an unfortunate consequence, as it would exclude organizations that play an increasing role in the research enterprise.

Accreditation of an independent IRB, for example, might use only the subset of standards pertinent to IRBs, but doing so would also require formal assurance regarding the functions covered by proposed standards that pertain to investigators, research institutions, and research participants, as well as standards that pertain to sponsors but that are not yet incorporated into NCQA or PRIM&R standards (but covered by ICH-GCP guidelines) (see discussion below). Another approach would be to accredit the organization that directly controls all the relevant elements of an HRPPP (e.g., a contract research organization that has a formal agreement with an independent IRB to review all its protocols, the research unit of a private firm, the unit of a federal agency that performs research, or a clinical trials cooperative group). One of the virtues of a nongovernmental voluntary accreditation process is its flexibility, and nongovernmental accreditation bodies should not find it difficult to accommodate disparate organizational structures. It is not yet clear, however, how the current proposed standards or accreditation processes would do so.

Although there is a natural tendency to develop standards and review procedures around a specific model, accommodation of innovative or unique organizations is central, and although “basing development on a commonly accepted template may benefit the accrediting organization, there is a danger that innovative structures or processes undergoing accreditation will encounter additional challenges or problems in the review process” (Hamm, 1997, p. 31).

In addition to accommodating distinct types of research infrastructures, the language of standards should acknowledge that even though the principles that underlie them apply to all human research, the criteria and mechanisms for review must be adaptable and must be based on the nature of the research being conducted and the context within which the research is to be performed. The committee heard strong, consistent comments that the proposed standards (in this case, those of PRIM&R) do not fully recognize either the diversity of institutions or the full range of research (AAU, COGR, NASULGC, 2001; Kulakowski, 2001; Ryan, 2001). The standards proposed by NCQA under contract with the VA, however, are necessarily limited in scope to VA facilities. Although the committee believes that the same principles for protection of the rights and interests of research participants apply to all research—for example, biomedical, behavioral and social, public health, and outcomes research—it is likely that the processes needed to comply with the standards will differ depending on the nature of the research. Thus, it is an open question whether the best accreditation strategy would be to use one set of operational standards for all research. That might well prove viable, but it also might prove better to encourage the evolution of different specific standards for different kinds of research institutions.

Those in the best position to make this determination will be organizations devising the nongovernmental accreditation processes, not this committee or the federal government. Whether to develop one set of standards or a few sets of standards specific to a few different classes of research organizations should not be decided by fiat but should be decided in light of experience gained through pilot accreditation programs that include medical and nonmedical sites.

Accreditation pilot programs can begin by focusing on the research institutions for which they were designed, but they might evolve in many different ways. In the future, there could be one or a few accreditation bodies and one or a few sets of accreditation standards, and many different kinds of organizations will certainly be involved in research with human participants.

RELATION OF THE STANDARDS TO THE EXISTING REGULATORY REQUIREMENTS

Recommendation 6: Base Standards on Existing Regulations

Accreditation standards should start from federal regulations for the protection of human research participants but should augment those regulations. The process should be iterative and continual, with evolution of both accreditation standards and the operations of accredited organizations, creating incentives for accredited organizations to improve.

Institutions that receive federal funds, that hold an assurance from OHRP, or that seek FDA approval must comply with the Common Rule or parallel FDA regulations. Therefore, it is important that any standards be considered in relation to the regulatory requirements; that is, are they consistent, supplemental, or contradictory? Many commentators at the committee's public forum, as well as committee members themselves, expressed concern that new standards for accreditation could impose another layer of bureaucracy on a system that is already sagging under the weight of paperwork, but would add little to the protection of human research participants (AAPP, 2001; Cornblath, 2001; Oakes, 2001).

Three issues to be considered in this context: (1) If the standards are identical to federal regulatory standards, both the institution and the accreditor are performing redundant tasks (presumably largely paperwork, assuming that the institution is already in compliance with the federal regulations) unless some simple means is found for the certification of compliance; (2) If the standards are inconsistent with federal regulations, confusion is likely to result; (3) If the standards are more demanding than federal regulations, a question must be raised: are the additional expectations likely to strengthen protections at a reasonable cost?

Accreditation standards should start from the base of regulations governing research with humans. These regulations, in turn, are based on a set of principles for the ethical conduct of research (see Recommendation 4). By the use of standards that emphasize processes of continuous quality improvement instead of an exclusive focus on regulatory compliance (see below), the way may be open to the development of future standards that center on HRPPP performance, in addition to the current focus on documentation. For example, an HRPPP that demonstrates that it can ensure informed consent because it has data showing that participants understand the protocols in which they are enrolled, could begin to supplant or augment paper audits of signed informed-consent forms. This strategy therefore has the potential to introduce the desired flexibility and focus on outcomes into the oversight system. Furthermore, this goal that standards continuously evolve supports the committee's recommendation (Recommendation 2) that HRPPP accreditation bodies be nongovernmental organizations, as the federal regulatory process does not possess the sensitivity and responsiveness to maintain pace with opportunities for improvement.

STANDARDS FOR QUALITY IMPROVEMENT AND SELF-STUDY

Recommendation 7: Incorporate Continuous Quality Improvement Mechanisms into Standards

Accreditation organizations should emphasize the process of self-study, evaluation, and continuous quality improvement among applicants. They should move beyond documentation of informed consent and protocol review, which, although essential, do not of themselves protect the rights and interests of research participants.

Standards provide an HRPPP with the opportunity for benchmarking, a continuous, systematic process used to make improvements. By periodically examining activities, policies, procedures, support functions, organizational performance, and the status of data collection and processing, an HRPPP can develop an approach to quality improvement. A sound system of self-assessment can identify the best practices in an organization and target areas in need of improvement. Compliance with regulatory requirements, in contrast, provides an important but irregular approach to ensuring that protections are in place. Thus, standards not only provide the basis for a system of self-study and improvement but also should incorporate the expectation of such a quality improvement system. This is not to say that self-study alone is sufficient. To maintain the integrity of the accreditation process, an HRPPP must conduct self-study as well as be subjected to external review (whether by an accreditation body or a regulatory agency).

Standards should aim to improve outcomes and should not overly prescribe how to achieve the specified objectives. Rather, they should focus on the core standards that apply across programs and that are essential to a quality HRPPP. Current proposed standards generally reinforce the documentation practices required by federal regulations but do not yet go beyond the regulations. In general, both entities seeking accreditation and accreditation bodies should identify exemplary performance and best practices, providing benchmarks for the research community at large and making information on organization performance openly available to the public and policy makers. In this way, for example, an HRPPP demonstrating a particularly reliable system for the monitoring of participant safety or the reporting of problems in ongoing research, might have an advantage over nonaccredited competitors in seeking support from sponsors or having access to participants, researchers, or students.

NEED FOR STANDARDS TO ENHANCE THE ROLE OF RESEARCH PARTICIPANTS

Recommendation 8: Directly Involve Research Participants in Accreditation Programs and HRPPPs

The formulation of accreditation standards, the accreditation process, and HRPPP operations should directly involve research participants.³

Current regulations lay a foundation for and even invite stronger involvement of those representing the interests of those participating in research. Yet, some “noninstitutional” members of IRBs have little experience as participants in research; they may be independent of the institution, but it does not follow that they represent the perspective of research participants. The regulations are necessarily nonspecific about the involvement of research participants in the review process and set a low standard for qualification. When HRPPPs are regularly judging the benefits and risks of studies that involve particular populations, there should be evidence that the review process directly involved those who genuinely understand and represent the perspective of those populations. This requirement could be incorporated into accreditation standards.

Practices regarding membership on data safety and monitoring boards (DSMBs) are even more diverse. The only stipulated expertise on DSMBs is technical: a clinician familiar with the medical aspects and a statistician familiar with data analysis. In instances in which they attend explicitly to safety and the ethical conduct of research, DSMBs are more apt to include a bioethicist or a lawyer than someone who brings the perspective of research participants. Accreditation standards—and even more so, the guidance documents that accompany them by giving examples of good practices—can improve the HRPPP system to ensure stronger representation of the interests of the research participants.

Given the primacy of the concepts of autonomy in research ethics and the training of IRB members, the relative lack of attention to standards and measures that would systematically cultivate these concepts in both the PRIM&R and NCQA proposed standards is somewhat surprising (see the discussion in the What's Missing section below). Several measures can be taken to bolster these concepts to improve the ethical conduct of research involving human participants. IRBs, DSMBs, research design teams, and merit review committees should increase their level of attention to the involvement of research participants or those who genuinely represent participants' perspectives in the design, selection, review, and monitoring of research involving human participants. In addition to including more research participants in the review and oversight process, standards could require institutions to engage in additional activities to improve the process for research participant involvement in the system.

Institutions that conduct research can create ombudsman programs, particularly for studies that may cause confusion among participants or that entail significant risks. The ombudsman can receive information that participants have about the studies in which they are involved (or in which they are contemplating participation). The same mechanism can be used by research staff or other employees of the research institution who may be uncomfortable with how a study is being conducted, if confidentiality is ensured and anti-retaliation policies are clear (and credible) for prospective whistle blowers.

IRBs can ensure safe, confidential, and reliable channels for the reporting of problems. The channels either can be linked to ombudsman programs or can be independent of them (e.g., having assigned staff and formal policies to encourage such reporting).

Investigators (or IRBs) can test whether participants' consent is well informed by empirically testing it and following up when necessary. Several methods have been studied and reported in the scant empirical literature on research ethics (Sugarman, 2000). One method is to use consent monitors—that is, staff who interview participants after the participants have given their consent to participate in a study to see if they understood the study, the risks and potential benefits, and their ability to leave the study at any time. This option is expensive and time-consuming and cannot be routine, but it could be used for particularly confusing or risky studies and could be done as a general sampling technique or research strategy to guide IRBs about the research that they review.

Likewise, consumer organizations can address the need for informed participant involvement by training representatives to participate directly in the design, review, and monitoring of research.⁴

Private organizations of citizens have long been a potent force in U.S. research policy. Hundreds of private voluntary health organizations are directly involved in advocacy for health research, and they often play decisive roles in decisions about research budgets and priorities, which is perhaps their best-known function. Their concerns do not stop at funding, however, but extend to the ethical conduct of research not only to encourage high-quality research to meet participants' health needs but also to protect the perspective of their constituents. Where the infrastructure already exists, HRPPPs merely need to solicit input more systematically and ensure that consumer groups are well represented on IRBs, DSMBs, and other design and oversight bodies. The constituencies for some conditions, however, are less well organized and may require funding from research sponsors, both public and private, to build the capacity for research oversight.

Accreditation programs can systematically solicit desired outcomes from research participants. In his book on accreditation, Michael Hamm (Hamm, 1997) alluded several times to the desirability of having a focus on outcomes and performance rather than process and structure. The outcomes most desired in an HRPPP are an independent review of risks and benefits and a genuine process of informed consent. Participants are directly relevant to the informedconsent process in particular. The literature on empirical studies of the informed-consent process suggests that investigators often do not know what participants hear, and investigators are poor judges of what participants understand.

Those who develop accreditation standards would do well to directly involve focus groups, consent monitors, and participant representatives (e.g., those who themselves have been involved in past studies or who are educated about the research process and ethical standards but who are also familiar with the interests of a constituency) in specifying the desired outcomes to be incorporated into accreditation standards. Accreditation bodies could invite private voluntary health organizations and other organizations representing research participants⁵ to help formulate points to be considered in the development of accreditation standards and modification of the standards as they evolve.

NEED FOR STANDARDS REGARDING ROLES AND RESPONSIBILITIES OF RESEARCH SPONSORS

Neither the PRIM&R nor the NCQA draft standards address standards for sponsors. The PRIM&R document defines sponsor as, “Any entity that provides funds or other resources to support the research. This entity could be a federal agency, corporation, foundation, institution or an individual” (see Appendix B, Glossary). It is noteworthy that in most cases it will not be the sponsor that is seeking accreditation as an HRPPP. However, there will be some examples in which the research institution that conducts and reviews the studies is also paying for a particular research project. In addition, when the sponsor is a federal agency, the assurance process results in an agreement between the sponsor and the research institution that federal regulatory requirements will be met.

The committee recognizes that it would be difficult to incorporate such standards into the accreditation programs for HRPPPs; however, it believes that such standards should exist. These standards would provide research institutions, investigators, and IRBs with a set of expectations that should be met when they review research protocols sponsored by external sources. The ICH-GCP guidelines provide a useful starting point, although they are narrowly focused on clinical trials.

Accreditation of HRPPPs could leave the responsibilities of research sponsors outside the accreditation framework but not necessarily outside the scope of regulation by the FDA or OHRP. FDA regulations, for example, place the mantle of responsibility for the ethical conduct of research on sponsors, and the ICH-GCP guidelines have a section devoted to sponsor responsibilities. For clinical trials of drugs, devices, and other products subject to FDA regulation, FDA staff would continue to hold sponsors accountable by site visits, audits, investigation, enforcement, and other activities already performed by agency staff. Sponsors may continue to be liable if they do not make reasonable efforts to determine whether participant protection systems are in place at research institutions where they are conducting research. Similarly, accreditation bodies should develop standards by which HRPPPs should determine the acceptability of funding from a given source.

The other alternative is to consider the research units within sponsoring organizations as the logical unit for accreditation, but this would require an entirely new framework and would entail accreditation of dozens of pharmaceutical firms, hundreds of biotechnology firms, and many federal agencies that directly sponsor research. This framework diverges sharply from the accreditation models proposed to the committee.

To address the role of sponsors, standards could include the following:

• The sponsor is responsible, where applicable, for implementing and maintaining quality assurance and control systems to ensure that studies are generated and documented in compliance with the protocol and applicable regulatory requirements.
• The sponsor should ensure that the peer review and design components of funded protocols meet the highest standards and that efforts are made to use the least number of participants possible while maintaining statistical relevance.
• The sponsor should ensure that the research team is appropriately trained and qualified to conduct the research.
• The sponsor should permit disclosure of the financial interests that investigators have in a research project as a result of the funding received for that project.
• The sponsor is responsible for reporting to all concerned investigators, institutions, and regulatory authorities any adverse events resulting from research studies.

REVIEW OF AVAILABLE DRAFT STANDARDS

The two draft sets of standards reviewed by the committee represent an initial step in constructing an accreditation system. However, standards are only as good as the guidelines and measures used to assess compliance with them. Thus, many questions that arise from review of the drafts might be resolved only when they are considered in the context of the guidelines that will accompany them and experience gained through pilot testing.

In reviewing the PRIM&R and NCQA standards the committee found it useful to assess them according to the following general criteria: (1) their scope and focus; (2) their relationship to the existing regulatory standards; and (3) the extent to which the standards can be consistently implemented, measured, and enforced, as well as their inclusion of various key elements (see Table 3-1).

TABLE 3-1

Elements in Three Sets of Standards and Guidelines.

In addition to the two sets of proposed accreditation standards examined, the committee considered the ICH-GCP guidelines on the basis of their inclusion of widely accepted guidelines (internationally and domestically) for research sponsors and investigators involved in clinical trials.

INTERNATIONAL CONFERENCE ON HARMONISATION GUIDELINE FOR GOOD CLINICAL PRACTICE

The ICH-GCP was developed as a handbook for researchers conducting clinical trials, particularly drug trials conducted by sponsors and researchers from more than one country (International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 1996, 1997). Although the guidelines presented in the ICH-GCP are not actually standards, they provide a clear and explicit set of best practices for those conducting clinical trials (see Box 3-1). The committee looked to the ICH-GCP because it includes defined goals for sponsors and investigators. However, it does not address, per se, the institutions or the setting in which the research will be conducted. As such, the ICH-GCP is “portable” and is therefore an important contribution to enhancing the protection of research participants, wherever the clinical trial is conducted. Aspects of the ICH-GCP serve as clearly delineated models for investigator and sponsor behavior, and, thus, the responsibilities contained within these models should be included in the development of guidelines for HRPPPs. The ideals or norms that the document espouses, however, would need to be translated into standards, and such standards would have to be applicable beyond clinical trials and biomedical research methods.

BOX 3-1

The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use is a project (more...)

RECOMMENDATION FOR INITIAL STANDARDS TO BEGIN PILOT TESTING

Recommendation 9: Use Modified NCQA Standards to Initiate Pilot Programs

Pilot accreditation programs should start from the accreditation standards and processes proposed by NCQA for VA facilities, as adapted for use in other organizational contexts. In expanding the draft NCQA accreditation standards for use beyond VA facilities, the standards should be strengthened in six specific ways as pilot testing commences.

The PRIM&R standards were prepared for a broad set of potential applicant organizations, which would include but not be restricted to academic health centers. The NCQA standards were explicitly prepared for accreditation of VA medical facilities. In this instance, the applicant pool is defined, and, in fact, pilot tests that will use those standards are being planned as this report goes to press.

As noted throughout this discussion of report recommendations, the committee regards the NCQA standards as an excellent starting point for accreditation of VA facilities. The committee recommends, however, that the NCQA standards be strengthened in six areas, to specify (1) how investigators will be reviewed beyond the review of the protocols that they submit for IRB approval;⁸ (2) whether and how research sponsors will be assessed in the accreditation process;⁹ (3) how participants will be involved in setting standards and accrediting HRPPPs;¹⁰ (4) how oversight mechanisms can ensure participants' safety in ongoing research;¹¹ (5) the steps that research institutions and their leadership can take to cultivate a culture that puts the safety and interests of research participants foremost;¹² and (6) mechanisms by which research institutions and, where applicable, research sponsors can be held accountable for ensuring sufficient funding, structural support, and professional rewards for HRPPPs.¹³

The NCQA standards, if improved as recommended, could also be used—by NCQA, the Association for the Accreditation for Human Research Protection Programs (AAHRPP), or other accreditation organizations—as the basis for the development of accreditation standards for non-VA research organizations.

Accreditation will not be successful until it is widely accepted as a mark of excellence. To accomplish this, it should serve as an educational tool to raise the median overall performance of an accredited organization. To do this, accreditation standards and the processes in which they will be used must incorporate consistent feedback from the parties involved in the various aspects of an HRPPP. As discussed above, the local aspects of this issue (i.e., aspects that apply to individual applicant institutions) should be enhanced in the NCQA standards. The committee is encouraged that both NCQA and AAHRPP include stakeholder representatives in their programmatic leaderships (see Recommendation 2). Those who encounter problems in the research protection system, irrespective of the perspective that they represent in that system, need simple, consistent ways to bring their concerns to light and to bring relevant information into the procedure for the review of the process at the level of both the HRPPP and the accreditation process.

It is the committee's understanding that the NCQA standards will be tested in a pilot study beginning in the spring of 2001.¹⁴ This is an important step in gauging the feasibility of the use of these standards for the accreditation process, and the committee encourages similar pilot testing with appropriately modified standards in non-VA research environments.

Footnotes

1

The committee does not mean that any member who could have a conflict with any conceivable protocol coming to an IRB for review should be excluded from service on an IRB but, rather, that the individual should recuse himself or herself from reviewing such protocols.

2

The principles laid out in The Belmont Report are one foundation (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979). Accreditation standards, however, should also incorporate the recommendations of the President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research (President's Commission, 1981, 1983), the recommendations of the Advisory Committee for Human Radiation Experiments (ACHRE, 1995), recommendations presented in reports of the National Bioethics Advisory Commission (NBAC, 1997, 1998, 1999a,b, forthcoming-a,b), the recommendations of the Office of the Inspector General of DHHS (DHHS OIG, 1998b, 2000b), and the recommendations of the General Accounting Office (GAO, 1996). In addition, recommendations from reports and declarations of private bodies and independent scholars should be incorporated. This presupposes that an advisory apparatus is available to cull this literature.

3

By “participants,” the committee refers to those whose background and expertise are credible to a lay constituency external to the research institution and who are knowledgeable about the research process and research protections. The term is further defined in Chapter 1.

4

The National Breast Cancer Coalition, for example, has Project LEAD (Leadership, Education and Advocacy Development) that trains advocates to serve on research review and advisory panels, and the National Alliance for the Mentally Ill has a program that trains members to serve on IRBs.

5

For research not on a particular medical condition, the constituency may be, for example, veterans at VA facilities or representatives of the general public familiar with research methods and ethical canons for general population studies.

6

For more information on this process, see http://www​.ncqa.org/Pages​/Programs/QSG/vastandards.htm.

7

For example, Standards 1.1, 1.7, 3.1, and 3.2 (Appendix B). Documentation standards are more specific, but many other standards are similarly hortatory.

8

For research programs involving only a small set of investigators, accreditors might contact all of them; for most programs, however, accreditors would need to sample investigators in a way that is independent of control by the IRB or the institution's research administration. How to do this will likely vary by institution and will have to be specified in advance by the accreditation body. The sampling procedure is likely to evolve during the pilot testing phase.

9

Some organizations do little or no externally sponsored research so they would be exempt from this aspect of accreditation review. Organizations that do sponsored research will vary widely in the number of protocols and the kinds and numbers of sponsors. For programs with extensive externally sponsored research portfolios, accreditation bodies will need to develop sampling methods that are credible and independent of the organization's IRBs and research administration. Standards for this aspect of review could initially start from the ICH-GCP guidelines noted in Table 3.1.

10

Accreditation bodies will need to develop methods to sample participants in a manner that is credible and independent of IRBs and research administrators of the organizations seeking accreditation. Participants were not surveyed in the 1998 survey of IRBs and investigators commissioned by National Institutes of Health (Bell et al., 1998), yet the committee believes that participant perspectives are essential to judging whether an HRPPP is operating effectively.

11

Chapter 3 describes some options for research monitoring and feedback. When organizations applying for accreditation conduct research that is monitored by DSMBs, for example, details of how those boards interact with investigators, IRBs, and research administrators would need to be evaluated for all or a representative sample of DSMBs. Reporting mechanisms for severe or unanticipated adverse events would similarly be necessary to evaluate all protocols or a representative sample of protocols. Ombudsman programs and reporting mechanisms for concerns, complaints, and other feedback mechanisms would be included. Pilot testing will likely reveal a wide variety of monitoring and feedback methods that will have to be accommodated in the accreditation process.

12

PRIM&R's Standard 1.16 calls for assessment of quality improvement programs, and NCQA's standards presented in Table C-3(B) do so with even more specificity. The committee believes that procedures for evaluating the informed-consent process in particular deserve special attention and will be both the foundation of effective protections and the best hope of shifting from documentation to performance measures.

13

Budget and staffing for IRB operations, monitoring and ombudsman programs, and other HRPPP components are not sufficient to evaluate quality and effectiveness. Insufficient budgets and staffing, however, would be clear indications of deficiencies. The committee sought information about budgets and staffing, but found few data. (The 1998 report by Bell and colleagues contains some data on IRBs and investigators at 491 institutions; it does not, however, include data on IRBs regulated only by the FDA, monitoring bodies, or administrative costs.) Extant data were insufficient for the committee to develop benchmarks for different kinds of organizations seeking accreditation. Such benchmarks will thus have to be established in light of experience from pilot testing.

14

As this report went to press, NCQA made their draft standards available for public comment. See http://www​.ncqa.org/Pages​/Programs/QSG/VAHRPAP​/vahrpapdraftstds.htm for further information.