ACHIEVING CONFIDENCE IN MEASUREMENTS FOR REGENERATIVE MEDICINE PRODUCTS

The process of defining and measuring the critical characteristics of regenerative medicine products is often listed as one of the fundamental challenges in the field, Anne Plant told the workshop participants. “Figuring out what the quality attributes are that define a product is not a straightforward thing to do,” she said. The quality attributes of regenerative medicine products can include identity, quantity, purity, sterility, viability, and markers of biological activity, Plant said. While FDA defines a number of quality attributes that must be provided with any regulatory submission, there are many different types of products in development, each with its own set of unique characteristics, making it challenging to develop common assays for product characterization. Furthermore, Plant said, for many regenerative medicine products that are in development there is not yet a complete understanding of their mechanisms of action. As a result, it can be unclear what needs to be measured in order to assess biological activity or identity.

Furthermore, it is often unclear which in vitro metrics will predict in vivo activity, Plant said, which can be challenging when the goal is to develop products that are both safe and effective at treating disease. Gaps in fundamental knowledge about biology are partly to blame for this situation, she said, but so too is the fact that there are many variables in the analytical assays and in the differences between the samples harvested from the patients who will receive these therapies. Because there are so many assay and sample variables, it is difficult to design experiments and measurement methods that can get a handle on all of these things simultaneously, Plant said. To complicate the situation even more, the cells and tissues under study are dynamic, living entities that are growing and possibly even changing.

Defining CQAs for a particular product is challenging without accurately measuring endpoints. It is important to ensure correct measurements, Plant said, but it is also important to take measurements that are meaningful to the clinical outcome. Ensuring the comparability of measurements is another major challenge for researchers and product manufacturers in this area, she said. Given the complexity of the measurements needed to characterize these products, she continued, it is important to have confidence that measurements can be compared. Comparability is especially important because there may be many changes to how a material is handled as a process moves from the research laboratory to the manufacturing setting, Plant said. Changes during the scale-up process can include new raw materials, new suppliers, new storage conditions, and even new technicians. Therefore, she added, product characterization is needed as the manufacturing process changes to make sure that the product generated in the manufacturing pipeline has the desired characteristics. If manufacturers are unable to demonstrate that a product has the same characteristics as it did in the past, they may be asked to perform additional experiments or even perform another clinical trial, Plant said. She noted that there is FDA guidance about how to qualify and validate measurements.¹

Characterizing an assay's precision, reproducibility, accuracy, robustness, sensitivity, specificity, dynamic range, response function, and limit of detection can lead to confidence that measurements are yielding data that can support good decision making, Plant said. Knowing these variables can help manufacturers draw comparisons during the process of scaling up and allows them to have confidence that when they get an answer that is unexpected, it is due to changes in the product and not the assay.

FDA requires validated assays for registration-enabling clinical trials. However, Plant said, validating assays as early as possible leads to better decision making at each step along the translation process and to more confidence that an observed effect is real. Developing confidence in an assay and collecting accurate measurements may be easier in the pre-clinical stage, she said, because there will be less variability in the samples being tested.

In Plant's experience, academic researchers will often not have a problem getting reproducible results from an assay in their laboratories, but when results of that assay are compared across laboratories, the results can vary considerably. One approach to addressing this issue is to go back and re-examine the parameter space and where variability in an assay can occur; by doing so, it is possible to create an assay protocol that generates comparable inter-laboratory results. “That gives you confidence that you know what you are measuring and that your measurement process is under control,” Plant said.

In some cases, the availability of reference materials can lead to measurement assurance. For example, NIST is using a standard reference material to calibrate flow cytometry beads made by different manufacturers.² Calibrating each manufacturer's beads to the NIST reference material will mean that each manufacturer's beads will be normalized to one another, Plant said. In other cases, when the measurement is too complex for a reference material composed of cells, it may be possible to build rigorous statistical models based on fundamental principles that researchers can use to evaluate relative precision and accuracy, as NIST has done for cell counting, she added.

Plant noted that a NIST-led consortium, in partnership with the Standards Coordinating Body, is working to improve measurement assurance in gene editing.³ This project will compare existing assays, define minimum metadata to report, design benchmark materials, and compare informatics platforms. A number of elements can contribute to more robust assay development, Plant said, including

• inter-laboratory studies
• experimental design
• testing assumptions
• traceability to a reference material
• statistical methods
• assay qualification
• consistent reporting

While working through this list can be time consuming and requires a great deal of effort, doing so may reduce the risk of making decisions based on wrong answers and potentially slowing down a development project, Plant said. Community efforts can mitigate the challenge of working through this list, she added, and standards development organizations can be good partners in such an effort. These organizations can codify best practices and define reliable protocols. Workshops, white papers, and data sharing can then spread these best practices and reliable protocols.

In summary, Plant said, developing assays that generate comparable data and allow for a better understanding of the important characteristics of a given product or system will allow the researchers and manufacturers in the field to learn from one another's experiences, share those data, and perhaps develop a better understanding of mechanisms of action and complex biology. NIST is trying to help the regenerative medicine field through hosting a number of workshops, many held in collaboration with FDA,⁴ she added.

POSSIBLE OPPORTUNITIES AND CHALLENGES FOR DETERMINING CRITICAL QUALITY ATTRIBUTES IN EARLY-PHASE CLINICAL TRIALS

The comparability testing of CQAs could be carried out more efficiently by tapping into global collaborations as a way of accessing a larger number of patients and managing costs, Linda Kelley said. As of April 2017, there were 55 approved cell-based therapeutic products (Bersenev, 2017), and only one-third of those had received regulatory approval in the United States. This difference is due in part to the geographic location of the patients who participated in clinical trials, Kelley said, but it is also a result of new infrastructures that are being implemented in other countries. The number of industry-sponsored and academia-sponsored clinical trials continues to increase, she said, making partnerships between industry and academia more common as a means of leveraging the science, know-how, and cost-sharing (see Figure 3-1).

FIGURE 3-1

Total number of academic versus industry-sponsored cell therapy trials, listed in databases. SOURCES: Linda Kelley, National Academies of Sciences, Engineering, and Medicine workshop presentation, June 26, 2017. Originally from A. Bersenev. 2016. Number (more...)

When such collaborations first started, there was a very steep learning curve on how to make them work, Kelley said, but that learning curve is starting to plateau. Valuable experience has been gained on how to make these collaborations effective, she said. Nonetheless, global collaborations still have constraints, particularly in navigating the different regulatory frameworks that exist worldwide. In the United States and Europe, for example, select cell and tissue products require pre-market approval, while in Japan many of the products are considered on a decision-to-treat basis. With regard to CQAs, the U.S. Pharmacopeia (USP) and the European Pharmacopeia play similar roles.

As an example of an international, multi-centered trial, Kelley discussed her experience carrying out Phase I and Phase II clinical trials with autologous CAR T cells. The trial originated in Brussels and included her institution in Tampa, Florida. Part of the purpose of the collaboration was to treat patients in both countries and to manufacture the cell products in two places. The idea, she explained, was not for her facility to manufacture cells for U.S. patients and for her collaborators in Brussels to be manufacturing cells for European patients, but rather for the products created at the two sites to be interchangeable. By creating redundancy in their manufacturing capacity, the hope was that this would allow the team to mitigate issues that arose at one site by potentially allowing manufacturing to continue at the other site, she said. Another reason for the dual site manufacturing was to create flexibility in the production process to accommodate the European regulatory agency's regulations regarding audits that can require facilities to shut down completely.

The challenge for Kelley and her collaborators was to demonstrate that the two manufacturing sites could produce comparable products so that the data from both sites could contribute to the results of the clinical trial. The process of creating autologous CAR T cells is lengthy and can take approximately 3 to 4 weeks, Kelley explained. The first step involves collecting T cells from a patient, and this is followed by transduction, expansion, and cryopreservation of the cells. Next the cells are subjected to a variety of quality control assays and release tests before the final product is shipped for treatment of the patient.⁵ Kelley and her colleagues hypothesized that by making sure that these steps were executed in the same way in both locations, they could create the same product in both facilities.

Ideally, Kelley said, the time to address CQA testing is during the original product development and also during technology transfer and whenever the manufacturing process changes. In the case of this particular collaboration, both she and her collaborators started with well-validated analytical methods with well-developed standard operating procedures, although at first there was no consensus between the two groups as to whose methods were the best. Where possible, the collaborators used known reference samples, and they used actual patient samples when doing their comparability testing. The collaborators also identified critical reagents, and in their initial studies the two groups shared exactly the same reagents, not just reagents from the same vendor or with the same lot number. The two groups established acceptance criteria, a process that required having technical experts in the same place at the same time. Ultimately, the team from Brussels traveled to Kelley's laboratory and spent 1 month working on this approach.

Kelley and her collaborators went through the process of establishing acceptance criteria before they started tackling the manufacturing process. Given the desire of the stakeholders in the collaboration to begin the clinical trial quickly, Kelley said, it was challenging to convince some of them that this time-consuming approach was necessary, but with all the planning for logistics and resources it would have been difficult to do it all at the same time. By agreeing on CQAs, analytical methods, and acceptance criteria, Kelley and her team gained confidence in the products they plan to manufacture in this collaboration moving forward (see Table 3-1).

TABLE 3-1

Critical Quality Attributes, Analytical Methods, and Acceptance Criteria.

Implementing timely and cost-effective CQA rapid release assays has been particularly challenging, Kelley said. One example concerned sterility testing, which has been a subject of conversation with regulators and stakeholders in the field for a long time. Currently, FDA requires that all parenteral biological products undergo sterility testing to ensure that the products are sterile when they reach the patient. Previously, the gold standard assay for sterility testing was the compendial sterility method, which takes 2 weeks, but FDA has now sanctioned validated and automated systems for sterility testing such as BacT/ALERT and BACTEC. Historically, the automated systems utilized culture media that is only FDA approved for testing blood products, and therefore they are being used off-label for cell therapy products. New media for sterility testing has recently been introduced to the market with enhanced characteristics that make them more appropriate for cell therapy products (e.g., “industry bottles”), Kelley said. There is good reason to move forward with using this type of technology, she said, but it will require lengthy revalidation. Revalidation can be a time- and cost-intensive process, but Kelley said she is planning on moving forward with a new study with the help of consultants to demonstrate the value of using industry bottles in her laboratory. In thinking about testing for every organism on the U.S. and European Pharmacopeia lists, Kelley held discussions with both organizations about reasonable expectations, and she was able to narrow the list based on the likely contaminants.

Another challenge Kelley mentioned was related to testing for replication competent retroviruses (RCRs), which, if present, can result in multiple integrations within host cell genomes and potentially lead to oncogenesis. RCR testing is currently required by FDA if a cell product is in culture for more than 4 days after transduction. However, a 2012 study found no RCR-positive samples across 29 ongoing gene therapy trials, Kelley said (Bear et al., 2012). Based on their findings, the study authors proposed that master cell banks used for production of infectious virus should continue to undergo rigorous RCR testing but argued that final T cell products that incorporate retroviral vectors and patient peripheral blood samples do not need to undergo active screening at defined time intervals. The reason, Kelley said, is that the original RCR testing method is costly and lengthy. Despite the findings from the 2012 study, FDA requirements have not changed, she said. Therefore, most groups are migrating to using quantitative polymerase chain reaction assays, a faster and more cost-effective option, she said. Doing so requires each laboratory to validate their assays, which can be time-consuming and costly. In closing Kelley asked, “Are we requiring too much for validation and not embracing the recommendations of scientists in the field?”

A COMMERCIAL PERSPECTIVE ON BUILDING PRODUCT ATTRIBUTES

In the early phase of translating products into the clinic, investigators often worry about the core biological hypothesis and demonstrating that the therapeutic approach is safe, said Robert Deans, the third presenter in the session. Regulatory agencies encourage that focus by asking questions of investigators about proof points and whether their methods and assays are providing statistical confidence regarding pre-clinical and early clinical results. On the commercial side, however, there are other attributes that become important, such as whether the product will meet effectiveness endpoints, if it can be produced at a reasonable cost that payers will reimburse, and whether physicians will adopt the new product.

To address these concerns as early as possible in the commercial phase of development, Deans said, companies build what is called a target product profile to guide decisions over the course of the development process (see Box 3-1). The target product profile is an extensive document that covers both the biological properties of the product and its safety and quality attributes.⁶ It also addresses many of the product's financial and commercial considerations, such as whether the ease and frequency of delivery is linked to compliance problems and whether patients can afford the product.

BOX 3-1

Example of Questions Addressed in a Target Product Profile (as presented by Deans, June 26, 2017).

The wave of new products resulting from CAR T cell technology shows how effective product development can be when there is a major clinical effect tied to a well-understood mechanism of action, Deans said. Also important from a commercial perspective is the ability to forecast how technological advances, such as the development of gene editing and gene therapy, might affect how competitive a product will be in the future against the current standard of care approach.

To demonstrate how a company assesses important product attributes, Deans discussed two examples. The first concerned an assay his company developed to demonstrate potency for a mesenchymal stem/stromal cell (MSC)⁷ product in the cardiovascular area. Once industry scientists had identified what Deans called the black box benefit they observed when their cells were injected into an in vivo animal model, they developed an in vitro surrogate assay that had comparable benefits to those seen in the animal model. The company next worked on identifying a negative control by using a cell line that did not stimulate angiogenesis, Deans said, and it used that cell line to compare proteomic and transcriptional profiles and identify key parameters that differed between the successful product and the unsuccessful product. The company also used genetic knockdown and antibody inhibition to find the core angiogenic factors involved in the product's beneficial effects. The knockdown models enabled the company to add those factors back in order to get an understanding of the minimal threshold for biological activity. This approach gave Deans and his colleagues a statistical basis for defining pass/fail criteria. Using the information gained from these studies, company researchers were able to quantify how its production lots varied in terms of their functional performance and the measurement of these parameters (Lehman et al., 2012). This effort resulted in these assays being implemented in a lot release assay panel and also in the development of intellectual property around cell composition.

DISCUSSION