Overview of New Single-Cell RNA Technologies

Fertig began with an overview of single-cell technologies by drawing an analogy to different preparations of fruit, comparing the different technologies to smoothies, individual pieces of fruit, and a fruit tart. She compared the previous generation of transcriptional profiling technologies (i.e., bulk RNA sequencing, or RNA-seq) to a fruit smoothie, in that each component of the system is included, but the cellular and molecular components are all blended together to investigate the resulting mixture. In contrast, single-cell RNA-seq can be compared to pieces of fruit, in that each piece of fruit is evaluated individually to observe its molecular state. She likened future single-cell technologies—specifically, spatial transcriptomics—to an elaborate fruit tart. These newer technologies have the potential to reveal not only the presence of each cell, but also the spatial alignment of those cells in a tissue, allowing a system to be characterized in its native context in order to understand its underlying processes. These single-cell RNA technologies are being used to better understand particular cell types involved in these systems, Fertig said. This was previously done using older technologies, such as flow cytometry, but single-cell RNA technologies provide higher-dimensional resolution than their predecessors. She discussed some of these new developments in single-cell RNA technologies related to cell types, axes of cellular identity, and cellular state transitions (Wagner et al., 2016).

Characterizing Cell Types and Axes of Cellular Identity

The single-cell development of the retina across developmental time-points and cell types has been characterized (Clark et al., 2019; Stein-O'Brien et al., 2019), and Fertig showed a dataset where dots are plotted three-dimensionally to represent individual cells in a retina. The dataset included more than 100,000 cells coded to indicate age and cell type, thus illustrating how the dataset transitions over time. This presentation allows the dataset to be viewed in two ways: either in terms of age or in terms of cell type. Annotation based on these labels limits characterization to a priori knowledge of the system, Fertig said, but the dataset has the power to characterize the entire transcriptional profile of the system beyond just cell types. For example, within each cell type, the single-cell approach can provide information about its phenotype, temporal progression, developmental trajectory, progress along the cell cycle, and spatial position (Wagner et al., 2016).

Encoding this additional information raises the question of how to explore these data in deeper way in order to determine what those phenotypes are, rather than merely clustering or annotating the cells, Fertig said. This question has been addressed in the field of mathematics using matrix factorization (Stein-O'Brien et al., 2018). Using Coordinated Gene Activity in Pattern Sets (CoGAPS), a Bayesian matrix factorization method, a large dataset containing thousands of genes can be analyzed in terms of amplitudes (i.e., gene weights for each biological process) and patterns (i.e., biological processes in each sample) (Fertig et al., 2010). The matrix factorization algorithm finds patterns that define the biological activity in a set of samples (e.g., time points) and the amplitude at which each gene contributes to that biological process. Matrix factorization can be used to learn about the dimensions within a dataset. By linking the results of matrix factorization back to genes or cellular site, it is possible to link back to observed phenotypes, which are fundamental to the system.

Matrix factorization can be applied to uncover the features of cellular identity using the single-cell reference dataset of retinal development, Fertig said (Clark et al., 2019; Stein-O'Brien et al., 2019). She presented a three-dimensional plot showing how some features that undergo the CoGAPS matrix factorization approach reflect individual retina cell types, such as the neurogenic cell type. In addition to learning about cell types—which was already possible with clustering methods—this approach can be used to learn about the fates that reflect changes in cell states. In these three-dimensional Uniform Manifold Approximation and Projection (UMAP) plots with weights of the patterns learned across cells through the matrix factorization, a transition region indicates a space where some cells occur in a pattern that is enriched for cell cycle genes. This pattern is consistent with the desynchronization of the cell cycle, as cells transition from stem cells to developing their final states. By looking across the different axes of this dataset, it is possible to tease out all the different cell types as well as the different transitory states related to age within one algorithm.

Modeling Cell-State Transitions

The inference of cellular state transitions offers a global perspective for observing the system, but the other axes provide more insights into occurrences within the system, Fertig said. Thus, understanding the inference of cell-state transitions and interactions requires new algorithms. Understanding one cell type does not account for additional molecular heterogeneity within that cell type, especially when attempting to understand that variation. Some variation may be associated with cell-state trajectories, such as in stem cells. In addition, there is the challenge of understanding the functioning of these interaction networks and the interaction between cells and molecules that drive phenotypes.

Matrix factorization is not the only approach for modeling cell-state transitions, Fertig said. Indeed, to look at dynamic cell-state transitions, other types of computational methods are more apt. In recent years, two dominant methods have emerged. The first of these is built on the notion of RNA velocity (Melsted et al., 2019), which is based on the idea that the relative maturity of a cell can be calculated by observing the ratio of spliced and unspliced gene products. This information can then be used to determine the cell's trajectory over time, which can be used to calculate the RNA velocity. The other approach relates to trajectory inference, or pseudotime, which involves observing cellular maturity in certain clusters—as well as cells' distances away from each cluster—to determine how the cells change over time and how they are ordered (Trapnell et al., 2014). Each of these metrics can be used to determine a cell's state transitions. These approaches, which use single-cell data, will help move the field through a systems approach that is data-driven but also integrate information about time from dynamic models, Fertig said.

Inferring Cellular Interactions Using Spatial Transcriptomics

Within this broader network-focused perspective, Fertig described a transition in the field from using single-cell data to using fully spatially resolved single-cell data. She showed breast tumor data from a dataset created in collaboration with 10X Genomics using its Visium platform. By annotating the data into separate regions, each region can be spatially resolved into a full transcriptional profile, and matrix factorization can be used to determine the axes of cellular identity in this dataset. This approach makes it possible to differentiate regions in a de novo way in order to identify the dominant signaling processes that distinguish various tumor regions. The same process can be applied within the immune system. Tumor patterns can be linked back to distinct molecular pathways to begin revealing the spatial interactions among tumor and immune systems. In conjunction with network inference algorithms, this approach can provide information about space, time, and regulatory networks. This contributes to the understanding of systems' potential for interaction and of the physical interactions within systems. In closing, Fertig said that single-cell algorithm development is a broad field with much promise. However, new approaches for matrix factorization and latent space analysis are still needed to answer the open questions in the field, such as studying cellular heterogeneity, conducting trajectory and velocity analyses, and identifying network analyses.

Fundamental Challenges

Francois framed the challenges involved in this type of modeling by describing an ideal scenario. Ideally, a high-throughput experimental method could be used to generate data such as mapped representations or complex interaction networks. Then machine learning and mathematical models could be used to generate predictions and reveal the mechanisms or principles of the system's organization. In reality, however, the challenge lies in the translation of big data, machine learning, and mathematics into predictions and mechanisms. Francois described what he calls the “connectionist nightmare,” which emerges, for example, when studying protein interaction networks for T cells (Altan-Bonnet and Germain, 2005; Lipniacki et al., 2008). In this case the mathematical modeling of the systems being studied generates numerous equations, making it difficult to understand what is occurring within the system. A similar effect occurs when using deep neural networks to study systems because it is difficult to understand how deep learning and machine learning networks work (Nielsen, 2015). Finally, this effect is related to adversarial examples, in which machine learning is fragile or highly vulnerable to the introduction of noise (Goodfellow et al., 2013). In such examples the introduction of a slight amount of perturbation to a machine learning process may substantially affect the accuracy of the classification algorithm, Francois said.

Another fundamental challenge lies in the interpretation of data and of the impact of how data are plotted or how they are interpreted, Francois said. To demonstrate this point, he invoked the heliocentric and geocentric models of the solar system. Although the heliocentric model is simpler to describe, the geocentric model of the solar system can be plotted using the same data. Similarly, when using an unsupervised method such as UMAP, the UMAP two-dimensional and three-dimensional representations of datasets are different, which raises the question about which representation to choose.¹ The same considerations apply to representations of datasets over time. When researchers are working with complex dynamical systems, they must choose how best to represent movement over time in two dimensions. Francois emphasized the importance of choosing the best way to represent data and the impact of these choices on how researchers think about the problems they study.

General Modeling Approach

Francois described the general approach used to model dynamic data by identifying a latent space. The first step is to take the data and reduce its dimensionality in a supervised way—which accounts for biological knowledge—using tailored algorithms, such as a reduction algorithm that reduces large networks (Proulx-Giraldeau et al., 2017). Evolutionary algorithms and auto-encoders can also be applied to understand biological data and identify the most relevant features within datasets (Beaupeux and François, 2016; Henry et al., 2018). Once the dimensionality of the complex dynamical system has been reduced, its dynamics can be modeled in low-dimension space. Low-dimension models are easier to study, and one can derive more general results there (e.g., theorems can be more easily proven). Models can be expanded back into higher-dimensional space to validate the models and reconstruct biological properties.

Generative Model of Cytokine Dynamics

This general approach has been applied to develop a generative model of cytokine dynamics, Francois said (Altan-Bonnet and Mukherjee, 2019). T cells interact in the immune system and produce cytokines which determine how the cell reacts to antigens. From that point there are numerous possibilities with various outcomes, Francois said. Next, the antigen concentration or the number of T cells can be varied, to generate IL-2 (a cytokine), which can rise or fall depending on the situation. Many other dimensions or parameters involved in this system can be manipulated to investigate cytokine dynamics. Francois highlighted the importance of ligand strength as a biochemical parameter within cytokine dynamics. Ligand strength affects immunotherapy because the immune response depends on ligand strength.

Francois and his colleagues developed a stepwise process with multiple readouts that uses a robot to study immune response in vitro at multiple time-points. The process begins with harvesting primary immune cells, after which mixtures of T cells and antigen-presenting cells (APCs) mixtures are prepared. Next the robot collects time series data, which are processed along with data collected via flow cytometer. The pipeline currently follows 7 cytokines and 12 markers and takes data at 12 time points, with 50,000 cells per condition. The result is a reliable and precisely generated map of the ways in which T cells react in the test tube in the presence of different antigens. This process generates highly multidimensional data, which are modeled using new techniques developed by Francois and colleagues, including evolutionary algorithms.

Timeline of Drug Development

Richelle began by describing the typical timeline of research activities in drug development. The process begins with studying a disease, with a focus on drug discovery and target identification. Emergent technologies are used to identify a host organism to be engineered. The aim of this step is to identify or engineer a host cell that will be able to produce the best drug that has quality attributes that enable the disease to be targeted as precisely as possible. During the process development phase, host organisms are cultivated in controlled conditions to create an optimal environment for producing the drug. Finally, this process is scaled up during the manufacturing phase in order to produce as much of the drug as possible—as quickly as possible—while still maintaining efficacy and safety.

This is a simplified timeline of the drug development pipeline, Richelle noted; in reality, there are many additional barriers and subprocesses that greatly influence analyses and development processes at every step of the way. For instance, a patient's genetics, sex, and age can influence the expression of disease and the molecular interactions identified in the prospective drug. Moreover, host organisms have their own genetic attributes that can influence protein or molecule attributes. However, the proper interaction between the target and the drug is defined based on these conformational attributes and can strongly influence the efficacy of the drug. Environmental factors can also affect the development process. If the environment is not optimal, it can influence the genetic expression of the host organism, which in turn influences the protein and molecule quality attributes that determine the drug's therapeutic ability to treat the target disease.

Large amounts of data are collected during the development process to monitor and manipulate these types of influencing factors, Richelle said. In her opinion, she said, the biopharma sector as it currently stands is far from being “big data.” Instead, she described the data generated in the biopharma field as “low and expensive” and often not easily accessible in machine-readable format² (Richelle and von Stosch, 2020). Furthermore, the lack of standardization in the existing databases (e.g., varying nomenclatures, the heterogeneity of the experimental methods and analysis methods resulting in many hidden variables) makes them difficult to exploit without extensive processing. With the digital transformation occurring in this sector, biopharmaceutical researchers are now faced with finding ways to integrate these diverse data sources and extract meaningful information. These challenges might take a decade to resolve, she added, but hopefully it will be less with the emergence of new technologies in this field.

Machine Learning as a Driver of Discovery

Machine learning is a powerful tool that helps researchers in all fields extract information from data, Richelle said. Some sectors are positioned for greater gains from the use of big data, while other fields may not see the same benefits. The benefits of machine learning depend on the relationship between the amount of data available and the impact of exploiting those data (Manyika et al., 2011). For instance, Richelle said, machine learning has been applied successfully in the fields of personalized marketing, in which the potential impact of exploiting the large volume of available data is relatively high. In the case of using machine learning to optimize bioprocess development, the potential impact of exploiting data is relatively high, but the amount of data available is relatively low. Still, even though the volume of available data is relatively low, machine learning will likely have a substantial impact on the biopharmaceutical pipeline, Richelle predicted. But large datasets “cannot speak for themselves,” especially when those datasets contain randomness that may result in spurious correlations due to non-causal coincidences, hidden factors, or the nature of big randomness (Calude and Longo, 2017). Furthermore, many of the algorithms used in machine learning are difficult to carefully scrutinize during peer review, for example. According to research by Patrick Riley, “many of the algorithms [used in machine learning] are so complicated that it is impossible to inspect all parameters or to reason about how exactly the inputs have been manipulated” (Riley, 2019, p. 27). Richelle added that typical machine learning processes are like black boxes: although they may be instrumental in developing predictive models, they do not provide explanations for their results. To address this challenge, she suggested that the body of empirical knowledge that has been developed in the field of biology should be incorporated into efforts to apply machine learning to drug development.

Applying Systems Biology to Drug Development

The factors that influence the drug development process are related and interdependent, Richelle said. These relations and dependencies need to be considered—including the feedback effects among these factors—in applying the systems thinking approach to observe the network of interconnected influence factors and to alter that network. Altering such networks is generally done to achieve three outcomes: (1) producing as much of the drug as possible, (2) producing the drug as quickly as possible, and (3) producing a drug that all data suggest is the best product to cure the targeted disease. Systems biology typically uses a network-based approach to organize large datasets and glean insights about complex biological systems, Richelle said. Coherently organizing large datasets into biological networks can provide non-intuitive insights on biological systems that in vivo experiments alone cannot provide. Network-based approaches also offer a platform for integrating and interpreting omics data to explore links between genotype and phenotype. For instance, this approach has been used to map metabolic networks and integrate data sources.

The systems biology approach can reveal more than just a network of reactions; it can generate an interconnected map of cellular functions. This approach has been used to develop algorithms that recapitulate the metabolism of specific cell and tissue types, offering useful insights into metabolic activity under these conditions, Richelle said (Opdam et al., 2017). These systems tools have proven to be invaluable at the level of preclinical research, she said. For example, in designing new drugs, the tools can be used to inform target selection and to make it possible to engineer cells to rewire their metabolism toward the production of a product of interest. While the systems thinking approach has brought much value to the study and manipulation of biological networks, it also has potential to be applied to many other influence factors across the drug development pipeline, Richelle said. The tools of systems thinking can be used for process design, monitoring, and control. They can also be applied to lower experimental effort, increase process robustness, and facilitate the implementation of regulatory requirements, such as quality-by-design and process analytical tools (Richelle et al., 2020).

Knowledge Gaps and Technical Limitations

Metabolic modeling tools can be used in drug development to expedite the process and integrate existing knowledge regarding the target disease, the protein, or the molecule being produced and the host organism, Richelle said. However, she highlighted three areas in which gaps in knowledge and technical limitations currently restrict the routine integration of systems biology tools into the biopharmaceutical product pipeline: (1) real-time bioprocess monitoring, (2) the complexity of metabolic networks, and (3) modeling with hybrid approaches (Richelle et al., 2020).

Dimensionality of Biological Systems

The panel discussion began with a comment from Moos on the theme of reducing dimensionality, and he referred the audience to Huang's earlier discussion about the characterization of cell state in terms of a state vector. Early on, it was suggested that the vector would need to contain 2,000 to 3,000 elements in order to fully describe a cell transcriptome (Huang et al., 2005). Moos asked whether that number of elements is necessary to convey most of the information, because (1) many of those elements are not relevant to the distinction between states and (2) some of the elements could be collapsed into simpler signal transduction pathway elements. Work on factorization approaches has begun to reveal that the dimensionality is a function of the biology that researchers are trying to capture, Fertig said, so there are no right or wrong dimensions. Instead, the focus is on the dimension that the research is trying to uncover in the system. In the days of bulk RNA-sequencing, researchers found that by representing two dimensions in a system it was possible to separate tumor from normal tissue. By increasing the number of dimensions it was possible to separate various tumor subtypes (Fertig et al., 2013). Fertig added that the hierarchy of dimensions in biological systems likely warrants a hierarchy of methods—in terms of multi-scale dimensionality—to uncover a system (Way et al., 2020). This suggests that the systems are fundamentally low dimensional but multi-scale, depending on which dimension the work is trying to capture. Furthermore, various computational methodologies will yield different dimensions or features, requiring different latent space methods based on the target dimension researchers are trying to uncover.

Accounting for Zeros in Datasets

A technical issue related to sampling can arise when a zero appears in a data matrix, and speakers were asked if that means “no expression” or “dropout.” Relatively few efforts have explicitly addressed this question, Moos said, although Fabian Theis and colleagues have started to do so using an autoencoder. The pipeline developed by Francois and his colleagues allows for multiple repeated experiments to account for these types of problems with data points, he said. This and other aspects of their process allow researchers to maintain some amount of quality control when data points are missing. Francois's team can use their dataset to correct for missing data points in a reasonable and coherent way, either by accounting for the missing data points or by using them to show that the experiment has not worked. Unlike single-cell RNA sequencing, this method can systematically evaluate many dimensions of the same data point.

For sampling reasons, researchers may not capture all of the potentially useful elements when using semi-supervised methods to search for the presence or absence of an activated signal transduction pathway, Moos said. He asked whether the one-or-zero issue might be a logistic regression problem that, in a neural-network approach, could help determine whether a neuron has fired. Embedding architecture into systems might help elucidate where things are occurring or not, Fertig said. Even factorization approaches can reveal different features depending on the prior knowledge encoded into the factorized genes. This multi-scale nature of systems is critical for understanding the processes involved, she said. A long-standing issue in genomics research is that the depth of knowledge generated depends on the depth of research focused on, for example, a specific cell type. In genomics, the data analysis should be just as hypothesis-driven as the experiment if the aim is to uncover some phenomenon, she emphasized.

Insights About Biologic Processes from the Projection Space

The projection space that works for visualizations of data correlations also provides insight into the biologic process or network, Francois said. For example, the manifold where data are projected is related to the relative magnitude of two broad types of cytokines, the ones more associated to innate responses versus the ones more associated to adaptive responses. This requires further explanation, she said, because there is likely to be interesting biological information where this low-dimension manifold sits, which might vary between different antigens and T cell receptors.

Using Immune Velocity to Phenotype Immune Health

The possibility of using immune velocity to phenotype a patient's immune health is an area that is still being explored, Francois said. It has not yet reached the stage where it is possible to look at what happens to patients based on their age, gender, and pre-existing conditions, he said. Currently, he and his colleagues are looking at different T cells and T cell receptors to see how immune velocity is defined. He and his team have found that immune velocity is a property of the antigen itself and that the approach can also be applied to other types of immune cells. He was optimistic that the tool he developed with his colleagues would eventually allow for quantifying those properties in various contexts. Moos asked whether immune velocity may be used as a functional test (e.g., for immune system–directed cellular cancer therapy). This is precisely what his group is aiming to achieve with their work, Francois replied.

Optimization in Process Design

Given that many developmental processes are modular by nature, Moos asked if the concept of sequential attractor states might help to simplify process design. A more global perspective would be helpful, Richelle said, but framing all processes as single, unitary operations is also problematic. She cautioned that focusing exclusively on optimizing every step in a drug development process (e.g., in pursuit of optimal conditions for an organism) can risk one overlooking the direct impact of these optimizations on the drug itself. Optimization efforts should consider the influence of that optimization on the molecule being produced and how that affects the efficacy of the drug. Solutions that initially seem suboptimal may actually be optimal for treating the disease, she added.

Hybrid Models of Machine Learning and Biological Systems

Many researchers are moving toward the combination of machine learning and the study of biological systems, sometimes called the hybrid model, Richelle said. However, there is more than one way to consider and integrate these two approaches. For example, in studying the influence of temperature on metabolism, some researchers might introduce numerous kinetic parameters to see how they affect each metabolic reaction, while others might try to combine these parameters into a single statistical effect. There is no one right approach, as different approaches fit different purposes: for instance, she said, developing a very complex model to describe the influence of the temperature on the metabolism at the metabolic reaction level might be conceptually interesting, but less useful for controlling and developing a process. The precise way that data-driven approaches are merged with systems biology will need to depend on the purpose of the merging.

Patient-Focused Biological Systems Approach

In thinking about a biological systems approach, it may be useful to consider if it can begin with a patient, Moos said. Although a patient represents a sample of one, each individual hosts many complex, interacting biological activities. Given the aim of developing a broadly successful suite of therapies, it might be worthwhile to position iterative regenerative drug development processes so that they begin by distinguishing patients who respond from patients who do not. Modeling should begin at the level of disease, and the disease should be a focus throughout the process, Richelle said. A model that is person-specific could be used to explore parallel patient-specific factors (e.g., genetics, age, gender) that influence the development of personalized medicines, thus providing opportunities to expedite the development process. One goal would be to use a patient's DNA sample to evaluate how the disease is manifesting in that patient. This information could be used to appropriately target the specific quality attributes of a molecule that would allow the disease to be targeted most efficiently in that patient. Theoretically, understanding the interactions among those factors could be used to rapidly produce a molecule that is targeted uniquely for the specific patient's expression of the disease, Richelle said.

Clustering Algorithms Versus Matrix Factorization

Many methods for identifying different cell types are based on clustering algorithms that treat all transcripts identically, Moos said, even though the determination of one cell fate over another may not depend on, for example, whether it is expressing aldolase and whether Bmp or Wnt signaling have been activated. Classical algorithms for comparing protein sequences recognize that changing an aspartate for a glutamate is not nearly as consequential as changing an aspartate for a phenylalanine. However, few clustering methods incorporate that recognition. Moos asked how this limitation of existing clustering methods might be addressed. Fertig said that these concerns have motivated her work with latent space representations and matrix factorization, rather than clustering methods, because the former approaches allow for gene reuse. A major limitation of clustering algorithms is that genes are forced to be a member of one class or another, despite the knowledge that genes are reused for multiple processes, as demonstrated by biological systems and systems-level approaches. This is a major advantage of matrix factorization over clustering, she added. Fertig and colleagues are exploring a transfer-learning approach to benchmark transitions from one cell type to another in datasets in which there is a known “ground truth” of a particular cell type. This avoids the reliance on a single gene; rather, it involves looking at gene signatures and how they are preserved across datasets and across cell types, she added.

Impact of Scale and Function Within a Single Transcript

Fertig also discussed that impact of scale and function within a single transcript, which is often ignored even in matrix factorization techniques. Her CoGAPS approach explicitly encodes the uncertainty in each matrix element as a variable. This allows for scaling how much that transcripts express, which can then be down-weighed to avoid bias toward the most highly expressed transcripts. This semi-supervised learning approach may be powerful if various features can be encoded into the system. Furthermore, she added, it may help researchers understand the relevant factors without introducing bias toward the most highly expressed or most correlated genes. The dominance of elements that are abundant rather than important (e.g., comparing transcripts for structural proteins versus receptors and transcription factors) is a limitation in this work that has not received adequate attention, Moos said. Certain practices, such as zero mean and unit variance, continue to endure despite the knowledge that important genes are being regulated in a bistable manner. He asked how this trend may be connected to the idea that multi-omics data—if they too are regulated in a bistable manner—may be amenable to simple scaling as 1s and 0s. Fertig said that this has been considered in terms of weighing down the system by the uncertainty matrix as well as by modeling regulation in a specific manner. The varying error terms in different modalities can then be included in an integrated framework. It can be useful to follow a smaller number of genes by repeating the same experiment over time, Francois added, because following a trajectory over time can reveal interesting geometric features in the data. If different experiments conducted at different times are compared, noise emerges that might not be controllable. Francois also commented on the critical period of transition between the monostable and bistable phase, during which a set of cells can be followed as a function of time. Following the entire trajectory can also reveal interesting features. The critical period of time between phase transitions is crucial in studying and modeling developmental systems, he emphasized.