Opportunities and Challenges for Digital Twins in Engineering

Current Methods and Practices

Baron noted that the automotive industry has been adapting digital twin technologies since the 1990s and emphasized that increased adoption of real-time digital twins could accelerate Industry 4.0³ and improve customer-oriented manufacturing, design, and engineering. She defined a digital twin as a dynamic virtual copy of a physical asset, process, system, or environment that behaves identically to its real-world counterpart. It ingests data and replicates processes to predict possible real-world performance outcomes. Processes, tools, and culture are affected, and people play a critical role in testing usability and function in digital twin design. She indicated that digital twins also provide an effective means of communication to account for how people understand and solve problems.

Duraisamy highlighted efforts to train offline dynamic system models to work online—more closely to real time—to attribute causes for events or anomalies; research is under way to create models that both run and make inferences faster. Although challenges related to identifiability, likelihoods and priors, and model errors remain, he noted that many digital twin applications could leverage simple models, algorithms, and decision processes to improve productivity. He described six “classes” of digital twins: Level 1 digital twins provide information to users; Level 2 digital twins assist operators with decision support; Level 3 digital twins empower managers for high-value decision-making with confidence; Level 4 digital twins assist organizations in planning and decision-making; Level 5 digital twins empower organizations to better communicate, plan, and consume knowledge; and Level 6 digital twins define organizations' decisions and create knowledge.

Grieves observed that digital twins typically have three components: (1) the physical space of the product and the environment; (2) the virtual space of the product and the environment; and (3) the connection between the physical and virtual spaces where data from the physical space populate the virtual space, and information developed there is brought back to the physical space.

Gahn described Rolls-Royce's digital engineering framework, where data flow from physical assets to digital models to continually update them. By mirroring physical engineering processes in a digital realm, customers could improve forecasting, reduce life-cycle cost, increase asset availability, and optimize performance. How a company leverages digital twins depends on the desired outcome—for example, eliminating an engineering activity could enable faster, cheaper progress. He emphasized that cybersecurity is a key consideration for digital twins that directly inform decision-making, and a digital twin of the cyberphysical system is needed to protect against adversarial attacks.

Deshmukh provided an overview of commercial digital twin applications for fleet management. When building a representation of an asset for a digital twin, capturing important sources of manufacturing, operational, and environmental variation is key to understanding how a particular component is behaving in the field. Sophisticated representations that capture these sources of variation component by component and asset by asset illuminate how operations could be differentiated. He underscored that reducing disruption, such as unscheduled engine removal or maintenance for major air carriers, is critical in the aviation industry.

Incorporating questions from workshop participants, Parviz Moin, Stanford University, moderated a discussion among the panelists. In response to a question about the security of centralized digital twins. Gahn replied that it depends both on the asset and the types of data. For example, reliability data are usually collated for a digital twin in a central location, but this creates a target for an adversary. Other applications might leverage the edge instead, which offers more security. He explained that securing the digital twin environment is a significant challenge, with data from the asset, data in transit, data stored between those two points, and data for modeling—if the systems are not secure, one cannot develop confidence in those data or the resulting decisions. Baron added that another level of data security to communicate between multifunctional teams would be useful. Moin wondered how proprietary information is handled and whether digital twins should be open source. Duraisamy noted that open sourcing could be beneficial but difficult for some stakeholders, and standards would be useful. Gahn emphasized the value of reference models, and Baron and Duraisamy suggested that a framework could be open source while instances of the data could be highly protected.

Moin inquired about the current state of the art across digital twins. Duraisamy explained that Level 1 digital twins are successful, and applications for Levels 2 and 3 digital twins exist in industry for simple decision-making where systems can be modeled to a high degree of confidence. Duraisamy added that because products are often designed to operate in well-understood regions, parameterization with real-world data could help digital twins become more useful in real-world settings. However, models often are not configured to handle rare events, and geometry changes might not be considered. If the goal is to use digital twins for difficult problems and difficult decisions, he continued, understanding model form error and developing strategies to quantify that error quickly are critical. Deshmukh noted that digital twins should learn continuously from field observations, and Grieves agreed that models will improve with more real-world data for validation.

Moin asked how to quantify the relationship between the amount of data collected and the confidence level in a digital twin's prediction, particularly for low-probability, high-risk events. Duraisamy said that all sources of uncertainty have to be factored in, and understanding whether the algorithms have the capabilities to make a decision within the right time frame is also key. Deshmukh added that ground-truth reality is critical to build digital twins and that models improve with more data, but a framework is needed to update the models continually. Grieves suggested developing a risk matrix to help understand the probability and potential impacts of particular events and to help determine which and how much data are needed. Moin inquired about the norms across use cases that allow one to standardize the quality of a digital twin. Grieves explained that the use case determines whether an online connection is needed for real-time data collection or if data should be collected over months or years.

Moin posed a question about the strength of coordination across component models in terms of model interoperability, data flow, matching data, and frequency of data update needs. Grieves and Baron indicated that coordination is poor. Baron elaborated that many of the data come from multiple sources, have different formats, and have different levels of tolerance in terms of validation and verification criteria. She described this as a difficult but critical problem to solve for digital twins: unifying the data and presenting them in context would allow for consistency throughout the evaluation.

Technical Challenges and Opportunities

Baron explained that an effective digital twin illuminates where systems relate to one another. Providing contextual relationships between these vertical functions⁴ (Figure 1) is both a challenge and an opportunity, as every function adds an important level of completeness to the digital twin. Holistic digital twins could build up and break down experiences, using AI to provide information and incorporating human collaboration when insights reveal problems that should be addressed.

FIGURE 1

Cross-functional collaboration enabled by digital twins. SOURCE: Elizabeth Baron, Unity Technologies, presentation to the workshop, February 7, 2023.

Duraisamy observed that discussions about implementing digital twins tend to focus on the sensors, the data, the models, the human–computer interfaces, and the actions. However, he said that to move from models to actions, the following “enablers” should be considered: uncertainty propagation, fast inference, model error quantification, identifiability, causality, optimization and control, surrogates and reduced-order models, and multifidelity information.

Grieves remarked that manufacturing operations is rich with opportunities for digital twins, where the goal is to use the fewest resources to predict and address future problems. For example, digital twins could improve quality control by enabling virtual testing of a product and help inform a supply network that exists alongside a supply chain. He stressed that interoperability and cybersecurity are essential to prevent threats to the safety of the manufacturing floor.

Gahn elaborated on Rolls-Royce's digital framework, which contains a market twin, program twin, product twin, component twin, production twin, and digital twin. Even within a single organization, each twin requires data from different business units; crossing multiple organizations is even more difficult, with the consideration for protecting intellectual property. Issues might also arise in regulation and certification, and he pointed out that in addition to technical challenges, digital twins have to overcome programmatic, commercial, and legal barriers to ensure the best outcome for the customer.

Deshmukh emphasized that the digital twin “invention to production journey is a team sport.” Challenges include defining the scope (i.e., type of problem and model transparency); migrating the analytics into production (i.e., the outcome determines the use of the edge versus the cloud, and the digital twin has to be capable of identifying analytic degradation and uncertainty with time); and considering data availability in production, a scalable data platform, and the right team balance (i.e., data scientists, software engineers, subject-matter experts, and business owners). Deshmukh remarked that opportunities for digital twins include enhanced asset reliability, planned maintenance, reduced maintenance and inspection burden, and improved efficiency. He stressed that these outcomes emerge by combining automated, continuous, and curated data; expertise from physics and data science; and capable, scalable, and configurable AI and ML.

Incorporating questions from workshop participants, Carolina Cruz-Neira, University of Central Florida, moderated a discussion among the panelists. She asked about regulatory barriers to digital twin adoption. Grieves explained that risk-averse regulatory agencies need to be educated on the value of digital twins, which could be achieved in part by demonstration. The automotive industry has embraced virtual crash testing because many more types of testing can be done at a much lower cost. Although a physical validation will likely always be required, he asserted that better information emerges via virtual testing (assuming the physics are correct). The most significant barrier to progress in this area, he continued, is the established culture of the test community.

Cruz-Neira posed a question about challenges related to the interoperability of and the mathematical foundations for digital twins. Duraisamy commented that when building a digital twin of a product at different stages of its life cycle, quantities of interest vary. However, the level of information needed and how to transfer that information across parts of the life cycle is not well understood. Mathematical models that provide information beyond these quantities of interest are useful for many applications, he said, including for interoperability. Cruz-Neira inquired about the challenges of sustaining a digital twin over a product's life cycle. Deshmukh said that models might not maintain their level of confidence when put into production; “guardrails” to constantly assess the outcome of the digital twin with respect to the ground-truth verification would be useful. Grieves added that digital twins should be “learning models” that capture information about the degradation of old products to better inform and maintain new products.

In response to a question from Cruz-Neira about the use of digital twins for training, Grieves described this application as a significant opportunity to allow for mistakes to be made in the virtual environment. Gahn said that his team has been pursuing the use of digital twins for training with augmented reality sessions for engine overhaul and maintenance. He added that the ability to arrange the manufacturing floor virtually for equipment placement is another opportunity.

Cruz-Neira asked about nontechnical (i.e., cultural, financial, and managerial) barriers to leveraging the full potential of digital twins. Baron explained that every vertical function (e.g., design, engineering, and manufacturing) has its own culture; if issues emerge where these functions converge, questions arise about responsibility. However, she said that high-level leaders should enable employees to work collectively to find affordable solutions by relying on data for contextual insight. Grieves pointed out that decision-makers are not digital natives, and digital twins “look like magic” to them. He advised educating, training, and building trust among decision-makers, ensuring that they understand the potential of the technology and will invest when appropriate. Duraisamy said that no single algorithm exists to solve all and decision-makers should understand that progress happens gradually by assimilating knowledge and incorporating rigor. Deshmukh suggested connecting business outcomes to technology innovations and encouraging leadership to invest incrementally. Gahn agreed that defining a tangible benefit related to specific stakeholders in a language they understand is critical

Research and Development and Investment

Baron presented a hierarchy of digital twins that described their potential capabilities. The “virtual twin” is a physically accurate, realistic digital representation of an asset, facility, or product that emulates its real-world counterpart. The “connected twin” integrates real-time and right-time data to provide insights into the performance of an asset at specific points in time, which requires significant human-in-the-loop interaction. The “predictive twin” leverages data to predict outcomes for the operations of complex facilities and equipment. The “prescriptive twin” leverages advanced modeling and real-time simulation for potential future scenarios as well as prescriptive analytics. The “autonomous twin,” the “nirvana of digital twins,” would use multiple real-time data streams to learn and make decisions to correct issues automatically and enable predictive and prescriptive analytics. She explained that the success of any digital twin relies on people collaborating. The pathway to this success also leverages increased computing capability to solve problems faster, as well as lessons from the past to better predict and address issues in the future.

Duraisamy stated that research on deriving efficient physics-constrained models, as well as those that assimilate information, is needed; ideally, these models would be probabilistic. Efficient and accurate models and algorithms are key to realizing the full potential of digital twins, he added. Investment in methods and algorithms for scalable inference and uncertainty quantification, identifiability, causality, and physics-constrained modeling is also critical—quantifying and effectively managing model form errors and uncertainties remains problematic. He encouraged the community to focus on open standards, common terminology, verification, and validation and championed foundational mathematics to advance digital twins.

Grieves highlighted the need for further research in ontologies and harmonization among groups; interoperability (from cells, to units, to systems, to systems of systems); causality, correlation, and uncertainty quantification; data–physics fusion; and strategies to change the testing and organizational culture. He pointed out that U.S. digital twin research is currently limited as compared to work in Europe and China, and he proposed that digital twins be introduced to students as part of a revised postsecondary curriculum.

Deshmukh explained that data are at the core of digital twin success, and digital twin adoption is the critical end point. He asserted that a scalable platform for structured and unstructured data and the ability to compute at scale are needed. Integrating data science and domain knowledge is critical to enable decision-making based on analytics to drive process change, he said, and tomorrow's “disruptors” will manage massive amounts of data and apply advanced analytics with a new level of intelligent decision-making.

Incorporating questions from workshop participants, Conrad Tucker, Carnegie Mellon University, moderated a discussion among the panelists. He asked them to elaborate on the significance of “interoperability” for digital twins. Duraisamy defined “interoperability” as managing “information” so that it travels seamlessly across the full chain. Baron explained that nothing is developed in a silo: many data have to interoperate in a digital twin for training, and many groups have to work together on a digital twin for problem solving. Grieves noted that machines on the factory floor have an interoperability problem that could be addressed with platforms that “translate” one machine to another. Although standards might not be needed, he continued, harmonization (especially with smaller manufacturers) is essential.

In response to a question from Tucker, Gahn noted that intrusion detection is only one component of protecting digital twins. Many frameworks are available, but a baseline level of protection prepares users with a plan to recover data and modeling tools if an attack occurs. He suggested that users consider protection both in terms of the cloud and the physical asset that moves around globally. A secure download from the asset to the Internet is needed to capture data, he continued, but questions arise about encryption and managing encryption keys.

Tucker asked the panelists how they would convince skeptics to invest sustainably in digital twins. Deshmukh advised focusing on the business problem that could be solved with digital twins: organizations that adopt the technology could offer better experiences for their customers. Gahn added that, to temper expectations, stakeholders should understand what the digital twin will not do (e.g., predict something that is not in the model). Grieves observed the difficulty of “selling” future opportunities; instead of presenting technical pitches to stakeholders, he suggested discussing potential financial performance and cost reduction in the near term. Baron proposed presenting information in context—volumetrically, visually, functionally, and personally—within a globally connected team. Duraisamy suggested highlighting the “digital threads” that could make products more versatile.

Tucker posed a question about opportunities for engineering education, and Grieves encouraged an integrative approach: curricula should include a focus on real-world problems, which would increase student understanding, motivation, and retention. Duraisamy pointed out that although the current curriculum, shaped in the 1960s, has value, it lacks several components. Students have become better system-level thinkers, which is an important skill to accompany mathematics and physics knowledge. He emphasized that concepts should be connected across the entire degree program. Gahn explained that Rolls-Royce values “T-shaped” engineers with breadth across disciplines. He suggested new strategies to introduce digital twin concepts, including microtraining or microcertifications. Deshmukh commented that digital upskilling for the current workforce is essential.

Current Methods and Practices

Celaya pointed out that using the term “digital twin” to describe ongoing computational modeling would not unlock new value or performance. New attributes are critical, and a digital representation could focus on a different perspective of an asset or machine such as reliability. He described digital twins as “living systems” and explained that the large critical assets and the asset fleets that comprise the oil and gas industry require different types of digital twins for optimization. He suggested increased partnership with technology providers to advance data management, use of the cloud, and modeling capabilities for digital twins, which could improve internal productivity and efficiency and enhance products for customers.

Kobryn provided an overview of the AFRL Airframe digital twin program, which focused on better maintaining the structural integrity of military aircraft. The initial goal of the program was to use digital twins to balance the need to avoid the unacceptable risk of catastrophic failure with the need to reduce the amount of downtime for maintenance and prevent complicated and expensive repairs. The program focused on how operators use the fleet—for example, anticipating how they would fly the aircraft and mining past usage data to update the forecasts for each aircraft. Kobryn noted that understanding the “state” of the aircraft was also critical—for instance, any deviations from the manufacturing blueprint, as well as inspections and repairs, would be tracked over the aircraft's lifetime. She explained that all of these data informed simulations to provide timely and actionable information to operators about what maintenance to perform and when. Operators could then plan for downtime, and maintainers could prepare to execute maintenance packages tailored for each physical twin and use them for updates. Moving forward, connecting the simulations across length scales and physical phenomena is key, as is integrating probabilistic analysis. Although much progress has been made, she noted that significant gaps remain before the Airframe digital twin can be adopted by the Department of Defense.

Francom explained that science-based stockpile stewardship emerged after the Comprehensive Nuclear-Test-Ban Treaty of 1996.⁵ DOE invested in simulation and experimental capabilities to achieve this new level of scientific understanding that leveraged available data and simulation capacity to ensure the safety and functionality of the stockpile. He noted that uncertainty quantification, which is critical in this approach but difficult owing to the different categories of physics and engineering involved, could be better understood with the use of digital families. For example, digital twins have the ability to leverage global and local data to refine the understanding of relevant physics and enhance the accuracy of predictive simulations. He indicated that many tools could be useful in collaboration with digital twins—for instance, design of experiments, surrogate modeling, sensitivity analysis, dimension reduction, calibration and inversion, multifidelity uncertainty quantification, and prediction and decision theory—but gaps in the state of the art for reasoning about uncertainty could be pursued further.

Harris remarked that smart cities are the model for sociotechnical systems of the future. He defined the use of digital twins for smart cities as virtual representations of engineered systems that allow engineers and operators to understand how a system performs over time and under future scenarios—that is, a decision-making tool with a real-time, cyber–human collaboration framework. He discussed the potential use of digital twins to monitor bridge structures, for which timescales are particularly challenging. In the past, sensing tools were installed to find damage over time, but it was difficult to understand localized damage. Structural health monitoring then shifted from purely depending on “experimental” measures to relying on “models,” which are more cost-effective than expensive sensor networks. However, he stressed that tracking aging effects in an operational environment over time is still difficult, and a mechanism that represents the physical system is needed.

Incorporating questions from workshop participants, Cruz-Neira moderated a discussion among the panelists. She posed a question about the difference between a digital twin and a simulation. Celaya replied that a digital twin is a living model that reflects the reality of the physical asset through time. Francom wondered if any value exists in making a concrete distinction between the two; instead, he said that the community should strive for inclusivity, focusing on how the modeling and simulation world could improve digital twins and vice versa. Kobryn agreed and added that the distinction only matters when one wants to spotlight special use cases for digital twins. She pointed out that the aspect of “updating” the digital twin with real-world data makes it unique. Cruz-Neira asked if a point exists when so many data are available on a physical asset that the physics-based model would be abandoned for an empirical model. Harris responded that, historically, that amount of data has never been available for large infrastructure. He explained that a realistic strategy to monitor structural health is to determine the current baseline with existing structures and use temporal measurements over time to understand boundary conditions and loading conditions. Celaya highlighted the value of leveraging physics with statistical inference rather than dynamic systems for these cases.

Cruz-Neira inquired about the critical elements for digital twins in engineering. Kobryn explained that digital twins help make the shift from steady-state, single-discipline engineering models and analyses to multidisciplinary efforts that blend the best models and simulations from computational and theoretical spaces with real-world data. Harris emphasized the need to consider what these digital twins will look like when integrated into larger systems.

Cruz-Neira posed a question about how to integrate unit-level models with a system-level twin. Francom pointed out that the system level often reveals one aspect of information and the unit level another. When this happens, the models are mis-specified and model form error exists. Because model form error is difficult to track and quantify, he suggested allowing for disagreement in the models and embracing the uncertainty instead of forcing the models to agree, which creates statistical issues.

Cruz-Neira asked how to collaborate with decision-makers to define the requirements of digital twins to ensure that they are helpful. Kobryn asserted that if the end user is not involved with the digital twin design, resources are likely being spent to solve the wrong problem. At AFRL, design thinking and human-centered design are prioritized. Cruz-Neira highlighted the value of making any assumptions transparent to the end user. Francom noted that a spectrum of digital twins would be useful to explore different assumptions about and explanations for certain phenomena. Kobryn added that model-based systems engineering could make these assumptions explicit for an audience with expertise.

Technical Challenges and Opportunities

Celaya explained that challenges and opportunities for digital twins are business case and context dependent. For example, having a modeler in the loop is not scalable for many use cases, and opportunities exist to better leverage computational power and data access. As a result, the data could move from the physical asset to the digital twin in a computational step, and the digital twin would observe how the physical asset is behaving and adapt. He also suggested collecting data with the specific purpose of the digital twin in mind, and he noted that standards and guidelines could help better link the use case to the model requirements. Instead of leveraging old workflows to sustain digital twins, he proposed developing a new paradigm for optimization to unlock the value proposition of Industry 4.0.

Kobryn presented the following four categories of challenges and opportunities for digital twins: (1) working with end users to understand what information they want from a digital twin and how often that information should be updated; (2) determining how to protect personal privacy and intellectual property, how to secure data and models to prevent unauthorized access and maintain prediction validity, and how to address liability for operational failures; (3) determining the level of simulation fidelity and the level of tailoring for individual assets or operators, and strategies to verify and validate probabilistic simulations; and (4) deciding how to reduce computation time and cost and collect an asset's state and usage data reliably and affordably.

Francom highlighted uncertainty quantification as both a challenge and an opportunity for any problem with high-consequence decisions. Extrapolation is particularly difficult when a digital twin in one regime is used to make predictions in a different regime, he noted, and balancing the physics and the empirical components of a digital twin is critical. Quantifying model form error is also a key challenge, especially when one cannot rely solely on empirical information, which is sometimes incorrect. To begin to address these issues, he mentioned that LANL is exploring opportunities to use information from neighboring systems and to cut the feedback loop when information could corrupt a digital twin.

Harris indicated that no good mechanism is available to deal with existing assets, even though most community stakeholders are concerned about maintaining the function and safety of existing infrastructure. Identifying a way to translate these old systems into more modern formats is needed, he said, although challenges will arise with scale. Most government agencies manage their own systems with limited resources, and working with end users to understand the problem the digital twins are being designed to address is critical. He explained that accessing data for critical infrastructure is especially difficult, given that the information is not publicly available for safety reasons, as is creating digital representations based on decades-old structural plans; however, opportunities exist to leverage new forms of data.

Incorporating questions from workshop participants, Tucker moderated a discussion among the panelists. He asked how a decision would be made to cut the feedback loop of a digital twin when the potential for corrupt information exists, and how one would verify that this was the right decision. Kobryn championed active assessment of data quality with automated technology. She cautioned that as a digital twin continues to be updated in real time, more computational challenges will arise, but implementation of zero trust networks could be beneficial. Celaya explained that domain knowledge could be used to validate information and check for corrupted sensors, corrupted data transportation, or noise. He emphasized the value of both technology and science to address this issue. Tucker wondered how a practitioner would determine whether the source of error is the sensor or the digital twin. Francom noted that the level of trust in the physics and the data is influenced by the fact that the data collection could be flawed. He added that industrial statistics and control charts could help detect whether the sensors are malfunctioning or the system itself is changing. Harris explained that because systems often work off of several sensors, opportunities exist to cross-reference with other sensors to determine whether the sensor or the model is causing the error. Kobryn suggested leveraging Bayesian statistical techniques, where new data are not overweighted. Celaya proposed that models be enhanced to reflect degradation phenomena of systems over time.

Tucker posed a question about the value of optimal experimental design, active learning, optimal sensor placement, and dynamic sensor scheduling. Kobryn described these as significant areas of opportunity for digital twins. For example, by using simulations to determine which test conditions to run and where to place sensors, physical test programs could be reduced and digital twins better calibrated for operation. Francom observed that there is much to be gained from these types of technologies, especially given the high cost of sensors. He highlighted the need to understand what is learned from a sensor; in some cases, sensor capability could be relaxed, but that kind of exercise requires thinking carefully about uncertainty.

Tucker invited the panelists to summarize the most significant opportunities in the digital twin ecosystem. Kobryn detailed an opportunity for product engineering to use digital twin techniques to develop a modeling certification basis for engineering parameters and to validate and design models for deployment. Francom recognized that researchers can learn much from data to improve modeling capabilities; a tighter integration of data and models could help pursue new scientific endeavors. Harris urged the community to gather demonstration cases and to cooperate with clients to understand the parameters in which they work and how a digital twin could solve their problem. Celaya highlighted opportunities for new types of optimization—in sustainability, decarbonization, and environmental protection—as well as for more frequent optimization with digital twins.

Research and Development and Investment

Celaya noted that significant modeling work remains for science to lead to engineering synthesis. He pointed out that methods to address trade-offs (e.g., model utility versus computational cost and speed versus uncertainty) are needed and expressed his support of software-defined modeling as well as contextualizing strong engineering principles in the current drive for empiricism. He encouraged increased investments to enable the scalability of new research as well as to exploit new computation paradigms (e.g., cloud, edge, and Internet of Things) with distributed models. Reflecting on which R&D areas should not receive investments could also be a valuable exercise, he added.

Kobryn described several “enabling technologies” for digital twins. She said that leveraging advances in computer engineering and cloud and mobile computing could help to develop architecture for computing, storing, and transporting; and advances in three-dimensional scanning and sensor technology could help advance understanding of the state of physical systems. The Internet of Things and the democratization of data could also be leveraged, and many opportunities exist in physics-based modeling and simulation to improve sensemaking and interpretation of data. Additional opportunities exist to take advantage of big data and AI, but she cautioned against relying too much on either pure data-driven approaches or first principles physics and instead encouraged fusing the two approaches in a modern architecture.

Francom underscored that investments should be made in uncertainty quantification. He described the need both to consider the information that is entering a digital twin and to recognize the many possible ways to obtain the same answer (i.e., nonidentifiability), which is reflected as uncertainty. He also highlighted opportunities for people with different expertise to collaborate to address the challenges inherent in these nonlinear systems.

Harris encouraged investments to enable smart systems, which should be built and preserved with the future in mind to be sustainable, agile, resilient, and durable. These systems could be realized by leveraging automation, advanced sensing, data analytics, and community-centered engagement. He explained that strategic investment is also needed for successful use cases that demonstrate how end users benefit from digital twins, for corporate partnerships, and for interdisciplinary collaboration. Digital twins could progress with increased investment in the void between basic and applied research, he added.

Incorporating questions from workshop participants, Bingham moderated a discussion among the panelists. He wondered if as digital twins become more accurate, users will confuse the “map” they offer for reality. Francom reflected on the problems that could arise if a decision-maker fails to understand that the digital twin is not reality; this is an example of the value of transparency and communication about digital twins. Kobryn highlighted the need for general training in digital literacy, asserting that many decision-makers have nonscientific expertise. She cautioned against overloading the nontechnical workforce with technical information and proposed helping them understand how to use (or not use) data and develop the critical thinking to understand when information can (and cannot) be trusted.

Bingham inquired about top-priority investment areas. Harris championed large-scale investment in a series of interdisciplinary team efforts and in the human aspect of digital twins. In response to a question from Bingham about investments in high-performance computing and efficient algorithms to deal with the complexity and scale of digital twin applications, Kobryn proposed using the high-fidelity capability from high-performance computing to synthesize data to train reduced-order models to take advantage of available data and computing capability at the operating end. Multidisciplinary teams are needed to reduce the order of models to account for relevant physics while leveraging AI and ML, she said.

Bingham asked if investments in education would be beneficial. Instead of creating an entirely “digital curriculum,” Celaya suggested enhancing the current core engineering curriculum with a new focus on computing capability, disparate sources of data, and uncertainty to improve students' data dexterity and analytical skills. Kobryn proposed that educators focus on systems engineering in context and provide more real-world experiences within multidisciplinary teams (e.g., Capstone projects) to better prepare students for the workforce. Francom suggested that educators build these real-world examples from weather and stock market data. Harris said that the Capstone experience is valuable, but more industry–academia collaborations would be beneficial. Celaya described an opportunity to teach students when to employ empirical science to inform decision-making. He reiterated the value of developing systems thinking, accompanied by open-mindedness, to model reality at different stages of abstraction.