Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions?

The evidence report clearly describes the importance of the potential long-term impacts of spaceflight-induced microbial changes on astronaut health and performance. Because this is a newly identified risk, the committee understands that there is an insufficient mechanistic understanding of induced microbial responses (pathogen and commensal) during spaceflight and spaceflight analogs and of host-microorganism interactions that are relevant to infectious disease. The evidence report was written in 2012 and should be updated with recent findings from a broader collection of relevant spaceflight and spaceflight analog publications, many of which were published after the report was written and are referred to below in the section on additional literature.

Does the Evidence Report Make the Case for the Research Gaps Presented?

This evidence report generally identifies and provides good context and broad support for the research gaps presented and for the existence of high-priority knowledge gaps. More research on the underlying mechanisms and the causality of the observed effects of spaceflight and spaceflight-analog culture on microbial responses and host-microorganism interactions on crew health is needed. The committee identified several additional gaps in the section below.

The committee noted that the knowledge gaps identified in this evidence report need to be updated to reflect the latest gaps (AEH 12-15) listed on the Human Research Roadmap summary website (see Box 2) (NASA, 2014f). These updated gaps, which are shown below, largely focus on the modeling of microbial risk and the evaluation of several key components of this risk, such as crew exposure, crew susceptibility, microbial concentration, and microbial characteristics (including genus-species identification).

BOX 2

Additional Research Gaps Identified in the Human Research Roadmap. AEH 12: Determine if spaceflight induces changes in diversity, concentration, and/or characteristics of medically significant microorganisms associated with the crew and environment aboard (more...)

For the individual research gaps identified in the Host-Microorganism report the committee has provided the following assessments.

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

The gaps noted in the Host-Microorganism report cover a wide range of topic areas. However, the committee identified additional significant research gaps that were not adequately addressed and remain to be explored.

The committee acknowledges that its focus is normally on evaluating research gaps related to identified health risks, rather than on research gaps related to the countermeasures applied to reduce those risks. However, for both changes in host immunity and changes in microbiota (bacteria, viruses, fungi, etc.), countermeasures induce perturbations whose effects on dynamic living systems are not sufficiently understood. Unlike countermeasures applied to well understood physical problems (e.g., g loads, atmospheric gases), the countermeasures applied to microorganisms and to humans may induce dynamic responses and pressures on microbial evolution that affect the long-term usefulness of those interventions. Research about the mechanisms underlying observed changes in microbial responses and host-microbe interactions and research about the impact of spaceflight on related mitigation strategies that deserve additional consideration include the following:

• Impact of infection either prior to or during spaceflight: All evidence of changes in virulence is from models infected post-flight (although the upcoming Micro-5 experiment on SpaceX-5 will address this gap in real time) (NASA, 2014m).
• Microbial risk assessment and clinical relevance: Development of quantitative and predictive models of risk assessment is important and is needed to supplement data from spaceflight and spaceflight analog biological experiments for cause-and-effect relationships. These data are essential to investigate the clinical significance of observed changes in microorganisms and host-microbe interactions.
• Impact of mission design and length: Improved understanding is needed regarding how mission design (including mission duration, food source, and life support systems) would influence the effects of the observed changes in microorganisms and their interactions with the host. Further evidence is needed on the effect of long-term spaceflight on heritable changes in microorganisms associated with virulence and pathogenesis-related characteristics and clinical relevance. Moreover, both short- and long-duration spaceflight and ground-based spaceflight analog studies are needed to understand both transient and heritable changes in microbes and host-microbial interactions, as well as microbial growth characteristics, and alterations in microbiome composition.
• Evaluation of virulence changes: Information on virulence changes in additional pathogens, alone or in the context of mixed microbial co-cultures, would greatly improve understanding of the impact of spaceflight on crew health risk.
• Partial/fractional gravity studies: Research on the effects of partial/fractional gravity (such as gravity encountered on the moon, Mars, and other planets) is needed to supplement microgravity studies and to improve understanding about how microorganisms are affected by gravity (or environmental conditions created by a lack of gravity) (Hemmersbach and Häder, 1999; Hemmersbach et al., 2001).
• Genotypic, molecular genetic, and phenotypic responses: Further understanding of this risk will require a fuller characterization of the effects of spaceflight and spaceflight analog environments on genotypic, molecular genetic, and phenotypic responses of microbial pathogens and commensal microbiota. This includes use of omics-based approaches, such as genomics/epigenetics, transcriptomics, proteomics, and metabolomics. Evidence from omics studies that generate large data sets should be focused on hypothesis-driven goals that facilitate the practical interpretation and integration of these data into a comprehensive and mechanistic understanding of cellular/molecular responses.
• Effect of changes in gene expression: To fully understand the mechanisms associated with spaceflight (and analog-spaceflight) changes in microorganisms, studies are needed to investigate the effect of spaceflight-induced changes in gene expression on stress resistance (including antibiotics and disinfectants), metabolism, pathogenesis, and virulence characteristics (of human, animal, and plant pathogens and commensals).
• Host tissue microenvironment: More information is needed on how potential spaceflight alterations of the host tissue microenvironment could change host-microbe interactions and commensal composition and thus affect host immunity and infection potential.
• Risk of infection by fungi or by reactivation of latent viruses: These risks are underrepresented in this evidence report. Latent viral reactivation during spaceflight, including the clinically relevant varicella-zoster virus, has been repeatedly documented, including one report of a crew member diagnosis prior to flight (Guéguinou et al., 2009; Mehta et al., 2014).
• Effect of sex/gender on infectious disease risks: Because males and females differ in the intensity, prevalence, and pathogenesis of microbial infections, further information is needed on the effect of sex/gender on infectious disease risk in flight. Sex hormones influence microbiota composition, microbial virulence, and immune responses.
• Cellular, mucosal, and humoral immunity: While ongoing experiments on the ISS are starting to address targeted aspects of research on cellular, mucosal, and humoral immunity, this is no more than a foundation for all of the knowledge needed to understand this risk. Studies in this area, including studies on the role of microbiome composition on this immunity, are also encouraged.
• Effect of physical and biological causative factors and their interconnections: There is potential for both physical and biological stimuli to initiate spaceflight- (and analog-) induced responses in microorganisms and host-microbe interactions. A better mechanistic understanding of both physical (e.g., fluid shear, mass diffusion, aeration, radiation) and biological (cellular, molecular, and biochemical) causative factors of spaceflight-induced alterations in microbial responses and host-microbe interactions that could negatively affect or benefit crew health is recommended.

Furthermore, the evidence report needs to include evidence about issues regarding the standardization in experimental methods and tools (e.g., strains used, culture media, sample processing and handling, environmental conditions, time of culture in spaceflight, different organisms used, different hosts used for infection studies) in spaceflight and spaceflight analog studies so that accurate comparisons and reliable conclusions can be made.

Does the Evidence Report Address Relevant Interactions Among Risks?

This evidence report discusses health risks that have the potential to interact with risks discussed in a number of other evidence reports, including (1) Risk of Crew Adverse Health Event Due to Altered Immune Response (Crucian et al., 2009); (2) Risk of Incompatible Vehicle/Habitat Design (Whitmore et al., 2013); (3) Risk of Radiation Carcinogenesis (Cucinotta and Durante, 2009); and (4) Risk of Therapeutic Failure Due to Ineffectiveness of Medication (Wotring, 2011). While a clear link was made to the Altered Immune Response report, links between these other reports were not immediately evident.

More attention should be paid to these interactions. For example, the microbial degradation and deterioration of spacecraft and spacecraft systems (including life support systems) is of concern, especially when coupled with the knowledge that spaceflight has been shown to profoundly alter microbial metabolism, antimicrobial resistance, and microbial community composition. Accordingly, the broader risk posed by microorganisms to crew health extends to the impact of microbes and interspecies microbial communication on the crew habitat/environment and spacecraft and systems integrity. Likewise, multiple reports have provided evidence for an association between altered intestinal microbiome composition and a wide range of diseases and disorders, including infectious disease, cancer, autoimmune diseases, inflammatory bowel disorders, diabetes, antisepsis, and asthma. Moreover, there is an obvious reciprocal relationship between the microbiota and medication, in which microbiota composition modifies the effects of medication and how medication affects microbiota.

What Is the Overall Readability and Quality?

Parts of this report were well-written, but the writing is uneven and the consistency and organization could be improved. One way to improve the report's quality and ease of review would be to standardize the format of the evidence reports.

Is the Breadth of the Cited Literature Sufficient?

The evidence report presented peer-reviewed publications with references in a consistent format. However, some meeting abstracts and bulletins were also cited, and that information was not easily accessible. Findings from a wider breadth of relevant microbial spaceflight and spaceflight analog publications should be included in the next iteration of the evidence report, many of which were published after the report was written in 2012. The committee is pleased to see rapid literature growth on this topic since the time of the report. Examples of pertinent literature for inclusion in the report include Baatout et al., 2012; Cohrs et al., 2008; Crabbé et al., 2013; Crucian et al., 2008; Foster et al., 2013, 2014; Goncharova et al., 1981; Grant et al., 2014; Hemmersbach and Häder, 1999; Hemmersbach et al., 2001; Horneck et al., 2010; Kim et al., 2013; Kish et al., 2002; Mardanov et al., 2013; McLean et al., 2001; Mehta and Pierson, 2007; Mehta et al., 2014; Mermel, 2013; Ott et al., 2012; Pacello et al., 2012; Pierson et al., 2005, 2007; Taylor, 1974; Taylor et al., 1975; van Loon et al., 2003; Wilson et al., 2002a,b; and Yi et al., 2014.

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

As with a review of any broad scientific field, additional perspectives could strengthen this report, despite having been authored, in part, by a renowned expert in the field of spaceflight microbiology. These evidence reports deal with inherently complex systems, and, as such, these systems may be prone to develop “emergent properties” not predictable on the basis of knowledge of individual components, nor by analysis of known interactions. This is most evident with, but not limited to, the microbe-immune system (Milanesi et al., 2009). As such, application of expertise from complex systems science (e.g., Arima et al., 2012; Kalinich and Kasper, 2014; Lakin et al., 2007; Milanesi et al., 2009) might be valuable moving forward. This input may prove especially useful when considering the immunosuppressive effects from a systems approach.

NASA also might consult external multidisciplinary infectious disease experts with specific knowledge in immunology, virology, and medical microbiology (from the clinical and academic settings). This would also provide additional opportunity for synergy of, and interaction between, this evidence report and the Altered Immune Response report.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

This report was developed in response to the 2008 IOM letter report. Thus, there were no other specific IOM recommendations on this topic.

Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions?

This evidence report provides a substantial amount of compelling information supporting the potential for long-term negative impacts of spaceflight on immune status, along with the resulting implications for astronaut health and performance. This risk has clear operational relevance and, as discussed below, studies are needed to more fully understand the mechanisms underlying these changes and the relationship between altered immune regulation and clinical disease.

Does the Evidence Report Make the Case for the Research Gaps Presented?

The evidence report generally provides good context, overview, and depth of knowledge and presents data on observed alterations in spaceflight and spaceflight analog immune function that clearly support the need to understand the causal relationship to disease. However, this report is less than critical in its analysis of observed spaceflight and spaceflight analog findings. Furthermore, this is a quickly growing field of research, and the next iteration of this evidence report should be updated to reflect the new research gaps (IM1–IM3 and IM–6IM8) identified on the Human Research Roadmap summary website (see Box 3; NASA, 2014g).

BOX 3

Additional Research Gaps Identified in the Human Research Roadmap. IM1: The extent to which spaceflight alters various aspects of human immunity during spaceflight missions up to 6 months. IM2: It is necessary to define a flight standard related to spaceflight-associated (more...)

The committee emphasizes the need to more fully consider the risks of long-duration spaceflights and to explore the evidence for changes in partial/fractional gravity environments. Current evidence is generally based on a relatively small number of observations, so conclusions from this evidence, including the impact of altered immune systems on NASA short-term missions, should be interpreted very conservatively until more spaceflight data are available. If such data are available, it would be helpful to have data on changes in environmental stress and the link between these changes and immune dysregulation, including data regarding the impact of recent additions to the ISS that might alleviate stress (such as advanced exercise equipment, Internet connectivity, and entertainment options). Furthermore, this report should more fully consider evidence from studies of ground-based analogs of immune dysregulation that simulate the environmental stress levels in space (e.g., the Antarctic winter-over and the Haughton-Mars Project on Devon Island). As noted in the discussion on the host-microbe interactions report, the evidence for this risk is incomplete without the data on the incidence of in-flight disease. A final note of clarification is that the term “microbiome” should be added to Figure 1 (p. 6).

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

The committee emphasizes the need for further mechanistic understanding of the observed spaceflight and spaceflight-analog changes in immune function and their clinical relevance to crew health (infectious and noninfectious disease). Issues to be explored to further understand the evidence base of this risk include the impact of spaceflight on acute and chronic inflammatory responses and on antibody production; physical and biological stressors and their impact; and the extent to which spaceflight-induced changes are a direct effect (gravity-sensing) or indirect effect (e.g., hydrostatic pressure, fluid shear). Statements made in the Immune Evidence report indicating that immune cells are “gravity sensitive” should be considered for revision, because the statements suggest that there is a direct effect of gravity on cells, while it is possible that these responses may be due solely, or in part, to secondary/indirect effects of microgravity. In the absence of further information on the mechanisms (both in vivo and in vitro), it is difficult to develop mitigation strategies. These additional research gaps deserve consideration:

• Sex/gender differences: As males and females differ in the intensity, prevalence, and pathogenesis of both infectious diseases and autoimmune diseases, further understanding of this risk will necessitate more information on sex/gender differences on innate mucosal and adaptive immune function.
• Microbiome composition: Given the association between microbiome composition and health status, investigations are needed to understand the impact of spaceflight on the crew microbiome and how that correlates with changes in immune status and disease risk. (An ongoing study to profile the crew intestinal microbiome is an example of the type of research that could serve as a foundation for future experiments [NASA, 2014n]).
• Knockout immunodeficient models: Studies using different vertebrate and invertebrate models of immune dysfunction can be useful in determining space radiation effects.
• Viral reactivations: Latent viral reactivations (e.g., Epstein-Barr virus and cytomegalovirus) during spaceflight (as measured by increased expression of viral DNA and proteins in crew bodily fluids) have been reported, and more needs to be understood about the factors contributing to reactivation and infectious disease and cancer risk.
• Cellular mechanotransduction studies: Cellular mechanotransduction studies could provide information on the signaling pathways/events and the spatial localization of signals between cytoplasm and nucleus that may be driving spaceflight and spaceflight-analog responses in immune cells.
• Wound healing: Studies focused on the effect of immune dysregulation and microbiota composition in wound healing could be helpful in understanding the risks of crew injuries.
• Impact of radiation: The relationship between immunity and radiation and its effect on immune system function is key to further understanding this risk.
• Tissue microenvironment: Further information is needed on the effects of spaceflight on the tissue microenvironment and on how resulting changes could alter commensal microbiota composition and thus impact host immunity.
• Risk assessment models: It will be important to develop quantitative and predictive spaceflight and spaceflight-analog models of risk assessment to supplement the data obtained from biological experiments. The development of such models could include (1) accessing information from the longitudinal follow-up of astronaut health to monitor potential spaceflight-induced alterations in immune function and adverse health events that may take years to manifest (especially given the small sample size), and (2) understanding the impact of current preventive measures on the risk for immune-related disease.

Does the Evidence Report Address Relevant Interactions Among Risks?

This evidence report has relevance to many of the other risks described in NASA's evidence reports, primarily the report on alterations in host-microorganism interactions (Chatterjee et al., 2012), but also reports on the risks regarding the design of the vehicle/habitat (Whitmore et al., 2013), on radiation (Cucinotta and Durante, 2009; Cucinotta et al., 2009; Wu et al., 2009), on the effectiveness of medications (Wotring, 2011), on nutrition and the food system (Perchonok et al., 2012; Smith et al., 2009), on exposure to dust and volatiles (James and Kahn-Mayberry, 2009), on sleep loss and circadian desynchronization (Whitmore et al., 2009), on interactions with the central and peripheral nervous systems (Cucinotta et al., 2009), and on the limitations of in-flight medical capabilities (NASA, 2014a). While a clear link was made between this evidence report and the Alterations in Host-Microorganism Interactions report, links between other evidence reports were not immediately evident.

What Is the Overall Readability and Quality?

This evidence report is less than critical in its analysis of spaceflight and spaceflight-analog research findings. Although the report includes information on the negative impact of spaceflight on the functional responses of immune cells isolated from astronaut blood samples (including reduced levels of phagocytosis, antimicrobial oxidative burst, and response to lipopolysaccharides), the location of the evidence is somewhat inconsistent, as some of this information is in the text and some is in the appendix. Integration of the evidence into the context of the HRP gaps would enhance the next iteration of this report.

Is the Breadth of the Cited Literature Sufficient?

The report covers a wealth of information in a rapidly changing field, and the next iteration of the evidence report will need to be updated with findings from numerous key publications, including evidence from spaceflight and spaceflight-analog studies on the impact of these environments on neuroendocrine/hormone function and its relationship to immune status and infectious disease risk, and evidence on sex-based differences in immune responses and resistance to infection.

The literature presented in this evidence report cites peer-reviewed publications with references in a consistent format. Some meeting abstracts and bulletins were also cited, and that information was not always easily accessible. Key literature on the impact of spaceflight-induced alterations in crew immune function on the ISS has been published since the Immune Evidence report was written. Examples of pertinent literature that could be added include Aviles et al., 2003a,b, 2004, 2005; Baatout et al., 2012; Belay et al., 2002; Crucian et al., 2014a,b; Kaur et al., 2008; Mermel, 2013; Milanesi et al., 2009; O'Donnell et al., 2009; and Yi et al., 2014.

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

The authors are highly regarded and are leading experts in spaceflight immunology and have assembled a credible report that provides a general overview of spaceflight and spaceflight-analog research findings. It may be beneficial to include an infectious disease expert in subsequent versions of this evidence report to assist in the interpretation of findings on host-microbe interactions relevant to immune system function and to help align current evidence regarding spaceflight-induced alterations in microbial responses and immune function with disease causality. As noted in the section about the Host-Microorganism report, it may be useful to include a content expert in systems biology and/or complex systems science in future revisions.

As noted in the Host-Microorganism report, it may be prudent to also include external multidisciplinary infectious disease experts with specific knowledge in immunology, virology, and medical microbiology. This would provide additional opportunity for synergy of, and interaction between, this evidence report and the Host-Microorganism report, with the added benefit of better standardization of the format and content of these reports. Because of the extensive interaction between this report and the other disciplines discussed above in the section on interactions, it may also be prudent to solicit input from other discipline experts.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

One of the recommendations made in the 2008 Letter Report was to carry out studies that would help explain the lack of correlation between observed changes in immune function and clinically evident disease in the crew. This remains a major gap in knowledge in the current evidence report.

Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions

The evidence report identifies and provides sufficient evidence for the relatively abstract risks cited, particularly for the three areas in which significant relevant experience is available. For the less well-defined risk involving the emerging issue of human-robot collaboration, the report correctly identifies the overall risk inherent in increasing reliance of human-robot collaboration. Given the rapid advances in this domain and the rapidly growing experience and literature base, future evidence reports will be able to more clearly define the specific risk issues. Particularly relevant for this purpose is recent ground robot research specifically addressing issues in collaboration between humans and robot systems (Ososky et al., 2013; Philips et al., 2011; Wiltshire et al., 2013).

Does the Evidence Report Make the Case for the Research Gaps Presented

The evidence report makes the case, and correctly identifies the research gaps, for the four contributing factors that were discussed: assignment of human and automation resources, perceptions of equipment, design for automation, and human-robotic coordination. Given the rapid evolution of the field of robotics and automation and the significant time before NASA systems will be defined, it will be very important to take maximum advantage of non-NASA developments before committing limited resources that may be redundant. Because the gaps are necessarily high level at this stage of development of human-robot collaboration, this is not a limitation of the current evidence report, and the issue has been appropriately recognized.

Unlike other evidence reports (such as the Altered Immune Response and Host Microorganism reports reviewed above), this report did not explicitly address the research gaps listed on NASA's Human Research Roadmap website (see Box 4; NASA, 2014j). However, because most of the research gaps overlap with the topics discussed in the evidence report, the committee believes that these research gaps are adequately covered. Future iterations of this evidence report should more explicitly address the stated research gaps.

BOX 4

Additional Research Gaps Identified in the Human Research Roadmap: Human and Automation/Robotic Integration. SHFE-HARI-01: We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, (more...)

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

The automation discussion in the report assumes that tasks are largely separable and that the primary decision is how to allocate responsibility between humans and automation. This has been the historical approach and has been adequate for missions and automation tasks in the past. This report does not address functions that will require active collaboration and cooperation among automation, robots, and humans. For long-duration space missions, greater flexibility in the distribution of responsibilities between humans and automation may be necessary, due to limits on the number of crew members and delays in communication with ground control personnel on Earth. The concept of active interaction and collaboration among humans, robots, and automation of all kinds should be addressed. Issues of human-automation communication, situational awareness, and trust are likely to engender additional HARI risk consideration.

Similarly, the report's discussion of situational awareness and automation transparency addresses some of the issues but does not consider emerging automation that may have some level of capability that could be considered cognitive. These may mitigate some risks, while creating others. In particular the concept of social robots, naturalistic interaction with automation, and modeling human-comprehensible cognitive decision making needs to be considered. These technologies are currently in early stages and represent a significant departure from traditional NASA approaches, but they hold the promise of significantly increasing human-system performance.

Specifically, the role of levels of autonomy in influencing the nature of human-robot interaction must be examined in the context of long-duration space missions. Beer and colleagues (2014) have investigated robot autonomy within the context of human-robot integration and propose a taxonomy for categorizing levels of robot autonomy and evaluating the effects of robot autonomy on human-robot integration, including variables such as acceptance, situational awareness, and reliability. With the increasing use of automation and robotics in manufacturing, the role of human-robot interaction and integration in manufacturing is being studied extensively; the findings from this domain will be valuable in the context of long duration space missions with high levels of stress and complexity of interactions. For instance, Hu and colleagues (2013) have investigated the importance of safety-based human-robot collaboration in the assembly of power protectors.

Does the Evidence Report Address Relevant Interactions Among Risks?

The evidence report explicitly identifies interactions between the risks it discusses and the risks covered in the evidence reports on human-computer interaction, critical task design, and training; it also explicitly discusses the need for examining these sets of risks in concert (Barshi, 2012; Holden et al., 2013; Sándor et al., 2013). There is significant overlap between this report and the report on human-computer interaction, but that overlap appears to be coordinated and managed well, at least for situations that come up at present. In discussing missions in the far future, it will likely be more difficult to justify the distinction made between these two reports. Consideration should be given to how best to ultimately merge or manage the cross-cutting issues between these taxonomic divisions.

What Is the Overall Readability and Quality?

The readability of the report is good, and the report appears to cover the important issues in identifying the risks with the exception of the areas previously noted.

Is the Breadth of the Cited Literature Sufficient?

The breadth of literature cited is quite thorough, especially for the more conventional aspects of HARI. The report quite correctly identified a number of issues associated with human-robot teaming and human-automation collaboration; however, the report could benefit from citing more recent publications that address issues of shared mental models between robots and humans, trust and confidence, human perception of robot action, and human-robot teaming. Furthermore, the report needs to explore the challenges and research gaps that are being seen in automation and robotic integration in fields such as manufacturing, distribution, and vehicle and human factors design. Examples of additional literature include Arif et al., 2014; Cabibihan et al., 2012; DeSteno et al., 2012; Fiore et al., 2013; Hancock et al., 2011a,b, 2013; Hu et al., 2013; Kahn et al., 2012; Lebiere et al., 2013; Lobato et al., 2013; Ososky et al., 2012, 2013; Phillips et al., 2011; Saulnier et al., 2011; Syrdal et al., 2010; Van Doesum et al., 2013; Vinciarelli et al., 2012; and Wiltshire and Fiore, 2014.

Much of the literature on robot teaming, social and cognitive robotics, and naturalistic interaction is recent and in journals that focus more on human factors than robotics (e.g., Cabibihan et al., 2012; Fiore et al., 2013; Lebiere et al., 2013; Leite et al., 2013; and Syrdal et al., 2010).

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

Author expertise is sufficient for the scope of the risks that were cited. Consideration should be given to adding some non-NASA authors involved in robotics research, in order to provide a broader perspective on expected HARI developments applicable to NASA. Not directly addressed is the issue of team performance, in particular human-automation collaboration that is likely to be required for missions traveling beyond the moon. The U.S. Army, in particular, has been actively exploring the concept of human-robot collaboration (Hill, 2014). A review of their efforts and of the composition of the research teams involved might be useful in considering possible additions to research team membership.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

The structure of the documents has changed significantly. In the 2008 report, the groupings of issues related to human factors were clustered in Chapter 23 of a single evidence book, Lack of Human-Centered Design, which addressed three risks related to human factors: inadequate information, poor human factor design, and poor task design. There were no previous recommendations relevant to the specific topics on HARI.

Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions?

The report presents the key characteristics of long-term space missions, including, among others, the absence of the safety net of ground control, the long duration of the missions, the varying levels of alertness of mission personnel, and limited resources, and it provides sufficient evidence for the risks that inadequate HCI poses to mission success when carried out under those unique and extreme conditions. The report also recognizes the key limitation that the evidence is drawn from “post-spaceflight crew comments, and from other safety-critical domains like ground-based power plants, and aviation” (Holden et al., 2013, p. 3). The closing sentence of the report succinctly captures how critical this risk is: “Without these improvements, errors due to inadequate HCI will continue to pose a risk to mission success” (Holden et al., 2013, p. 36).

Does the Evidence Report Make the Case for the Research Gaps Presented?

The report uses the framework of the eight core contributing factors associated with the risk of inadequate HCI (derived from the Human Factors Analysis and Classification System), and it makes an excellent case for the identified research gaps. It recognizes that future work “must focus on identifying the contributing risk factors, evaluating their contributions to the overall risk, and developing appropriate mitigations” (Holden et al., 2013, p. 3). Moreover, it identifies the importance of the rapid emergence of touch-based interfaces, which will pose additional risks in operations carried out under extreme conditions, especially when the users wear pressurized gloves during extravehicular activity and are also subjected to vibration. The evidence report recognizes how critical the risks associated with inadequate HCI are and also acknowledges the absence of a structured process associated with human-centered design.

The evidence report acknowledges the research gaps in the Human Research Roadmap (see Box 5; NASA, 2014k), and the next iteration could explore those in greater depth.

BOX 5

Additional Research Gaps Identified in the Human Research Roadmap: Human-Computer Interaction. SHFE-HCI-01: What are the effects of vibration and acceleration on crew task performance and how can those effects be mitigated? SHFE-HCI-02: We need to understand (more...)

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

While the evidence report draws upon findings and experiences in a wide array of related application domains, the risk of inadequate HCI can be better understood and mitigated by exploring research and lessons from emerging domains that involve human-device interactions that are comparable to those of space missions. Such interactions include high-stress environments in which the time for processing and responding to large amounts of information is very short, available solutions or feedback is limited, the risk of failure due to improper processing of information is high, and the trust that humans have in these devices is critical (Moreno et al., 2013; Xie et al., 2014). These include

• immersive environments such as those found in gaming (Lalor et al., 2005; Nahrstedt et al., 2011);
• augmented reality (Seo and Lee, 2013);
• firefighting (Rosengren et al., 2014);
• high-speed racing (Grand Prix, NASCAR) (Lee, 2004);
• war games (e.g., situational awareness-based effective responses) (Bainbridge, 2010);
• health care (e.g., checklists in surgery, organ transplants) (Gawande, 2013; Pronovost and Vohr, 2010);
• expedition-type work, e.g., NEEMO (NASA Extreme Environment Mission Operations) (NASA, 2014d), Antarctica expeditions; and
• long-duration submarine trips (Kimhi et al., 2011).

Touch-based interfaces are significantly transforming human-device interactions in everyday life so much so that a young child given a paper book is dismayed that the contents do not change when the page is “swiped.” The impact of the rapid proliferation of touch-based interfaces on the cognitive perceptions and responses of mission personnel who have to simultaneously deal with physical/mechanical devices during the mission must be studied in order to mitigate the risks associated with dealing with different modes of human-device interaction. An example of such a study is the one being conducted by Honeywell for the Federal Aviation Administration that is investigating the efficiency of touchscreen devices vis-à-vis pilot workload, accuracy, and fatigue when operating in turbulent environments (Bellamy, 2013). Montuschi and colleagues (2014) discuss recent developments in HCI in the context of emerging natural user interfaces such as gesture, body poses, speech, and gaze.

Methodologically, the systems engineering concepts of failure-modes and effects analysis (McDermott et al., 2008) may provide answers for many of the critical questions (including the lack of standards for space missions) raised in the closing paragraph of the discussion on “Contributing Factor 2, Informational Resources/Support,” which address the critical issue of the numerous “unknown unknowns” associated with long-term space missions (Holden et al., 2013, p. 16).

Does the Evidence Report Address Relevant Interactions Among Risks?

The report makes an excellent case for a systems or integrated approach to addressing the interactions among the risks associated with HCI, HARI, critical task design, and training. It specifically recognizes the relative importance of efficiency, effectiveness, and complexity associated with the various interactions.

What Is the Overall Readability and Quality?

The report is authored well from the points of view of readability and quality.

Is the Breadth of the Cited Literature Sufficient?

The breadth of the cited literature is sufficient from a classical HCI perspective. As noted above, the report could benefit from citing more recent research on HCI issues in the area of gaming (see, for example, Tedjokusumo et al., 2010) and other high-stress domains such as firefighting and car racing (e.g., Grand Prix, NASCAR), where the human-device interaction is critical to the successful execution of the task.

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

Yes. However, the authorship could be expanded to bring in the newer perspectives and understandings associated with emerging technologies in human-device interaction.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

Yes. The report uses the “quality-of-evidence” criteria specified in Recommendation No. 1 of the 2008 report (IOM, 2008, p. 11) in assessing the risk of inadequate HCI. It also recognizes the inadequacy of evidence at levels I and II. The report addresses the need to discuss the issue of “too much information” on performance (IOM, 2008, p. 83) by discussing the importance of information granularity in enhancing HCI. Further, the report addresses the cross-cutting issues of manual controls, displays, fatigue, and spatial disorientation (see IOM, 2008, p. 82) in detail through the specific sections highlighting each of these issues using the framework of “core contributing factors” in the context of HCI.

Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions?

This evidence report provides a number of descriptions of anecdotal reports and examples from experiences on the Shuttle, ISS, and Constellation programs. These examples provide the reader with the starting points for understanding the context, but more could be done to identify and analyze the key research gaps that are identified by those accounts. In several places, the report offers broad generalizations that could be followed up by specifics on the research gaps. An example is the discussion on repetitive stress/strain injuries on page 7, which could benefit from more specifics on what relevant research has been done in this area (including the substantial literature beyond space-specific research) and the specific gaps that remain to be filled. The committee recognizes that many areas of research are covered in this evidence report, and that priorities will need to be determined regarding where to focus efforts to identify research gaps.

Does the Evidence Report Make the Case for the Research Gaps Presented?

The report identifies a number of research gaps but, as noted above, the case would have been stronger if a deliberate effort had been made to tie each of the gaps to specific human health and safety risks. Updated versions of this report should explicitly include those listed in the Human Research Roadmap summary (see Box 6; NASA, 2014h), under the “Gaps” section to enable more efficient cross checks across NASA documents.

BOX 6

Additional Research Gaps Identified in the Human Research Roadmap: Risk of Incompatible Vehicle/Habitat Design. SHFE-HAB-03: We need to understand how new aspects of the natural and induced environment (e.g., vehicle/habitat architecture, acoustics, vibration, (more...)

While all the information presented made sense individually, it did not necessarily provide a comprehensive framework encompassing habitat design and habitability factors. Without such an overarching vantage point, the list of covered topics appears randomly selected and incomplete. A short paragraph at the beginning of the evidence report might be helpful to summarize how this report fits within the bigger picture of space habitat design concerns and to explain more clearly what is addressed and what is not.

The evidence report places considerable emphasis on anthropometry and on noise, without explicit discussion of the consequences of inadequate design in these areas; these limitations of the report are discussed on the next page. The other research gaps (e.g., visual environment, vehicle volume and layout) are adequately addressed in this evidence review, although, as noted above, these are each broad areas of research with many unknowns for long-duration spaceflights.

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

As this evidence report covers a broad range of both basic and applied research, a number of research gaps could be considered, but other than those noted in the section above, no specific issues were identified by the committee.

Does the Evidence Report Address Relevant Interactions Among Risks?

Although human system integration with the habitat/vehicle is discussed throughout the report, there could be further efforts to address the relevant interactions among the various risks described in this evidence report and also among the risks described in other evidence reports.

Interactions among risks that are not described include

• interactions where changes in human health (such as changes in muscle strength and bone stability caused by vibration and G-forces, as well as visual acuity) could occur on long-duration flights and affect the crew's interactions with the vehicle and habitat;
• interactions of acceleration and vibration (see, e.g., Griffin, 2001); and
• interactions that could affect behavioral health and performance (such as lighting issues related to fatigue and circadian rhythms).

The section on anthropometry discusses a number of interactions of fluctuating human body dimensions with suit design, fit, and seating/restraints/equipment (along with other vehicle and habitat design issues).

What Is the Overall Readability and Quality?

The executive summary and risk overview are succinct and provide a well-written summary of the evidence. The body of the evidence report could have been greatly strengthened by paying greater attention to synthesizing and analyzing the individual experiences and examples so that specific risks could be identified more clearly. For example, in the section on restraints, an analysis of the examples could have included the key design features that could potentially cause injury or health risks, rather than making the general statements that “restraints that are overly complex, difficult, or time-consuming to set up or get in/out of, will not be used by the crew” (Whitmore et al., 2013, p. 26).

Throughout the report there is a need for increased attention to health risks and how the various design features affect crew health, rather than paying attention to more general operations issues. Broad descriptions, such as introduction of the extravehicular mobility units (p. 11), the issues with the closure mechanisms on rack doors (p. 13), the lighting analysis (pp. 17–18), and the introduction of the equipment section (p. 26) could be shortened and edited to focus on health risks. The descriptions of the ISS issues regarding accessibility and stowage (p. 31) and other issues are important, but need to be tied to health and safety risks. However, the gaps need to be focused on future spacecraft and further exploration missions, rather than on the specific design issues of the ISS or of Constellation.

Furthermore, the report would have been enhanced by professional editing as well as careful consideration of which figures and photographs were needed to make the point regarding risks. The organization and readability of the report could also have been strengthened by using a single template for all of the risk factor sections, so as to provide a consistent style and format between factor descriptions.

Is the Breadth of the Cited Literature Sufficient?

Given the breadth of research covered by this evidence report, the authors had to be selective about what literature was cited. However, in many places throughout the report the literature that is cited relies too heavily on NASA studies and needs to be expanded into the broader literature. This is particularly true for areas where the risks have strong terrestrial analogs such as noise, anthropometry, and visibility. For example, the current literature on short- and long-term noise effects on cognition could be cited (e.g., Szalma and Hancock, 2011). The evidence report notes that most of the information comes from observations and case studies and summaries of subjective experience data from spaceflights and training. Efforts to provide more research-based examples, where available, are encouraged.

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

As with all reports that are written by authors working for a single federal agency, the report could benefit from the involvement of independent, multidisciplinary researchers, engineers, and clinicians. For example, input from the biomechanics discipline, particularly on the subjects of impact, vibration tolerance, and sustained acceleration, is needed, and those sections could be augmented by accessing the vast expertise of the defense laboratories, especially at Wright-Patterson Air Force Base and Fort Rucker. The coverage of the noise problem is limited and dated.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

In the 2008 report, the groupings of issues related to human factors were clustered in an evidence book, Lack of Human-Centered Design, Chapter 23, which addressed three risks related to human factors: (1) inadequate information, (2) poor human factor design, and (3) poor task design. There were no previous recommendations relevant to the specific topics on vehicle and habitat design.

Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions?

The evidence book makes an excellent case for developing a better understanding of the work that is and will be performed by astronauts, but there are important omissions. Among the most salient omissions of the evidence report is the recognition of the enormous responsibilities that ground personnel face in preparing realistic schedules, accurate procedures, and optimum presentation of information to guide task performance in space.

The report discusses inadequate task design in general terms and, to illustrate potential consequences, uses examples drawn almost exclusively from aviation. It cites evidence from the ISS crew comments database to support the claim that poorly prepared and erroneous procedures are a persistent problem for the ISS crew. No concrete examples are provided, although many are available from recent studies (e.g., Stuster, 2010) and others have been provided anecdotally by former ISS crew members. For example, a former ISS commander described to the committee several problems with procedures, including inconsistencies in format, inconsistencies in nomenclature used for equipment and materials, and having to jump from one set of procedures to another in order to perform a task (Lopez-Alegria, 2014).

The evidence report defines the risk of inadequate task design in somewhat ambiguous terms as concerned with tasks, schedules, and procedures, with most tasks performed using human-computer interfaces. The executive summary correctly describes three contributing factors to the risk: (1) operational tempo and workload, (2) procedural guidance, and (3) technical/procedural knowledge. Unfortunately, the emphasis is consistently on the astronaut's ability to accommodate to the procedures and schedules, rather than on the developers and schedulers performing their jobs appropriately. In addition, the report focuses on cognitive elements (or effects?) of task performance and only occasionally acknowledges the possibility of physical limitations and risks. Identifying gaps as either physical or psychological/cognitive, while recognizing that interactions are almost certain to occur, might improve the focus of the evidence book. Alternatively, excluding motor tasks that do not include a high cognitive component (e.g., physical exertion during EVA) might be appropriate. With the exception of the June 1997 Mir incident, there is no review of mission risk related to operational tempo or workload for spaceflight.

The Human Factors Analysis and Classification System (HFACS) differentiates between errors (unintentional behaviors) and violations (willful disregard of the rules and regulations) (Shappell and Wiegmann, 2000). In terms of this HFACS distinction, the evidence report discusses only errors, with no review of violations. A summary of unsafe acts using the HFACS would be useful. In particular:

• What are the incidence rates of errors versus violations?
• When violations occur, why did crewmembers execute a decision contrary to flight rules?

Does the Evidence Report Make the Case for the Research Gaps Presented?

The evidence report provides justification for all of the research gaps identified. However, the data provided are almost exclusively summative, based on crew report. Thus, a large gap exists in formative evaluation that would identify when tasks are too stressful for crew members. The evidence report correctly points out that delays between training and knowledge utilization will cause an increase in workload as crew members attempt to access required knowledge. Automation is both an aid and a potential crutch, especially when knowledge and understanding of automated systems become compromised.

Updated versions of this report should use subheadings for categories of research gaps, including those listed in the Human Research Roadmap summary (see Box 7) under the “Gaps” section, to enable more efficient cross checks across NASA documents.

BOX 7

Additional Research Gaps Identified in the Human Research Roadmap: Critical Task Design. SHFE-TASK-01: How can workload measures and tools be developed to unobtrusively monitor and trend workload throughout the mission design and verification cycle in (more...)

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

The role that task analyses should play in the human-centered design process is described thoroughly in the evidence report. However, the report fails to include, in its list of research gaps, a discussion of the need for task analyses of the work to be performed on expedition-class missions. The absence of task analyses for ISS operations and for future long-duration expeditions represents a major oversight that could be corrected easily and soon. A July 2014 NASA research announcement includes a study of the general abilities required for planetary expeditions, which almost certainly will begin with a systematic analysis of expedition tasks; an analysis of the work to be performed on those expeditions is the first step in the Human Factors Method, and will be necessary to identify the skills and abilities required to perform the work (NASA, 2014c). As knowledge is gained from this work, it can be incorporated in future revisions of the evidence report.

The report would benefit from the addition of information about the type of job aids that have been provided to the ISS crew and about the crew's response to the various formats: longer videos of full procedures, hyperlinks to shorter videos showing specific tasks, paper copy, online only, or other formats.

Another gap is the need for research to identify the optimum procedure format for different types of tasks. It is understandable that this was overlooked because there have been no task analyses conducted to identify categories of similar tasks. It should be noted that one of the fundamental products of a properly conducted task analysis is the identification of a job incumbent's information requirements at each step in the work to be performed. Knowledge of information requirements is directly relevant to the design of appropriate job aids.

The report should also address the effects of time and scheduling as they relate to the risk of critical task design. A persistent scheduling problem aboard the ISS is the failure to allocate sufficient time to prepare for tasks, assemble materials, and stow equipment properly after task completion. The on-orbit constraints result in crew members running behind schedule almost continuously, which is the single greatest contributor to stress identified by participants in a study of confidential astronaut journals (Stuster, 2010). Involving crew members in developing schedules and a review of new procedures by former ISS crew members are two options that could be explored. Further gaps remain regarding the effect of delayed communications with the ground on task performance during interplanetary expeditions; more needs to be learned about how to write procedures to support autonomous operations.

The list of research gaps is generally thoughtful, with the discussion of human-in-the-loop evaluations being particularly appropriate, especially because spacecraft design will continue to evolve and include greater automation. A few additional gaps should be considered:

• Trainability as an aspect of task design: The evidence report acknowledges task design. Perhaps, a reciprocal gap should be addressed.
• Acknowledgment of cognitive changes suggests that the concept known as “universal design” might be worthy of studying for applicability to spacecraft procedures.
• Function allocation, the second step in the Human Factors Method, would logically follow the carrying out of the task analysis proposed above. Determining which tasks are appropriately performed by humans, and which are more appropriately performed by computers or robots, is a rational process that must be informed by an understanding of task requirements, human capabilities and limitations, and operational conditions (mostly constraints). Although the general rule is to automate when possible, a contra intuitive approach might be warranted on long-duration space expeditions where meaningful work might be in short supply. Roald Amundsen said that boredom is the worst enemy of the polar explorer, and most expedition leaders during the heroic era made sure that there would be plenty of tasks to keep the crew engaged (Stuster, 2011).
• Meaningfulness of work: The evidence report refers to “nuisance operations” as tasks to be avoided, but work that is considered to be trivial, repetitive, or a nuisance under normal conditions might prove to be rewarding on long-duration expeditions. For example, Fridtjof Nansen's engineer disassembled and reassembled the Fram's steam engine three times while locked in the ice during the Norwegian Polar Expedition of 1893–1896; it was a repetitive but meaningful task that kept the engineer and his assistant fully engaged (Stuster, 2011).
• Crew autonomy and guided input: The evidence report states that “automated support for these planning tasks should allow crewmembers to manage daily tasks and ensure that these tasks are performed appropriately when ground support is unavailable” (Sándor et al., 2013, p. 8). Long-duration missions beyond low Earth orbit will likely require significantly more crew autonomy and “guided” input (e.g., guided by expert systems on board). Input from the human factors engineers who are contributing to the design of self-driving automobiles may provide additional input into this discussion.
• Physical workload: The evidence report emphasizes cognitive workload, which is appropriate. However, physical workload also should be addressed, especially for planetary expeditions that will presumably involve surface operations and will be accompanied by muscle atrophy and perhaps other negative environmental effects, such as vision degradation.
• Automated job aids to correct operator errors: Research on task design should include consideration of automated procedures that might be developed that would have the capability of detecting errors and suggesting mitigating actions, in the same manner that a global positioning system (GPS)-guided trip computer can detect a wrong turn and then provide turning directions to regain the route.
• Design approaches: Automated defibrillators (or other similarly designed devices), which are now available in almost all public places and designed to be used by anyone without prior training, might offer a model for the design of procedures to be followed under emergency conditions.
• Workload measurement techniques: Self-reports of workload may be unreliable so more objective measures are needed.

Does the Evidence Report Address Relevant Interactions Among Risks?

Links are provided in this evidence report to the NASA evidence reports on inadequate HCI (Holden et al., 2013), training deficiencies (Barshi, 2012), sleep loss and circadian rhythm (Whitmore et al., 2009), and behavioral conditions and psychiatric disorders (Slack et al., 2009). Additional relevant interactions include vision impairment (Alexander et al., 2012) and those related to physical deconditioning. Also, problem solving in an autonomous environment will be required on future expeditions and should be studied.

What Is the Overall Readability and Quality?

The evidence report starts off with excellent justifications for applying the HFACS to the design of procedures and equipment. The style becomes uneven beginning around page 12 and occasionally lapses into jargon-filled sentences that are difficult to follow. Clarification of the jargon would be helpful. The committee recognizes that this is a technical subject. Overall the report is well written.

Is the Breadth of the Cited Literature Sufficient?

Many references are old, which is not necessarily a negative, but there are recent studies that could be cited concerning problems with procedures and schedules on the ISS and about analogous conditions (e.g., medical procedures). Also, several of the references could not be found; references should be accessible and should be cited in a format that allows the reader to easily locate and read them (e.g., URL followed by “accessed on” date).

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

The expertise of the authors is sufficient, although additional perspectives are always valuable. Additional areas of expertise could include task design for health care and input on the importance of effective hand-off procedures. Videography or graphic design expertise is also relevant, especially after the research has been conducted to identify the optimum presentation modes, or formats, for displaying procedures.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

In the 2008 IOM report, the discussion regarding task design was part of the review of a broad chapter, Chapter 23 in Lack of Human-Centered Design, that addressed three risks: (1) inadequate information, (2) poor human factor design, and (3) poor task design. The 2008 report recommended

• the use of more recent examples of task design problems, which appears to remain an issue;
• further discussion on the “gaps associated with determining the impact on efficiency of an error-reducing task design” (IOM, 2008, p. 85);
• discussion of relevant interactions, including how “the design of tasks that necessitate EVA must consider the physical and other limitations created by the EVA suit” (IOM, 2008, p. 85).

Further discussion of each topic listed in the 2008 report is needed.

Does the Evidence Report Provide Sufficient Evidence, as Well as Sufficient Risk Context, That the Risk Is of Concern for Long-Term Space Missions?

This evidence report discusses inadequate training in general terms and cites evidence from the ISS crew comments database to support the claim that training for the ISS missions is long and stressful, and often provides inadequate preparation. No concrete examples were provided, but the 2010 report documenting the Journals Flight Experiment includes several examples in which astronauts claim that the training they received only slightly resembled the task they were required to perform in orbit and that they were required to familiarize themselves with tasks during off-duty time just prior to performing them (Stuster, 2010). That is, evidence suggests that, among other problems, training can be misaligned and missing, despite crew members having devoted 3 years to preparing for a 6-month expedition. The report recognizes the additional risks that could result from inadequate training for long-duration expeditions and from communication lag times that will constrain customary support options.

The evidence report makes a reasonable case for the risk context and its importance for complex tasks that require high levels of situational awareness. The committee expected that the report would have included examples from previous spaceflight experience, but the evidence presented in the review draws largely from analog environments, especially military and commercial aviation. A relatively large database of accidents that have been documented in actual spaceflight conditions exists and could provide more directly relevant insights. For example, Zimmerman (1998) outlines several cases in the development of space stations, and Shayler (2000) does a commendable job of reviewing accidents in the U.S. and Soviet space programs. The National Research Council report Preparing for the High Frontier: The Role and Training of NASA Astronauts in the Post-Space Shuttle Era also offers useful information (NRC, 2011). The situations surrounding STS-37 (landing winds) and STS-87 (Spartan satellite grapple) included notable anomalies and the problems that emerged in the Spartan experience, in particular, were largely attributed to inadequate training. The evidence report needs additional examples taken directly from the field of spaceflight to illustrate the risks of inadequate training.

Does the Evidence Report Make the Case for the Research Gaps Presented?

Justification is provided for the three research gaps that are identified:

• Lack of modeling and simulation platforms that can be leveraged for training on emerging technology
• Inadequate or unavailable training programs
• Inconsistencies with training, individual attributes, and mission demands

The statements concerning gaps are vague and incomplete. The case made for task training is reasonably compelling, but the cases for the research gaps are less clear. In particular, an interplanetary space expedition will require launching a crew and then probably transferring that crew to another space vehicle before they depart on the outbound transit to their destination. Information support from ground controllers will be an important part of task performance during the early phase of the outbound transit, but at some point communication lags will become too long for immediate feedback from ground controllers rendering real-time voice and text communication impossible. What will be the algorithm to switch from direct interaction to autonomous crew capability? At what point in the transit will this be considered? This condition does not exist in spaceflight currently but could be simulated on the ISS and is an unexplored gap for which a research platform already exists. Also, one example is provided in the report to make a strong case for understanding the utility of “just-in-time” training, especially when experimental schedules do not necessarily align with crew rotation schedules.

In addition to expanding the research gap section, in general, authors of updated report versions should consider structuring the section around categories of research gaps, including those listed in the Human Research Roadmap summary (see Box 8), as suggested throughout this letter report.

BOX 8

Additional Research Gaps Identified in the Human Research Roadmap: Performance Errors Due to Training Deficiencies. SHFE-TRAIN-01: We do not know which validated objective measures of operator proficiency and of training effectiveness should be used for (more...)

Are There Any Additional Gaps in Knowledge or Areas of Fundamental Research That Should Be Considered to Enhance the Basic Understanding of This Specific Risk?

Various types of training are discussed in the evidence report, including pre-mission training, refresher training, and just-in-time training (by which the author meant contingency training under emergency conditions). However, in general, the discussions of training are abstract, that is, without reference to actual tasks or specific knowledge, skills, and abilities that might be required for mission success—or to survive an emergency. Like the companion report on inadequate critical task design (i.e., procedures) (Sándor et al., 2013), this review of training risks fails to include, in its list of research gaps, the need for task analyses of the work to be performed on expedition-class missions. The absence of task analyses for the ISS operations and for future long-duration expeditions handicaps understanding of associated risks and limits discussion to vague and unanchored generalities. What skills and knowledge will be necessary for successful performance on a 6-month asteroid mission or a 3-year mission to Mars? In considering such missions, the first step is to identify the tasks that will be performed and the specific and general abilities that will be required of the crew. One of the products of a properly conducted task analysis is identification of the abilities needed to perform the work successfully. Knowing what abilities are needed leads directly to the design and delivery of appropriate training. On page 11 the report states, “Research is required to develop appropriate generalizable, skill-based training,” but this research requirement was not included in the list of gaps presented.

As mentioned above in the review of the evidence report on critical task design, NASA's recent research announcement (July 2014) includes a study on the general abilities required for planetary expeditions (NASA, 2014c). NASA issued a similar research announcement in 2013 related to “novel adaptive and context-sensitive refresher training and/or just-in-time training methods and tools for autonomous crews performing tasks such as robotic or maintenance activities” (NASA, 2013b). The resulting research will likely feed into future iterations of this evidence report by identifying the work to be performed on these expeditions and the accompanying needs for training in specific skills and abilities. One issue that should be examined is increasing the efficiency of training.

Reviewing other areas in which training is critical for successful task performance (e.g., surgery) might lead to the identification of additional research gaps (see discussion below on the breadth of the cited literature). Some of the work on failure analysis in the medical field is patterned on work in commercial and military aviation, which might prove to be circular, but the manner in which this same body of material has been adapted to the medical environment might provide transferrable insights. Also, the evidence report provides little in terms of specific types of training, frequency of training, and adequacy of refresher experiences for astronauts.

Another area that needs further discussion is the assessment of the skills and experience that astronauts and their trainers bring to a mission and the training for that mission. A former shuttle astronaut and the ISS commander who spoke to the committee described NASA's approach to training as being task-focused for shuttle missions and skill-focused for the ISS expeditions (Lopez-Alegria, 2014); that is, U.S. ISS crew are expected to be experts in everything, which might be an unreasonable approach and certainly contributes to the training burden about which astronauts complain. Selecting some personnel with specific skills and then providing task-focused training and job aids and other support during task performance might be more effective. Furthermore, the factors that affect crew members' trust in the training and trainer need to be explored. The workshop speaker noted the challenges of receiving training from novice instructors with no spaceflight experience. Competence and spaceflight experience are among the factors that could be examined.

Does the Evidence Report Address Relevant Interactions Among Risks?

The evidence report mentions only “inadequate task design” as a related risk. Beyond task design, the most salient training-related risk is inadequate personnel selection, which involves both the initial criteria for selection and the criteria used to select a specific crew composition. The necessary skill sets and experience are both areas that should be examined. For example, the report mentions on several occasions that rote memorization and training for infrequent task performance are brittle, erode over time, and are not generalizable. In addition, all members of the crew also must be demonstrably adept at getting along with others in isolation and confinement. Getting along with others is a skill that can be trained, but it might be more effective to select individuals who have a history of exhibiting key traits and abilities—such as perseverance, fidelity, and affability—in addition to specific technical expertise. In other words, many of the risks associated with inadequate training could be mitigated by proper personnel selection.

Other risks that might interact with the risk of inadequate training include those related to human-robotics interaction (Marquez et al., 2013), circadian rhythm (Whitmore et al., 2009), vestibular function (Paloski et al., 2008), and possibly vision (Alexander et al., 2012). The principles of universal design could also be explored in considering the design of tasks and the associated training. A simple example might be the use of scalable fonts in displayed instructions.

What Is the Overall Readability and Quality?

The evidence report is well-written overall and easily readable, but it could be improved with greater specificity.

Is the Breadth of the Cited Literature Sufficient?

The emphasis in the material cited is on aviation-related research, which is relevant and probably unavoidable. However, the committee also notes that many of the references are not easily accessible and that there seems to be a relative dearth of citations to the peer-reviewed literature. A relevant body of literature that could be included is the medical literature concerning failure avoidance, training, and team support. In the past 20 years, the health care field has focused on reducing patient care errors and has developed a number of practices, and this body of literature could be explored for relevance to spaceflight training (see Box 9).

BOX 9

Examples of Analogous Situations and Training Paradigms. Mentors and trainers: Explanation of procedure execution errors on page 1 of the evidence report seems roughly analogous to the role played by faculty surgeons during surgical training. After medical (more...)

Is the Expertise of the Authors Sufficient to Fully Cover the Scope of the Given Risk? Is Input from Additional Disciplines Needed?

This evidence report has a single author who has extensive research expertise in addressing the cognitive issues involved in the skilled performance of astronauts, pilots, and flight/air traffic controllers. This research focus is appropriate for the risk. However, this evidence report, as with each of the evidence reports on a broad topic, would benefit from additional perspectives. This input, with additional expertise on task analysis and human factors engineering included, should be helpful in identifying the risks of inadequate training for expedition-class missions.

Has the Evidence Report Addressed Previous Recommendations Made by the IOM in the 2008 Letter Report?

In the 2008 IOM report, the discussion regarding training was part of the review of a broad chapter that focused primarily on teams (i.e., Chapter 15 “Performance Errors Due to Poor Team Cohesion and Performance, Inadequate Selection/Team Composition, Inadequate Training, and Poor Psychosocial Adaptation”). The 2008 report states, “The section on training (pp. 12–13) mentions previous studies that show different kinds of training have an impact on ‘performance.’ But the issues are treated in a very general way. What kind of training? For what kinds of individuals? Conducted by whom? How was performance measured?” (IOM, 2008, p. 66). The questions asked in the 2008 report have been addressed for the most part by pulling out the training issues as a separate evidence report. The 2013 evidence report provides significantly expanded coverage on the risk of inadequate training and highlights the research needed on types of training and measurement of its impact, although, as noted in this review, greater specificity is needed, where feasible.