U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Academies of Sciences, Engineering, and Medicine; Division on Earth and Life Studies; Board on Environmental Studies and Toxicology; Committee on Endocrine-Related Low-Dose Toxicity. Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals. Washington (DC): National Academies Press (US); 2017 Jul 18.

Cover of Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals

Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals.

Show details

5Lessons Learned and Reflections on the Statement of Task

During the course of its deliberations, the committee developed a general strategy to collect and evaluate information regarding low-dose effects (see Chapter 2). The committee then applied two aspects of its strategy (investigation and analysis) to consider whether exposure to phthalates (see Chapter 3) or polybrominated diphenyl ethers (PBDEs; see Chapter 4) was associated with low-dose endocrine effects. The committee principally used systematic-review methods to complete and document these aspects. This decision was driven partly by the statement of task, which specified that the committee should complete systematic reviews of human and animal toxicology data for two or more chemicals that affect an endocrine hormone system. The committee's reliance on systematic-review methods is also consistent with recommendations made by previous National Academies committees (NRC 2014a,b).

The focus of the report on systematic review does not imply that it is the only evaluation tool needed to address all questions about low-dose toxicity. As described in Chapter 2, other options for investigation and analysis are available for use instead of or in conjunction with systematic review. Selection of the approaches will depend on the nature of the question and the potential health, social, and economic implications the answer will have. The committee strategy also describes potential options for future actions. For example, one potential action could be an update of an existing toxicity assessment. Although this type of action is consistent with the iterative strategy developed by the committee, completion of the additional steps needed to support this action were not pursued because the committee was not asked to complete risk assessments for any individual chemicals or classes of chemicals.

The purpose of this chapter is to provide additional discussion of lessons learned as they relate to committee efforts to address the statement of task and related issues concerning low-dose effects of endocrine active chemicals (EACs). It also provides lessons learned from performing the systematic reviews and integrating the human and animal evidence.

COMMITTEE STRATEGY

Development of a generic strategy for evaluating evidence of low-dose effects in Chapter 2 was deliberate and reflected the committee's desire to provide a framework that could be applied to many agents of concern regardless of the at-risk population, toxicity end point, or mechanism. Application of this generic strategy into one that meets the US Environmental Protection Agency's (EPA's) need to evaluate low-dose endocrine effects will require a combination of scientific and policy decisions on the agency's part. For example, implementation of an active surveillance program will likely necessitate that EPA identify specific EACs, dose ranges, populations, and end points to be monitored. The agency might need to perform multiple scoping exercises that lead to the development of specific questions. The problem formulation activities to address these questions will ultimately result in investigation and analyses tailored to address issues related to low-dose endocrine effects. Box 5-1 provides some examples of targeted analyses the committee performed to answer questions that arose during the course of working on the case example of phthalates in Chapter 3.

Box Icon

BOX 5-1

Examples of Targeted Analyses of Existing Data Performed by the Committee.

CHARACTERIZING ADVERSITY

The committee discussed whether the effects of exposure to phthalate or PBDE exposure are adverse. The issue of adversity has been the subject of debate, and a number of definitions have been proposed (Kerlin et al. 2016). There have been recent attempts to develop clear criteria to evaluate whether an effect seen in nonclinical toxicology studies is adverse (Kerlin et al. 2016; Palazzi et al. 2016; Pandiri et al. 2017). The committee considered that guidance when drafting its definition of adverse.1

A determination of whether an effect is adverse requires expert judgment and should be based on evaluation of the effect in both the animal and human literature. For example, in the case of phthalate-induced effects, hypospadias was considered adverse because it represents a morphologic effect that might affect reproductive performance or behavior (Bubanj et al. 2004; Schlomer et al. 2014). Whether changes in either fetal testosterone concentrations or anogenital distance (AGD) are adverse has been the subject of discussion in the scientific community (Howdeshell et al. 2017). Several mechanisms, including androgen receptor antagonism and inhibition of androgen synthesis enzymes, can contribute to phthalate-induced reductions in fetal testosterone production (Howdeshell et al. 2017). In animal models of phthalate toxicity, multiple studies have found no effect on apical reproductive end points when treatment-related reduction in fetal testosterone was less than 40%, suggesting a point below which apical effects in studies in rodents are not observed (Gray et al. 2016). Likewise, a reduction in AGD is a biomarker of reduced in utero androgen concentrations (Thankamony et al. 2016) and might exhibit a similar threshold before changes in apical reproductive end points are seen. Several studies have reported that newborns born with hypospadias and cryptorchidism have shorter AGDs than infants without abnormalities (Hsieh et al. 2012; Jain and Singal 2013; Thankamony et al. 2014). In addition, several studies in adult males have reported that men who have reduced fertility—including lower sperm concentration, count, and motility—have shortened AGDs (Eisenberg et al. 2011, 2012; Mendiola et al. 2011, 2015; Eisenberg and Lipshultz 2015). At this time, the degree of AGD shortening necessary to observe apical end points remains unknown. It is therefore unclear that changes in testosterone or AGD in the absence of other apical reproductive end points would meet the first part of the committee's definition of an adverse effect—“a biological change in an organism that results in an impairment of functional capacity.”

The committee also considered whether phthalate-induced changes in fetal testosterone or AGD might increase the susceptibility of an organism to other influences. Sufficient androgen activity is required for proper male reproductive-tract development, and this activity is a consequence of multiple inputs at the molecular level. In the rat model, data have shown that a phthalate-induced reduction in fetal testosterone made the animal more susceptible to the effects of another chemical (linuron) that also targeted the androgen signaling system, although the reduction by itself was insufficient to produce an adverse reproductive effect (Hotchkiss et al. 2004). On the basis of those data, a phthalate-related reduction in fetal testosterone or reduction in AGD would be considered an adverse effect because they meet another part of the definition: a biological change that results in “an increase in susceptibility to other influences.”

The committee considered the PBDE-related effects on cognitive function to be adverse. The committee's systematic review supports the findings made by Lam et al. (in press) of an inverse association between developmental PBDE exposure and effects on IQ in children. Lam et al. reported a decrease of 3.7 IQ points per 10-fold increase in serum PBDE concentration. It is important to differentiate between effects seen at the individual and population levels (NRC 2009). First, the magnitude of response to PBDEs and other EACs can vary within a population because some individuals might be more affected and others less so by the same exposure (NRC 2009). At the individual level, large changes in IQ (e.g., 10 points) are considered adverse. Smaller changes on an individual level, such as a shift of 3-4 points, might not be associated with a functional impairment in cognitive function and are often within the range of variability when an individual is retested on an IQ test (Watkins and Smith 2013). However, given widespread exposure across the population, small changes in IQ could shift the entire population distribution in the direction of decreased function, which has quantifiable consequences integrated over the population, such as increased fraction of population with very low IQ or reduced aggregate economic output (Axelrad et al. 2007; Bellinger 2012). Therefore, PBDE-related reduction in IQ would be considered an adverse effect because they meet the first part of the definition: “a biological change in an organism that results in an impairment of functional capacity.”

The committee's reasoning as to whether the changes produced by phthalate or PBDE exposure are adverse is likely to be applicable to other chemicals and end points. Some lessons learned from the committee's deliberation include the following:

  • There is usually agreement regarding adversity of more severe outcomes because they fit into the first part of the definition concerning “impairment of functional capacity.”
  • Changes in continuous end points, such as hormone levels or biomarkers, might not lead to demonstrable “impairment of functional capacity” at the individual level. However, such effects might fit into the second or third parts of the definition of adversity relating to “a decrease in the capacity to compensate for stress” or “an increase in susceptibility to other influences.” Decisions to label such changes as “adverse” would be strengthened by experimental data that demonstrate reduced compensatory capacity or increased susceptibility. In the absence of such data, scientific judgment (e.g., analogy to other end points) and policy considerations (e.g., the severity and magnitude of the possible effect) might be involved.
  • The small magnitude of change of a continuous end point might not have demonstrable “impairment of functional capacity” at the individual level but could have quantifiable consequences on functional capacity when considered over the entire population. Such shifts in the population distribution might result in more individuals in the tail of the distribution or reduced aggregate function over the population. Therefore, when evaluating end points with respect to the first part of the definition, consideration needs to be given to effects over the population.

REFLECTIONS AND LESSONS LEARNED FROM THE SYSTEMATIC REVIEWS

Selection of Example Chemicals

The specification that the systematic reviews be performed on chemicals that “act through an endocrine-mediated pathway” and “that affect the estrogen, androgen, or perhaps other endocrine systems” led to extensive discussions about how stringent the committee should be about using mechanistic data to guide chemical selection. The committee's goal was to identify candidate chemicals that were presumed to be associated with endocrine effects. Although that approach seems straightforward, mechanisms are poorly understood for some of the candidate EACs that were initially considered. The committee eventually selected two chemical classes (phthalates and PBDEs) that had varying amounts of mechanistic information to illustrate different approaches that EPA might need to use when assessing low-dose endocrine effects. Other factors that influenced chemical selection were whether relevant animal and human data were available and the types of end points associated with each candidate chemical.

The committee also recognized that it might be advantageous for EPA to build on existing systematic reviews that are published in the peer-reviewed literature. During the course of its discussions, the committee became aware of a systematic review being conducted by Lam et al. (in press) on developmental exposures to PBDEs and human neurodevelopment. The willingness of Lam and coworkers to share an early draft of their study provided the committee with a unique opportunity for the committee to meet its objective of demonstrating how to build on a published systematic review. The committee remains indebted to Lam and coworkers for their generosity.

Methods

The committee initially performed a series of scoping exercises—conducting informal literature reviews and hosting a workshop—that helped define chemicals of interest, populations of concern, exposure windows, health end points, and other factors that led to the development of the specific research question and the appropriate PECO (Population, Exposure, Comparator, and Outcome) statements that guided the committee's investigation (Higgins and Green 2011; IOM 2011). Broadly stated, the phthalate research question asked whether in utero exposure to phthalates was associated with reproductive effects (as assessed by changes in fetal testosterone during gestation or at delivery, AGD, or the incidence of hypospadias) in male nonhuman mammals or humans. The PBDE research question asked whether developmental exposure to PBDEs was associated with neurobehavioral effects (as assessed by changes in learning, memory, attention, or response inhibition) in nonhuman mammals or humans. The systematic reviews performed by the committee were therefore hypothesis driven and designed to answer a set of focused questions. Although the project was intended to address issues surrounding low-dose effects, the committee did not constrain exposure to low dose in the PECO statement and therefore did not use it as an eligibility criterion. That decision was a deliberate choice because the committee did not want to a priori constrain the investigation by any preconceived notion of low dose. Instead, the committee chose to address whether effects occurred at a low dose as a separate subsequent step in the process.

The committee discussed at length whether it could provide EPA with advice about when a systematic review should be performed but decided it could not be more specific because that decision will depend on the availability of data and resources, the anticipated actions, the time frame for decision making, and other factors. As the committee can attest, one disadvantage in conducting a systematic review is that it can be time and resource intensive, particularly for individuals that have not previously conducted a systematic review. Some steps are inherently resource consuming. For example, two individuals independently perform most steps in the systematic-review process, such as data abstraction and risk of bias determinations. The committee therefore recognized that there was a need for the inclusion of investigation methods that do not rely on systematic-review methods. One form of accelerated evidence synthesis that has been suggested by practitioners of systematic review is a rapid review, in which components of the systematic-review process are simplified or omitted (e.g., the need for two independent reviewers) to produce information in a more timely manner (Khangura et al. 2014; Polisena et al. 2015; Tricco et al. 2015). The committee recognizes the need to streamline the process; however, evidence suggests that rapid reviews should not be viewed as a substitute for a systematic review and that the time savings correlate with decreased methodologic quality or robustness (Harker and Kleijnen 2012; Featherstone et al. 2015). For some questions, systematic review methods are not expected to be time intensive (e.g., questions with small literature bases or data-poor chemicals). For other questions, the ability to streamline, increase efficiency, or automate portions of the review process might be considered by EPA. Efforts are currently under way to automate individual tasks in a systematic review, such as the literature search, study selection, and data extraction (Tsafnat et al. 2014; Jonnalagadda et al. 2015). The committee recognizes that the methods and role of systematic review and meta-analysis in toxicology are evolving rapidly and EPA will need to stay abreast of these developments, strive for transparency, and use appropriate methods to address its questions.

The committee used the National Toxicology Program's Office of Health Assessment and Translation (OHAT) method (Rooney et al. 2014; NTP 2015) to evaluate the confidence in the evidence. Overall, the committee found the OHAT method was relatively easy to implement but identified a few challenges. For example, a body of evidence from experimental animal studies was given an initial confidence rating of “high” on the basis of the study-design issues that are inherent in experimental studies. That approach is consistent with the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) approach for considering randomized controlled trials. Multiple factors were then considered to determine whether to downgrade (e.g., risk of bias concern or unexplained inconsistency in the results) or upgrade (e.g., large magnitude of effect or evidence of a dose-response relationship) confidence in the body of evidence. In practice, the committee found that bodies of evidence had multiple risk of bias issues and found it challenging to determine whether to downgrade confidence by one or more levels on the basis of the problems with study design or conduct observed. The method would benefit from clearer guidelines or examples to illustrate the consideration of factors for downgrading or upgrading confidence.

Several other lessons became apparent during the course of the study. They include the need for improved reporting of study design, conduct, and results on the part of scientists to facilitate risk of bias evaluations; approaches for displaying behavioral and categorical data within a systematic review; and inclusion of subject-matter expertise on the review team, including expertise in meta-analysis and other statistical approaches. It is often not clear if a study used best practices in the study design and conduct but failed to report those good practices, or if there were issues in study conduct that would result in potential bias. Researchers should be encouraged to follow reporting guidelines (e.g., the ARRIVE guidelines for reporting of animal studies and the STROBE statement for reporting of observational studies in epidemiology). They should also use methods to minimize bias in research conduct particularly for issues where there is empirical evidence that risk of bias practices can affect the effect size (e.g., the key issues of randomizing animals to treatment or and blinding of outcome assessors to study groups).

The committee also found that quantitative data analysis methods can be valuable in evaluating a body of evidence. For phthalates, the use of common end-point measures across most studies enabled extensive use of meta-analysis techniques, the results of which contributed to evaluations of the precision, heterogeneity, magnitude of effect, and presence of a dose-response gradient across studies. Use of meta-analysis techniques is much more robust than relying on individual study results, because they account for potential heterogeneity across studies and the statistical power of each individual study. For PBDEs, the variety of test methods and end points limited the use meta-analysis, although the committee did demonstrate its use for one of the more commonly reported end points. In this case, the committee's preferred option would have been to conduct benchmark dose modeling on individual studies; however, many studies did not report sufficient details on the study design (e.g., group sizes) or results (e.g., standard deviations) to estimate benchmark doses. Thus, the committee had to rely on reported lowest-observed-adverse-effect levels (LOAELs) and no-observed-adverse-effect levels (NOAELs) based on pair-wise statistical significance in evaluating studies. The use of LOAELs and NOAELs is less than ideal because they depend highly on individual study-design characteristics; therefore, apparent differences among studies might be explained by design differences, such as sample size or dose spacing, rather than true inconsistency. Indeed, the committee found that some of its concerns about consistency and heterogeneity in the PBDE studies were ameliorated by the results of the meta-analysis performed on the Morris water maze data.

As discussed above and elsewhere in the report, the committee used existing methods and software for performing the systematic reviews, the meta-analyses, and the evidence integration. The selected methods and software were primarily chosen because of the committee's experience with using them and should not be taken as an indication of an endorsement of them or that they are the recommended approaches for EPA to use. The committee recognizes that other software, statistical packages, and evidence integration approaches could fit EPA's needs.

LESSONS LEARNED FROM EVIDENCE INTEGRATION

In keeping with the statement of task, the committee demonstrated how human and animal data streams can be integrated and used to determine whether a likely causal association is supported by the evidence. Determinations regarding causality were constrained by the PECO statements that were described earlier. The committee synthesized the animal and human evidence and reached hazard conclusions using an OHAT framework (NTP 2015). Earlier chapters describe the data integration steps in more detail. Using the OHAT framework and language used to describe the strength of the evidence, the committee reached a number of conclusions regarding the endocrine toxicity of phthalates and PBDEs. Two example causality statements are provided below.

  • Diethylhexyl phthalate (DEHP) is presumed to be a reproductive hazard to humans, and there is moderate evidence that decreased AGD in humans occurs following low-dose exposure to this phthalate.
  • Developmental exposure to BDE-47 is presumed to pose a hazard to intelligence in humans, and there is a moderate level of evidence that effects on IQ occur following low-dose exposure to this congener.

The Use of Mechanistic Data for Evidence Integration

The committee considered mechanistic data, which included pharmacokinetic information, in reaching its final hazard identification conclusions for each end point. For example, the committee used mechanistic evidence to support the finding that DEHP effects on AGD in humans were biologically plausible. The OHAT hazard identification scheme also allows consideration of mechanistic data to upgrade or downgrade the initial hazard determination. The committee found that the guidance in the OHAT handbook on the level of evidence needed to upgrade or downgrade the initial hazard determination on the basis of mechanistic data was somewhat lacking, and there are few OHAT monographs or published systematic reviews that have used the approach. The method would benefit from additional clarity in the guidelines or particularly from examples to illustrate mechanistic evidence that would provide strong support for biological plausibility of the observed effect sufficient to justify upgrading or downgrading the hazard conclusion.

Mechanistic data were also used to evaluate whether there was dose-response concordance between humans and animals. In the case of DEHP effects on AGD, the committee noted significant species differences in phthalate metabolism and clearance, and effects on fetal testosterone (Ito et al 2005; Gaido et al. 2007; Johnson et al. 2012). In addition, qualitative and quantitative differences in phthalate pharmacokinetics also occur between rodents, nonhuman primates, and humans (Kessler et al. 2004; McKinnell et al. 2009; Kurata et al. 2012). The committee found comparing evidence on dose-response relationships between animal and human studies to be challenging and imprecise because animal studies often measure external administered doses (usually without measures of internal dose), whereas human studies measure biomarkers of internal dose (with estimates of the external administered dose being uncertain). Toxicology studies that measure internal dose metrics, including metrics that are similar to those used in human biomonitoring and those most relevant to the target tissue dose, could help address those challenges.

Broader mechanistic questions concerning how EACs might alter normal hormone function at low doses have been raised in the scientific literature (Skakkebaek et al. 2011; Vandenberg 2014; Maqbool et al. 2016). Although those questions are potentially important for risk assessment of EACs, they were deemed beyond the charge of the committee.

End-Point Concordance

The methods used to assess fetal testosterone, AGD, and hypospadias were qualitatively similar between animal and human studies, and this simplified the analysis of whether the responses seen were concordant or discordant. In addition, end-point consistency between human and animal studies provided additional confidence in biological plausibility for hazard identification. Moreover, because similar end points were evaluated in multiple studies, meta-analyses were possible for several end points of interest.

In the case of PBDE effects on cognitive function, change in IQ was a primary measure in children, and a wide array of neurobehavioral assays were used in the animal studies. Although the assays evaluate cognitive function in animals (e.g., tests of learning and memory), the types of tests used, the timing of evaluation during postnatal development, and other methodologic differences restricted the ability of the committee to synthesize the data using meta-analysis or other methods.

OTHER ISSUES

A related element of the statement of task asked the committee to consider adverse outcome pathways (AOPs) and high-throughput data in its analysis. The committee found that using AOPs and high-throughput data was difficult because molecular initiating events involved in the phthalate reproductive effects remain unclear, and steroidogenic assays that are used in current high-throughput assay systems (e.g., ToxCast) often rely on human adrenal cell lines of unknown mechanistic relevance for phthalate toxicity. In the case of the PBDEs, several potential mechanisms have been proposed (Costa et al. 2014); however, none have been conclusively linked to the neurobehavioral outcomes evaluated by the committee. Although high-throughput data were not helpful in the committee's analyses of phthalates and PBDE, such data could be used for priority setting and other uses. For example, chemicals that exhibit endocrine activity in a high-throughput assay might be given a higher priority for future testing. That approach is consistent with the EPA's Endocrine Disruptor Screening Program. High-throughput data might be used to differentiate chemicals on the basis of bioactivity potency to set priorities for chemical testing. High-throughput data might also be used to support read-across methods for chemicals that have few human or animal data. In addition, the use of reverse toxicokinetic methods to support in vitro–to–in vivo extrapolations that can be used to convert an in vitro concentration into an estimated serum concentration will be important (Wetmore et al. 2012). The committee anticipates that methods will emerge to support the use of in vitro data streams in systematic reviews.

The committee also considered whether animal toxicity studies could predict the low-dose effects seen with phthalates or PBDEs in people. The committee found that, although animal data could help identify hazards associated with phthalates and PBDE exposure, they were unable to predict exposures at which effects occurred in people. Indeed, differences in exposure between animal studies and those observed in the general human population spanned several orders of magnitude. Strategies to help resolve apparent discrepancies often rest on an improved understanding of health effects seen in people, revisions to animal-testing guidelines that help improve their predictive value, consideration of animal studies that include additional end points, and efforts to use mechanistic and pharmacokinetic data to bridge these seemingly disparate data streams (NRC 2007, 2009). The recent National Academies report Using 21st Century Science to Improve Risk-Related Evaluations (NASEM 2017) emphasized the need to align environmental and test-system exposures and develop the models and methods necessary to do so. Thus, new test methods, models, and approaches will need to evolve to address the apparent discrepancies.

REFERENCES

  • Axelrad DA, Bellinger DC, Ryan LM, Woodruff TJ. Dose-response relationship of prenatal mercury exposure and IQ: An integrative analysis of epidemiologic data. Environ. Health Perspect. 2007;115(4):609–615. [PMC free article: PMC1852694] [PubMed: 17450232]
  • Bellinger DC. A strategy for comparing the contributions of environmental chemicals and other risk factors to neurodevelopment of children. Environ. Health Perspect. 2012;120(4):501–507. [PMC free article: PMC3339460] [PubMed: 22182676]
  • Bubanj TB, Perovic SV, Milicevic RM, Jovcic SB, Marjanovic ZO, Djordjevic MM. Sexual behavior and sexual function of adults after hypospadias surgery: A comparative study. J. Urol. 2004;171(5):1876–1879. [PubMed: 15076297]
  • Costa LG, de Laat R, Tagliaferri S, Pellacani C. A mechanistic view of polybrominated diphenyl ether (PBDE) developmental neurotoxicity. Toxicol. Lett. 2014;230(2):282–294. [PMC free article: PMC4028440] [PubMed: 24270005]
  • Eisenberg ML, Lipshultz LI. Anogenital distance as a measure of human male fertility. J. Assist. Reprod. Genet. 2015;32(3):479–484. [PMC free article: PMC4363236] [PubMed: 25533333]
  • Eisenberg ML, Hsieh MH, Walters RC, Krasnow R, Lipshultz LI. The relationship between anogenital distance, fatherhood, and fertility in adult men. PLoS ONE. 2011;6(5):0018973. [PMC free article: PMC3092750] [PubMed: 21589916]
  • Eisenberg ML, Shy M, Walters RC, Lipshultz LI. The relationship between anogenital distance and azoospermia in adult men. Int. J. Androl. 2012;35(5):726–730. [PubMed: 22519659]
  • EPA (U.S. Environmental Protection Agency). State of the Science Evaluation: Nonmonotonic Dose Responses as They Apply to Estrogen, Androgen, and Thyroid Pathways and EPA Testing and Assessment Procedures. U.S. Environmental Protection Agency; Jun, 2013. 2013. [November 11, 2015]. https://www​.epa.gov/sites​/production/files​/2016-01/documents/nmdr.pdf.
  • Featherstone RM, Dryden DM, Foisy M, Guise JM, Mitchell MD, Paynter RA, Robinson KA, Umscheid CA, Hartling L. Advancing knowledge of rapid reviews: An analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Syst. Rev. 2015;17(4):50. [PMC free article: PMC4415284] [PubMed: 25925676]
  • Gaido KW, Hensley JB, Liu D, Wallace DG, Borghoff S, Johnson KJ, Hall SJ, Boekelheide K. Fetal mouse phthalate exposure shows that gonocyte multinucleation is not associated with decreased testicular testosterone. Toxicol. Sci. 2007;97(2):491–503. [PubMed: 17361019]
  • Gray LE Jr., Furr J, Tatum-Gibbs KR, Lambright C, Sampson H, Hannas BR, Wilson VS, Hotchkiss A, Foster PM. Establishing the “biological relevance” of dipentyl phthalate reductions in fetal rat testosterone production and plasma and testis testosterone levels. Toxicol. Sci. 2016;149(1):178–191. [PMC free article: PMC4715258] [PubMed: 26454885]
  • Harker J, Kleijnen J. What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. Int. J. Evidence Based Healthcare. 2012;10(4):397–410. [PubMed: 23173665]
  • Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. [June 1, 2017]. www​.handbook.cochrane.org.
  • Hotchkiss AK, Parks-Saldutti LG, Ostby JS, Lambright C, Furr J, Vandenbergh JG, Gray LE Jr. A mixture of the “antiandrogens” linuron and butyl benzyl phthalate alters sexual differentiation of the male rat in a cumulative fashion. Biol. Reprod. 2004;71(6):1852–1861. [PubMed: 15286035]
  • Howdeshell KL, Hotchkiss AK, Gray LE Jr. Cumulative effects of antiandrogenic chemical mixtures and their relevance to human health risk assessment. Int. J. Hyg. Environ. Health. 2017;220(2 Pt. A):179–188. [PMC free article: PMC5401775] [PubMed: 27923611]
  • Hsieh MH, Eisenberg ML, Hittelman AB, Wilson JM, Tasian GE, Baskin LS. Caucasian male infants and boys with hypospadias exhibit reduced anogenital distance. Hum. Reprod. 2012;27(6):1577–1580. [PMC free article: PMC3357195] [PubMed: 22434852]
  • IOM (Institute of Medicine). Finding What Works in Health Care: Standards for Systematic Review. Washington, DC: The National Academies Press; 2011. [PubMed: 24983062]
  • IPCS (International Programme on Chemical Safety). Part 1: IPCS/OECD Key Generic Terms Used in Chemical Hazard/Risk Assessment. Part 2: IPCS Glossary of Key Exposure Assessment Terminology. Geneva: World Health Organization; 2004. [November 11, 2015]. IPCS Risk Assessment Terminology. http://www​.inchem.org​/documents/harmproj/harmproj/harmproj1​.pdf.
  • Ito Y, Yokota H, Wang R, Yamanoshita O, Ichihara G, Wang H, Kurata Y, Takagi K, Nakajima T. Species differences in the metabolism of di(2-ethylhexyl) phthalate (DEHP) in several organs of mice, rats, and marmosets. Arch. Toxicol. 2005;79(3):147–154. [PubMed: 15798888]
  • Jain VG, Singal AK. Shorter anogenital distance correlates with undescended testis: A detailed genital anthropometric analysis in human newborns. Hum. Reprod. 2013;28(9):2343–2349. [PubMed: 23838161]
  • Johnson KJ, Heger NE, Boekelheide K. Of mice and men (and rats): Phthalate-induced fetal testis endocrine disruption is species-dependent. Toxicol. Sci. 2012;129(2):235–248. [PMC free article: PMC3491958] [PubMed: 22700540]
  • Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: A systematic review. Syst. Rev. 2015;4(78) [PMC free article: PMC4514954] [PubMed: 26073888]
  • Kerlin R, Bolon B, Burkhardt J, Francke S, Greaves P, Meador V, Popp J. Scientific and Regulatory Policy Committee: Recommended (“best”) practices for determining, communicating, and using adverse effect data from nonclinical studies. Toxicol. Pathol. 2016;44(2):147–162. [PubMed: 26704930]
  • Kessler W, Numtip W, Grote K, Csanády GA, Chahoud I, Filser JG. Blood burden of di(2ethylhexyl) phthalate and its primary metabolite mono(2-ethylhexyl) phthalate in pregnant and nonpregnant rats and marmosets. Toxicol. Appl. Pharmacol. 2004;195(2):142–153. [PubMed: 14998681]
  • Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: An emerging approach to evidence synthesis in health technology assessment. Int. J. Technol. Assess. Health Care. 2014;30(1):20–27. [PubMed: 24451157]
  • Kurata Y, Makinodan F, Shimamura N, Katoh M. Metabolism of di (2-ethylhexyl) phthalate (DEHP): Comparative study in juvenile and fetal marmosets and rats. J. Toxicol. Sci. 2012;37(1):33–49. [PubMed: 22293410]
  • Lagarde F, Beausoleil C, Belcher SM, Belzunces LP, Emond C, Guerbet M, Rousselle C. Nonmonotonic dose-response relationships and endocrine disruptors: A qualitative method of assessment. Environ. Health. 2015;14(1):13. [PMC free article: PMC4429934] [PubMed: 25971433]
  • Lam J, Lanphear BP, Bellinger D, Axelrad DA, McPartland J, Sutton P, Davidson L, Daniels N, Sen S, Woodruff TJ. Developmental PBDE exposure and IQ/ADHD in childhood: a systematic review and meta-analysis. Environ. Health Perspect. In press. [PMC free article: PMC5783655] [PubMed: 28799918]
  • Maqbool F, Mostafalou S, Bahadar H, Abdollahi M. Review of endocrine disorders associated with environmental toxicants and possible involved mechanisms. Life Sci. 2016;145:265–273. [PubMed: 26497928]
  • McKinnell C, Mitchell RT, Walker M, Morris K, Kelnar CJH, Wallace WH, Sharpe RM. Effect of fetal or neonatal exposure to monobutyl phthalate (MBP) on testicular development and function in the marmoset. Hum. Reprod. 2009;24(9):2244–2254. [PMC free article: PMC2727403] [PubMed: 19491204]
  • Mendiola J, Stahlhut RW, Jorgensen N, Liu F, Swan SH. Shorter anogenital distance predicts poorer semen quality in young men in Rochester, New York. Environ. Health Perspect. 2011;119(7):958–963. [PMC free article: PMC3222997] [PubMed: 21377950]
  • Mendiola J, Melgarejo M, Monino-Garcia M, Cutillas-Tolin A, Noguera-Velasco JA, Torres-Cantero AM. Is anogenital distance associated with semen quality in male partners of subfertile couples? Andrology. 2015;3(4):672–676. [PubMed: 26097129]
  • NASEM (National Academies of Sciences, Engineering, and Medicine). Using 21st Century Science to Improve Risk-Related Evaluations. Washington, DC: The National Academies Press; 2017. [PubMed: 28267305]
  • NRC (National Research Council). Toxicity Testing in the 21st Century: A Vision and a Strategy. Washington, DC: The National Academies Press; 2007.
  • NRC. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press; 2009. [PubMed: 25009905]
  • NRC. Review of the Environmental Protection Agency's State-of-the Science Evaluation of Nonmontonic Dose-Response Relationships as They Apply to Endocrine Disruptors. Washington, DC: The National Academies Press; 2014a.
  • NRC. Review of EPA's Integrated Risk Information System (IRIS) Process. Washington, DC: The National Academies Press; 2014b. [PubMed: 25101400]
  • NTP (National Toxicology Program). Handbook for Conducting a Literature-based Health Assessment Using OHAT Approach for Systematic Review and Evidence Integration. Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences; Jan 9, 2015. 2015. [September 21, 2015]. http://ntp​.niehs.nih​.gov/ntp/ohat/pubs/handbookjan2015_508​.pdf.
  • Palazzi X, Burkhardt JE, Caplain H, Dellarco V, Fant P, Foster JR, Francke S, Germann P, Gröters S, Harada T, Harleman J, Inui K, Kaufmann W, Lenz B, Nagai H, Pohlmeyer-Esch G, Schulte A, Skydsgaard M, Tomlinson L, Wood CE, Yoshida M. Characterizing “adversity” of pathology findings in nonclinical toxicity studies: Results from the 4th ESTP International Expert Workshop. Toxicol. Pathol. 2016;44(6):810–824. [PubMed: 27102650]
  • Pandiri AR, Kerlin RL, Mann PC, Everds NE, Sharma AK, Myers LP, Steinbach TJ. Is it adverse, nonadverse, adaptive, or artifact? Toxicol. Pathol. 2017;45(1):238–247. [PMC free article: PMC5225139] [PubMed: 27770107]
  • Polisena J, Garritty C, Umscheid CA, Kamel C, Samra K, Smith J, Vosilla A. Rapid Review Summit: An overview and initiation of a research agenda. Syst. Rev. 2015;4:137. [PMC free article: PMC4583747] [PubMed: 26407674]
  • Rooney AA, Boyles AL, Wolfe MS, Bucher JR, Thayer KA. Systematic review and evidence integration for literature-based environmental health science assessments. Environ. Health Perspect. 2014;122(7):711–718. [PMC free article: PMC4080517] [PubMed: 24755067]
  • Schlomer B, Breyer B, Copp H, Baskin L, DiSandro M. Do adult men with untreated hypospadias have adverse outcomes? A pilot study using a social media advertised survey. J. Pediatr. Urol. 2014;10(4):672–679. [PMC free article: PMC4153791] [PubMed: 24613143]
  • Skakkebaek NE, Toppari J, Söder O, Gordon CM, Divall S, Draznin M. The exposure of fetuses and children to endocrine disrupting chemicals: A European Society for Paediatric Endocrinology (ESPE) and Pediatric Endocrine Society (PES) call to action statement. J. Clin. Endocrinol. Metab. 2011;96(10):3056–3058. [PubMed: 21832106]
  • Thankamony A, Lek N, Carroll D, Williams M, Dunger DB, Acerini CL, Ong KK, Hughes IA. Anogenital distance and penile length in infants with hypospadias or cryptorchidism: Comparison with normative data. Environ. Health Perspect. 2014;122(2):207–211. [PMC free article: PMC3915266] [PubMed: 24316680]
  • Thankamony A, Pasterski V, Ong KK, Acerini CL, Hughes IA. Anogenital distance as a marker of androgen exposure in humans. Andrology. 2016;4(4):616–625. [PMC free article: PMC6225986] [PubMed: 26846869]
  • Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, Perrier L, Hutton B, Moher D, Straus SE. A scoping review of rapid review methods. BMC Med. 2015;13:224. [PMC free article: PMC4574114] [PubMed: 26377409]
  • Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst. Rev. 2014;3:74. [PMC free article: PMC4100748] [PubMed: 25005128]
  • Vandenberg LN. Low-dose effects of hormones and endocrine disruptors. Vitam. Horm. 2014;94:129–165. [PubMed: 24388189]
  • Watkins MW, Smith LG. Long-term stability of the Wechsler Intelligence Scale for Children–Fourth Edition. Psychol. Assess. 2013;25(2):477–483. [PubMed: 23397927]
  • Wetmore BA, Wambaugh JF, Ferguson SS, Sochaski MA, Rotroff DM, Freeman K, Clewell HJ III, Dix DJ, Andersen ME, Houck KA, Allen B, Judson RS, Singh R, Kavlock RJ, Richard AM, Thomas RS. Integration of dosimetry, exposure, and high-throughput screening data in chemical toxicity assessment. Toxicol. Sci. 2012;125(1):157–174. [PubMed: 21948869]

Footnotes

1

Adverse effect: a biological change in an organism that results in an impairment of functional capacity, a decrease in the capacity to compensate for stress, or an increase in susceptibility to other influences (adapted from IPCS 2004).

Copyright 2017 by the National Academy of Sciences. All rights reserved.
Bookshelf ID: NBK453246
Contents

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (31M)

In this Page

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to PubMed

Recent Activity

ClearTurn OffTurn On

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...