Conservative Management and Outcome

The primary source of ammonia is protein breakdown; therefore, the cornerstone of MM for chronic UCD is dietary protein restriction. Because of the limited daily protein allowance, some physicians administer amino acids not made by the human body, known as the essential amino acids (EAAs). However, there is no strong consensus for EAA use,^5,25,28 and outcomes have not been studied.

The most important medical advances in the acute and chronic treatment of UCDs were the development of substrates to promote alternative pathways of waste nitrogen excretion.^18,29 Sodium benzoate combines with the amino acid glycine to form hippurate, which can be readily excreted in the urine, removing 1 atom of nitrogen for each molecule of benzoate provided. Sodium phenylacetate combines with glutamine to form phenylacetylglutamine, which is also readily excreted, removing 2 atoms of nitrogen for each molecule of phenylacetate. The same phenylacetate pathway is exploited with the administration of sodium phenylbutyrate (Buphenyl) or its prodrug glycerol phenylbutyrate (Ravicti), which both have a much less noxious odor and taste than does phenylacetate itself. Intravenous infusion of combined sodium benzoate and sodium phenylacetate (Ammonul) is now used routinely for the treatment of acute hyperammonemia, whereas sodium benzoate, sodium phenylbutyrate, or glycerol phenylbutyrate is commonly used for chronic, long-term oral administration.^25,30 With regard to outcomes, no oral medication has been shown to be superior,³¹ and all of these treatments unfortunately carry risks of adverse reactions. Dietary protein restriction combined with the employment of substrates to promote alternative pathways of waste nitrogen excretion is the most effective medical treatment for UCDs.^18,29

The general consensus is that the development of alternative pathway therapies and their subsequent widespread availability by the mid-1990s has resulted in decreased mortality and morbidity from these disorders.^18,29 However, even this is in dispute: The results from a meta-analysis of UCD articles published between 1978 and December 22, 2014 (Burgard et al⁴), reported no improvement in survival in more than 3 decades. Outcomes in patients with neonatal-onset UCDs are typically the worst; 24% to 50% of these patients have been reported to succumb to hyperammonemia.^4,32 Additionally, most survivors show significant neurodevelopmental disabilities, often correlated with the severity of the enzyme deficiency. In a longitudinal natural history study (LS) performed by our NIH-funded Urea Cycle Disorders Consortium (UCDC), a member of the Rare Diseases Clinical Research Network (RDCRN),⁸ the proportion of patients with a poor cognitive outcome (intelligence quotient [IQ]/developmental quotient <70) was high, ranging from 47% to 68% in the various UCDs. This was observed in patients <4 and ≥4 years of age. Poor cognitive outcome was not fully explained by age of onset (<4 vs ≥4 years), peak ammonia level, or duration of the initial admission. At present, the variables associated with a poorer cognitive outcome in patients with UCDs who are managed medically are not well understood, making progress in improving this approach difficult. Duration of hyperammonemic coma,² peak ammonia level,³³ and the number of episodes of hyperammonemia³³ have all been posited to correlate with cognitive outcome, but these studies were uncontrolled and oversimplified neurocognitive measures.

Although hyperammonemia appears to be the most important contributor to neurological injury, more recent evidence points to the effect of secondary toxic metabolites, which accumulate in specific UCD subtypes, on cognitive outcomes.¹⁴ Numerous studies demonstrate that the biomarkers glutamine,³⁴ arginine,¹⁴ citrulline,¹⁴ and nitric oxide¹⁶ are also important correlates of neurocognitive functioning. These biomarkers may in part explain differences in neurocognitive outcomes between UCD subtypes.^8,14

LT and Outcome

LT was initiated in the early 1990s before alternate pathway medical therapy was widely available.^22-24 There were anecdotal case reports of LT in 4 of the UCDs (CPS1D, OTCD, ASSD, and ASLD).^22,23,35 A larger survey of 16 US patients who had received LTs from 4 major transplant centers found that 14 of these patients had survived at least 1 to 6 years posttransplantatation.²⁴ Their neurological status posttransplantation was in most cases moderately to severely impaired and correlated closely with their condition before transplantation. However, the quality of their lives seemed to have improved. Subsequently, to determine whether aggressive medical therapy could improve the outcome of severe neonatal-onset UCD, 5 such patients were monitored at Baylor College of Medicine before and after LT, including 2 male patients with OTCD, 2 male patients with CPS1D, and 1 female patient with OTCD and intractable hyperammonemia.³⁶ Three of these infants had serial developmental testing (Griffiths scale) before and after transplantation. It was found that their pretransplantation overall developmental scales (51, 86, and 56, respectively) were stabilized posttransplantation (70-83, 80-76, and 51-47, respectively, at 2 subsequent evaluations). Therefore, the authors recommended that early transplantation be considered the treatment of choice in these severely affected infants.

As a result of the above-mentioned early successes in patients with UCDs, LT is now frequently performed to treat UCDs. A retrospective analysis from the United Network for Organ Sharing (UNOS) that included 186 patients with UCDs who underwent LT³⁷ showed an increased frequency of LT for UCDs and organic acidemia over the last decade, with 5-year survival rates of 88% to 99% depending on age. A more recent review of the UNOS database³⁸ identified 265 pediatric and 13 adult patients who underwent an LT for a UCD between 1987 and 2010. The majority (68%) of these patients were transplanted before age 5 years. Overall 1-, 5-, and 10-year survival rates of 93%, 89%, and 87%, respectively, were observed. Univariate and multivariate risk factor analyses did not reveal any significant demographic, anthropometric, or laboratory characteristics of donors or recipients that were predictive of posttransplant death or graft loss.

Transplanted livers may come from living related or deceased donors. Pediatric patients with UCDs may also be eligible for split LT, which involves the division of the donor liver from a deceased adult between a pediatric recipient and an adult recipient to maximize the benefit of each available donor organ. In the aforementioned recent UNOS review, approximately 95% of patients received livers from deceased donors,^37,38 likely primarily because of the automatic high-priority status received by affected children with UCD. Among deceased-donor-liver recipients with UCD, only 15% received a split liver. One analysis suggested that posttransplant mortality was higher with split LT.³⁷

Unfortunately, children who undergo transplantation typically continue to have cognitive and motor delays not associated with age or weight at transplant, sex, ethnicity, liver graft type, or the hospital where the transplant occurred. No prospective studies of LT vs MM in UCDs have been performed. A 2017 retrospective review of data collected from questionnaires sent to 928 Japanese institutions³⁹ identified 177 patients with a UCD diagnosed between January 1999 and March 2009, including 42 recipients of living-donor LTs. Among patients for whom the maximum ammonia concentration (MAC) was ≥300 μmol/L and full-scale IQ (FSIQ) scores were available, 11 of 21 LT recipients had FSIQ scores ≥70 (which the authors report as “normal”), whereas only 11 of 54 MM patients had FSIQ scores ≥70 (P = .08). The authors concluded that LT may prevent further neurodevelopmental complications in children with a MAC ≥300 μmol/L. In a separate analysis employing data from the same questionnaire,⁴⁰ these authors report that specifically among patients with neonatal-onset UCDs and a MAC ≥360 μmol/L, 8 of 18 LT patients had FSIQ scores ≥70, whereas among those receiving MM, only 4 of 28 patients had FSIQ scores ≥70. Additionally, among patients with neonatal-onset UCDs and a MAC <360 μmol/L, 5 of 5 patients who received LT had FSIQ scores ≥70, whereas among the patients receiving MM, only 2 of 7 patients had FSIQ scores ≥70 (P = .013). The authors conclude that a MAC ≥360 μmol/L is a marker of poor neurodevelopmental outcomes, that patients with neonatal-onset UCDs and a MAC ≥300 μmol/L at onset should undergo LT, and that even patients with neonatal-onset UCDs with a MAC <300 μmol/L should undergo LT to protect the brain.

Process of Making UCD Treatment Decisions

Patients UCDs who receive MM are always at elevated risk for serious disability or death, but this risk is not easily quantifiable and may change at various times in their lives.⁴¹ This ambiguity makes treatment decisions, such as if or when to pursue a high-risk procedure like LT, particularly challenging for the caregivers of children with UCDs. In other medical conditions, the decision to perform transplantation is often made because organ function has failed, and the choice is one of life or death. However, for patients with UCDs, the decision can be more complex, as outcomes from medical therapy vary widely and because transplantation is ideally initiated when patients are stable rather than critically ill. As described previously, there is also limited empirical evidence to support clinical guidance on treatment alternatives for patients with UCDs. This exposes these important treatment decisions to high levels of uncertainty and personal judgment, forcing patients and their families to pursue intervention in the absence of clear, evidence-based information. Despite the importance of, and complexity surrounding, the decision to continue MM or consider LT, no research has been conducted on how families of patients with UCDs make these treatment choices.

Gaps in Knowledge Addressed by This Project

The objectives of this study were to provide patients with UCDs and their families with evidence-based information that is currently lacking regarding management alternatives (conservative MM vs LT), including answers to the following questions:

• What is the risk of mortality and morbidity in the 2 approaches?
• What can patients expect in terms of cognitive development and/or other developmental outcomes?
• What are the potential pros and cons in terms of considerations for quality of life (QOL)?

To maximize the utility of the information produced from this comparative effectiveness analysis, we integrated a qualitative component that examined the decision-making experience of families affected by UCDs to identify key factors that families consider in evaluating and reaching a treatment choice. From information collected directly from families and their providers, we built an original conceptual framework that explores how key factors interrelate to drive treatment decision-making in this population. These important insights into the UCD patient experience informed the efforts of our collaborators, the National Urea Cycle Disorders Foundation (NUCDF) and The George Washington University (GWU), to craft a dissemination strategy for the evidence produced from this study. This carefully developed strategy is both responsive to the expressed needs of patients and their families and aligns with the decision-making process described through the analysis. Our specific aims were as follows:

• Aim 1: To study 2 UCD patient cohorts, 1 cohort treated with MM and the other treated by LT, comparing survival rate, neurocognitive function, and patient-reported QOL.
• Aim 2: To examine, through a representative sample of patient families and their providers, how UCD treatment decisions are made, describing the factors that influence a family's decision to continue conservative MM or proceed to an LT.
• Aim 3: To develop a strategy to disseminate the study findings from aim 1 that align with the decision-making process illustrated through aim 2 and that is responsive to the expressed needs of patients with UCDs and their families.

Study Time Frame

This study was activated in March 2016 and closed to enrollment in May 2018 after meeting all recruitment and enrollment milestones.

Participants

For aim 1 in the prospective cohort study design, we relied on the merger of uniform historically and prospectively collected data from (1) participants in the largest LS of the natural history of UCDs conducted by the NIH-funded UCDC,⁴² (2) patients referred by NUCDF, and (3) patients referred by SPLIT. Eligible participants included North Americans with neonatal-onset UCD diagnoses of CPS1D, OTCD, ASSD, or ASLD born primarily after July 1, 1996, the date after which alternative pathway medications became commercially available. Neonatal onset refers to the most severe subgroup of UCD, characterized by the complete absence of enzyme activity, thereby resulting in hyperammonemia in the neonatal period (<29 days of life), unless first ascertained by family history or newborn screening. These conditions often result in recurrent episodes of hyperammonemia and consideration for LT. Access to detailed natural history data provided the means to identify participants who had undergone LT and comparable participants who continued to rely on conventional MM, as well as the information necessary to create essentially unbiased comparisons of outcomes by treatment group.

Recruitment efforts were led by CNHS as the coordinating center for the study and facilitated by the UCDC network and NUCDF. Enrollment in the UCDC LS was initially offered to NUCDF and SPLIT participants who were not already enrolled in order to take advantage of the uniform LS research protocol, especially in capturing neurocognitive function and QOL information. Patients interested in participating in this study but who elected not to enroll in the UCDC LS were afforded similar follow-up options coordinated by experienced neuropsychologists.

Study Outcomes

In order to compare the 2 interventions (MM vs LT), NUCDF and those families receiving care at sites participating in the UCDC developed the primary outcomes representing their chief concerns: (1) survival, (2) neurocognitive development status, and (3) QOL. Because UCDs can prove lethal at any time, families identified survival as a top concern, as well as the cognitive function that is critically important for independence and the ability to realize one's potential. Validated measures already being collected from those participants enrolled in the UCDC LS were used for all participants in this study. These included neuropsychological measures of IQ, executive function, motor function, learning and memory, and mental and emotional/behavioral health outcomes, as well as QOL assessments based on validated instruments, the Pediatric Quality of Life Inventory (PedsQL) and the 36-item Short Form Health Survey (SF-36) (Table 1).

Table 1

Validated Tests and Scales.

Sample Size Calculations and Power

PASS 12 (NCSS, LLC⁶⁰) was used to evaluate statistical power based on a projected sample size of 90 MM and 80 LT participants, in 2-tailed testing at an α = .05. For neurodevelopmental and QOL assessments, the study was considered to provide 80% power to detect a modest 0.4 SD effect size difference between groups, assuming 2 repeated measurements per person correlating at a 70% intraclass correlation coefficient. A 0.4 SD effect size represents a moderate, 6-point score difference considered by colleagues with expertise in neuropsychology to represent a difference that could have clinical implications in standardized neuropsychological scores (mean [SD], 100 [15]). Similar and even smaller QOL differences are detectable, which tend to have lower mean standardized scores (80-87) and similar SDs (13-16). For mortality, assuming the rate in the MM group remained at 28%, the study had 80% power to detect an increase in mortality in the LT group of approximately 60%, which was robust to differences in sample size between 80 and 90 per group. Based on these considerations, the study was well powered to detect a clinically meaningful difference in neurocognitive outcomes and QOL and was adequately powered to detect a moderate difference in mortality between groups.

Data Collection and Sources

Deidentified data from the UCDC LS, including sociodemographic, clinical, laboratory, neuropsychological testing (NPT), QOL assessment, and vital records data, were supplemented with comparable data from participants in this study (PCORI) who were recruited and enrolled through NUCDF and CNHS (Figure 2). We used data from eligible LS participants for analysis, as permissible per the LS consent form. For those patients not enrolled in the LS and only participating in this study, we confirmed eligibility and enrolled participants either in person or via telephone. We then collected both historical and concurrent data, including those obtained from a review of medical records and from patients or their families through standard interviews. The study team administered a QOL assessment by telephone or in person, and a qualified neuropsychologist conducted NPT in the patient's home or in the clinic.

We selected NPT to measure key cognitive outcomes of interest. Study neuropsychologists conducted a battery of tests appropriate for each patient's age-matched norms (Table 1). These tests were the same or overlapped those currently used in the UCDC LS. The study team developed a brief report of the findings, which was delivered to the participant/caregivers shortly after testing.

For the QOL assessments, the PedsQL parent report or parent proxy report to caregivers of participants aged 1 month to 18 years was administered, as well as age-appropriate PedsQL child questionnaires to participants between 13 and 18 years of age. Participants >18 years of age responded to the SF-36v2 questionnaire and the PROMIS® questionnaires. The site neuropsychologist or the site PI reviewed PROMIS questionnaires within 30 days of administration and followed up with the participant and the family as appropriate.

Analytical and Statistical Approaches

To support a fair comparison of the study outcomes of survivorship, QOL, and neuropsychological function, the study relied on 2 analytic approaches to achieve requisite comparability between those who continued to receive MM and those who elected to receive LT. The 2 methods, PS and RS matching, seek to identify and bring into balance characteristics that may influence the outcome assessment and that differ between treatment groups at the time of the treatment decision. The principal differences between these approaches as applied to this study are that (1) PS matching focuses on making summary comparisons between study participants, whereas RS matching makes comparisons at time slices to make full use of the longitudinal data; and (2) RS matching also uses imputation to estimate missing covariate and outcome values, whereas PS matching relies on the last available assessment without imputation.

For PS analysis, we used the approach defined by Imbens and Rubin,⁶¹ creating comparable groups based on group member risk factor (covariate) profiles derived from their propensity (probability) to move from conservative MM to LT. First, covariate-balancing PSs were generated using “pscore”⁶² in Stata 15 (StataCorp).⁶³ This approach uses a logistic regression model that predicts treatment assignment (MM vs LT) from sociodemographic characteristics and indicators of UCD severity selected based on prior experience, especially in the UCDC.^32,42,64 Severity indicators included UCD type (proximal vs distal [ie, a block in the urea cycle before or after the production of citrulline, respectively]); frequency and severity of hyperammonemic events (HAEs) in terms of ammonia level; and the frequency of extended hospitalizations. Sociodemographic characteristics included the patient's decade of birth and parental level of education. This process was used to achieve basic comparability between the MM and LT groups with covariate balance evaluated using “pstest” in Stata 15.⁶³ Initial covariate balance permitted the assignment of age in the MM group comparable with the age of transplantation in the LT group (here, both ages are referred to as the index age).

Subsequently, more detailed covariable indicators of severity were used to achieve better balance before outcome analysis. The selection of comparable MM and LT groups and subgroups (PS blocks) during the period before the index age followed the procedures outlined in chapter 13 of Imbens and Rubin.⁶¹ The selection of covariables began with the choice of main effects, followed by the choice of higher-order and interactive effects based on the change in the diagnostic log-likelihood ratio (dLR) associated with the addition of a term to the logistic regression model estimating the probability of LT. Initially, at each step, the main effect variable with the greatest dLR of at least 1.0 was added to the model until all variables reflecting assessments made before the index age were evaluated. A similar process was used to select the second wave of squared terms and interactions of main effects based on the addition of the term with the highest dLR ≥2.71 to the evolving logistic model.

Finally, the PS was used to construct strata, the number of which is determined by sample size and the heterogeneity of the PSs. The degree of overlap between the MM and LT groups in each PS strata was evaluated by t tests, and participants with nonoverlapping PSs were excluded. These methods were used to achieve greater pre–index age covariate balance in the analysis, comparing outcomes of mortality, QOL, and NPT scores to estimate the average treatment effect in the LT treated (ATT). The degree of similarity between groups was evaluated using χ² testing for discrete covariables and t tests for comparisons of measures of central tendency in continuous covariables.

In addition to estimating the crude ATT in the common support sample (CSS) based on information available as of the date each person was last seen, the study team employed 1 other method to improve covariate balance in evaluating the risk of mortality and 2 other methods when estimating the ATT in relation to QOL and NPT. For mortality analysis, we used Cox regression in Stata 15 to estimate the PS, index age, and pre–index age frequency of the HAE-adjusted hazard ratio (HR), secondarily also stratifying on PS blocks. For QOL and NPT outcomes,⁶⁵ we first used the “teffects ra” command in Stata 15 to control for PS when estimating ATT. Second, we used the newly available “kmatch”^66,67 procedure to implement ridge matching on PS, followed by linear regression analysis to control for index age and age at the time of outcome assessment. The “kmatch” procedure was chosen because it supports both matching and statistical adjustment in the same analysis and because it provides advantages in terms of smaller mean squared errors.

In QOL and NPT analyses, not all patients/families provided assessments of these outcomes. Logistic regression analyses were implemented to evaluate whether those providing and not providing QOL and NPT assessments differed with respect to demographic and UCD severity indicators. The models included the presence/absence of QOL and NPT assessments, study group (MM/LT), and the interaction of these effects. Interactive differences would raise the potential for bias due to missing assessments, while the main effects of missing assessments would affect the generalizability of the results. P values were generated based on χ² tests of differences in the aforementioned effects.

In addition to the PS method described above, we implemented an alternative way of defining the control group to LT using an RS-matching methodology.⁶⁸ Recall that in the PS-matching methodology described above, the MM control group includes all those who have not undergone LT as ascertained at the last follow-up date, a time after the matched index date. RS matching defines the control group differently based on the patient's status at index date to avoid possible ascertainment time bias. In our implementation of RS matching, shown in Figure 3, the RS at a given index age included all those who had not undergone LT at that age but could have. Thus, the RS at a given index age included those patients who were alive at index age but either had not undergone LT at last follow-up or who received an LT but at a later age than the index age. Look-back for these patients started at birth and ended at the index age, while follow-up started at the index age and ended at LT or end of follow-up in the database, whichever came first. The control group in this RS included those patients matched to the LT group.

Figure 3

Example of RS Matching.

Matching at each index date took into account similarity in several characteristics, including important markers of a medical condition's severity and a medical condition's history up to the index age. It exactly matched on whether the UCD sequence variant was proximal or distal, whether the patient had ever been admitted to a hospital from birth to index age, and whether the patient had any record of coma or intracranial pressure from birth to index age. Among this exact match, the selected controls were those most similar to baseline and hospitalization history for HAEs. More specifically, baseline characteristics included the education level of the patient's parent and the patient's birth decade. Hospitalization history from birth up to index age included the number of hospital admissions, the total length of stay (LOS) in the hospital, and the maximum ammonia level. The similarity was measured by Euclidean distance (shortest distance between 2 points in the multivariable space), the square root of the sum of squared differences, between characteristics of the LT patients and control patients in the RS. Because of right-skewed distributions, hospitalization history variables were rescaled (natural logarithm scale) before the distance calculation. We varied the method caliper for matching with Euclidean distance to explore balance in the 2 groups and chose the caliper with the better trade-off between balance and sample size.

Our implementation used multiple imputations to impute missing baseline characteristics, historical characteristics of HAEs, or outcomes over time. The imputation method assumed a missing at random pattern and used a chain of equations to generate imputed data sets using information from a set of observed covariates. Then, the results were combined from different imputations after matching and outcome model fitting. The full procedure is described in detail in Appendix A (Table 15A and Figure 1A).

Outcomes models were mixed-effect models, using library lme4 and coxme in R,^69-71 adjusting for repeated measures from the same participant by having a random subject effect. These include time-to-event analyses for death outcome with a Cox regression model adjusting for multiple covariates and a mixed-effect linear regression for QOL outcomes. Because of the uncertainty in these results and the sparser neuropsychological outcomes, we did not fit those outcomes. The detailed list of variables in each regression is provided for each outcome within the tables in the Results section.

Study Overview

We employed an adapted grounded theory approach, using qualitative interviews and focus groups, to examine the decision-making experiences of families affected by UCDs. This approach helped (1) identify key factors that families consider when evaluating and reaching a treatment choice and—as an additional effort that went beyond the original aim as initially proposed—(2) build an original conceptual framework that explores how these factors interrelate to drive treatment decision-making in this population. The approach for this component of the study is based on pragmatism, a philosophical orientation toward research where the focus is on developing actionable conclusions that can be applied in real-world practice. Aim 2 was developed in the spirit of this philosophy, designed to broaden the field's understanding of the decision-making experience of families affected by UCDs so that, in the short term, providers may better support patients with UCDs through difficult treatment choices and so that, in the long term, the health care system may evolve to better respond to the expressed needs and values of this population.

The study protocol (aims 1, 2, and 3) was approved by the Children's National IRB, which has a reliance agreement with GWU's IRB.

Data Collection and Sources

We collected qualitative data directly from caregivers of children affected by UCDs (n = 35) and their clinical providers (n = 26) via semistructured phone interviews. Caregivers and clinical providers were sourced through both NUCDF and connections at CNHS. Following these interviews, the study team conducted 2 in-person, 90-minute focus groups (n = 19) to validate the interview findings and clarify interview concepts that were not yet fully developed. Interviews and focus groups were recorded and transcribed verbatim for use in the analysis.

The study team developed the interview and focus group guides by integrating findings from a limited number of studies describing relevant evidence-based health care decision-making models in pediatric illness as well as initial discussions with key informants from NUCDF. The most relevant of these available studies, which examine decision-making for a large array of pediatric illnesses ranging from relatively minor incidents of upper respiratory infection⁷² to complex disorders, such as cancer⁷³ and cystic fibrosis,^74-79 are those in which a transplant is offered as a treatment.^76,80-82 Key informants, including metabolic specialists and families of children with UCDs, reviewed draft guides. The guides were revised based on feedback from this group before being used in the field and were subsequently reviewed and refined after an initial round of 3 pilot interviews.

Participants

We collected data directly from caregivers and clinical providers of children born in the United States who were diagnosed with 1 of 4 neonatal-onset UCD types (CPS1D, OTCD, ASSD, or ASLD) for which LT is a consideration as a treatment for hyperammonemia. These data were collected from the same patient population outlined in aim 1. We employed stratified purposeful sampling to recruit an initial pool of caregivers through NUCDF from the patient community. Recruitment focused on identifying participants who varied in terms of disease severity and transplant status. This initial purposeful stratification accounted for the assumption that the severity of the disease may mediate why, how, and when families consider LT as a treatment option and the perspective of both transplanted and nontransplanted patients.

In accordance with the grounded theory approach, the study team selected participants through a theoretical sampling technique in which patients were chosen to gain a deeper understanding of emerging key domains and to facilitate the development of new theoretical concepts. Under theoretical sampling, researchers follow an iterative process where they move back and forth between data collection and analysis so that emerging concepts and relationships can be tested and refined in the field through additional sampling until a point of saturation is reached (ie, additional interviews reflect only already-identified concepts and no new information is gleaned from the collection of additional data).⁸² For example, initial interview discussions suggested that major childhood transitions, such as beginning grade school and entering adolescence, have a notable influence on treatment choice. Thus, the research team made a concerted effort to recruit participants who have experienced these transitions and could expound on their meaning. Stratified purposeful sampling was used as an initial strategy to recruit a national cross-section of UCD providers for participation in the interviews. Initially, providers were recruited via NUCDF and the UCDC to reflect variations in location and specialty type, including metabolic disease physicians, gastroenterologists, genetics counselors, and dietitians. This accounted for the assumption that hospital centers may vary in terms of their treatment philosophy and approach to counseling and that different individuals on the care team interact with patients with UCDs at different points throughout their decision-making experience. Provider participants were then selected through theoretical sampling as discussed previously. The characteristics of caregiver and provider interview and focus group participant samples are summarized in Tables 2 and 3. Both parent and provider participant recruitment continued until multiple investigators concluded that the saturation of information had occurred. Focus group recruitment followed a similar strategy.

Table 2

Characteristics of Interviewed UCD Caregiver Participants and Children.

Table 3

Characteristics of UCD Provider Interview Participants (N = 26).

The study team recruited a representative sample of 26 providers and 35 primary caregivers for participation in semistructured qualitative interviews. We chose an additional 19 primary caregivers to participate in focus groups, which helped validate information obtained through earlier interviews. The focus groups included 8 to 12 participants each, stratified by transplant status.

Analytical and Statistical Approaches

The study team used thematic content analysis to identify key patterns within the data and to categorize collected information into recurrent or common themes. Data content analysis was based on Strauss and Corbin's systematic procedures for grounded theory work.⁸² We conducted initial data abstraction through the line-by-line open coding of a cross-section of 14 interview transcripts by 3 to 4 coders, which allowed key issues to emerge directly from the collected data and ensured that important aspects of this phenomenon were not precluded through the use of a more selective coding scheme. We used open coding to generate a preliminary set of codes, which we continuously refined through research team consensus until saturation was reached and a final structure of codes and subcodes emerged. We then systematically applied this coding structure across all transcripts. Each transcript was coded by at least 2 coders (often by 3-4 coders) with substantial experience in qualitative methods and analysis procedures. After each round of coding (ie, approximately 4-6 transcripts per round), all coders met to discuss the workability of the current coding structure, propose refinements and additions to the coding scheme, and troubleshoot variations in coding. Data collection and analysis, as described in the sections above, were conducted through an iterative process. The study team conducted coding and consensus-based analysis alongside additional participant recruitment and interviewing so that the developed findings could inform additional rounds of data collection, which in turn would validate existing themes, provide additional context and nuance to identified concepts, and reveal new areas for exploration. This continued until all investigators agreed that data saturation had been reached. Coding progress was tracked through a working spreadsheet. Interrater reliability was checked periodically through randomly selected coder comparisons.

We used selective (defining core variables) and axial coding (identifying relationships in the data) techniques to move beyond a typology of participant accounts in additional scholarly efforts that augmented the initial aim of the study. (Decision-making in UCD formed the basis of Dr Maya Gerstein's PhD dissertation, and her work, which includes 2 as yet unpublished manuscripts, expands our understanding of this topic.) Through selective and axial coding, we identified core concepts, exploring the relationship between key themes, and ultimately built a framework that describes how patients with UCDs and their families reach treatment decisions and the key clinical, social, and/or system-level factors that influence this process. To facilitate this level of analysis, we used common qualitative analysis techniques such as charting (ie, reorganizing data according to thematic content in side-by-side charts to visualize/compare a range of perspectives across patients), mapping, and interpretation (ie, using diagrams and tables to physically explore the relationship between themes⁸³). The study team examined differences and similarities in reported experiences to facilitate the identification of issues that are common across groups as well as the factors that create differences in the way patients and their families approach and experience this decision. We managed all interview and focus group data using NVivo 11 software (QSR International).⁸⁴

Data Collection

Qualitative data were collected directly from caregivers of children affected by UCDs (n = 31) and their clinical providers (n = 19) through a total of 8 semistructured focus groups (2 in-person and 2 web-based caregiver focus groups, and 1 in-person and 3 web-based provider focus groups). Focus groups were conducted in sessions lasting approximately 90 minutes. The focus group sessions were recorded and transcribed verbatim for use in analysis.

Focus group guides were developed by integrating findings from a limited relevant evidence base and initial discussions with key informants from NUCDF, as well as from the patients and providers previously interviewed for aim 2 of the study. Draft guides were reviewed by key informants, including families with children affected by UCDs and metabolic genetic physicians specializing in UCDs. The guides were revised and refined based on feedback from this group before being used in the field.

Sampling

The target population included caregivers whose children were born in the United States after 1996 and diagnosed with 1 of 4 UCDs (ASLD, ASSD, CPS1D, or OTCD) for which LT is a consideration. Stratified purposeful sampling methods were used to recruit caregivers identified by NUCDF from the patient community. Recruitment focused on identifying participants who varied in terms of (1) disease severity (ie, neonatal vs late onset) and (2) transplant status (ie, MM vs LT).

Stratified purposeful sampling was also used to recruit a national cross-section of UCD providers for participation in focus groups. Providers were recruited via the UCDC and NUCDF to reflect variation in location and type of provider, including metabolic disease physicians, gastroenterologists/hepatologists, genetics counselors, advanced practice nurses/nurse practitioners, and dietitians.

Data Analysis

Initial data abstraction was conducted through the line-by-line open coding of 2 focus group transcripts, which allowed key issues regarding dissemination of evidence and information-sharing to emerge directly from the collected data and ensured that important aspects of this phenomenon were not precluded through the use of a more-selective coding scheme. Open coding was used to generate a preliminary set of codes, which was continuously refined until a final structure of codes and subcodes emerged. This coding structure was then applied systematically across all focus group transcripts. Thematic content analysis was used to identify key patterns within the data and to categorize collected information into recurrent or common themes, which are described in the Background section of this report.

All focus group data were managed and analyzed using NVivo 11 software.

Sample Characteristics

A total of 31 caregivers participated in 4 focus groups. Dissemination and use of evidence was the only topic of discussion in 2 of these focus groups. NUCDF staff organized and observed the 2 in-person focus groups and assembled the 2 web-based focus groups, and they provided the web platform for holding the focus groups. Of the 24 providers who initially agreed to participate in the focus groups, 19 clinical providers participated, including 11 providers in the in-person focus group; all 4 provider focus groups focused exclusively on dissemination and use of evidence.

Development of Dissemination Strategy Document

Development of a caregiver- and provider-informed dissemination strategy followed a 3-step process. First, the research team used the information collected and analyzed as described previously to produce an outline of a dissemination strategy document, which was reviewed and refined by NUCDF. Second, the research team and NUCDF collaborated on drafting, refining, and finalizing the draft strategic document, sharing editorial control over the project. Third, the draft document was shared with the entire study team for review and final input before being submitted to PCORI as a deliverable for aim 3 of the project.

PS-Based Analysis

Figure 4 illustrates the process of applying predefined selection criteria to create the PCORI-eligible sample from the merger of UCDC LS data and data from participants enrolled directly in the protocol. Unfortunately, the study team was unable to enroll additional patients from the SPLIT registry primarily because eligible SPLIT registry participants were already enrolled in the LS. The 810 total UCD registrants were trimmed to 283 eligible patients: 179 patients receiving MM and 104 patients who had undergone LT. We selected North American, early-onset patients (onset within 28 days of birth) with an eligible diagnosis of CPS1D, OTCD, ASSD, or ASLD. The MM patients were further trimmed by excluding patients (n = 22) who died before the earliest age of transplant, 16 days, in the LT group. The 2 groups were further refined to ensure comparable severity by selecting participants whose first recorded HAE occurred by age 3 years, leaving 87 MM patients and 101 LT patients for analysis.

Figure 4

Selection of PCORI-Eligible Cohort.

Assignment of Index Age

To ensure equal treatment, we assigned MM patients an age that was distributed similarly to the age of transplantation in the LT group (both ages are referred to as the index age). We used pscore to assign an initial PS based on the following criteria: decade of birth, UCD type (proximal vs distal), method of diagnosis (clinical signs, family history, newborn screening), presence and type of symptoms at diagnosis, age at UCD onset, age at first HAE, and maximum ammonia level during the first HAE. Each of these criteria related directly to early UCD control or severity.

This process, based on pscore in Stata, achieved satisfactory covariable balance, approaching 0% bias across covariates based on the creation of 5 PS strata following the exclusion of 1 nonoverlapping MM patient (Figure 5). Random index-age assignments within PS strata were allocated based on the γ distribution in the MM group to mimic the distribution of stratum-specific ages of transplantation as indicated in Figure 6. It was necessary to make minor adjustments to the random assignments to ensure that the index age was not greater than the age of death or greater than the last QOL or NPT assessment.

Figure 5

Covariate Balance in Initial PS Used for Assignment of Index Age.

Figure 6

Comparison of Index Age in MM With Transplant Age in LT.

Generation of Final PSs Used in Analysis

The assignment of comparable index dates in the MM and LT groups allowed for the use of more detailed covariable indicators of UCD severity. This led to a better balance in a comparison of outcomes between treatment groups. UCD severity indicators included main effects, higher-order terms to account for nonlinearity, and interactions between main effects to account for nonadditive effects. The main effects included UCD type, ln(count of HAEs), percentage of HAEs with ammonia levels of ≥400 μmol/L and between 150 and 399 μmol/L, ln(maximum HAE LOS), and ln(count of HAE LOS >8 days), as well as level of parental education and patient birth decade.

Other effects included squared terms and interactions of main effects based on the addition of the term with the highest LR ≥2.71 to the evolving logistic model. This second-level process resulted in the addition of squared terms for birth decade and maximum HAE LOS and interactions between the numbers of HAEs with caregivers having “no college”; birth decade with percentage of HAEs with ammonia levels between 150 and 399 μmol/L; maximum LOS with caregivers having “some college”; UCD type with number of HAEs having an LOS of >8 days; and percentage of HAEs with ammonia levels at 150 to 399 μmol/L with number of HAEs having an LOS of >8 days, as well as with birth decade squared.

The distribution of PSs was used to define the region of overlap, which is the region of comparability between MM and LT treatment groups (also called the common support region), and to identify strata that provided the best balance between the 2 treatment groups. Figure 7 provides a breakdown of those participants included (on support) and not included (off support) in the common support region based on 3 PS strata. Those regions highlighted in blue and gold (off support) represent participants outside the common support region, for which there were no members of the other treatment group with comparable PSs. Achieving overlap (ie, group comparability) requires a substantial loss of numbers (LT, n = 43; MM, n = 35) at the extremes of PSs, leaving a small number available for matching in the lower and upper strata of the common support region (Figure 7). Thus, only 109 (58%) of the 187 original participants selected on the basis of increased severity are considered sufficiently comparable to contribute to the comparison of study outcomes. The small comparable groups, n = 51 in MM and n = 58 in LT, preclude subgroup analysis and leave the study able to detect only large differences in outcomes between groups.

Figure 7

Common Support Region Based on 3 Strata Defined on the PS.

Using criteria defined by Rubin,⁶¹ Table 4 and Figure 8 provide an assessment of the overall and variable-specific covariate balance achieved among participants in the common support region. The median level of percentage bias (Table 4) overall is reduced from 41.9% to 8.6% based on the PS definition of the common support region. Only log(number of HAEs) and the interaction of parents with no college by log(number of HAEs) raised moderate concerns about the level of within-covariable bias. The other covariables used in PS creation attained levels of bias reduction that raised no concerns. This is also illustrated in Figure 8. More detailed evaluations of overall balance within strata and by each covariable used in propensity scoring are included in Appendix A. Those MM and LT patients included in the common support region are referred to here as the CSS.

Table 4

Results of Bias Adjustment Using PSs to Define the Common Support Region.

Figure 8

Balance Between MM and LT Following Propensity Scoring.

Evaluation of Participants Included and Not Included in the CSS

Tables 5 and 6 compare the clinical and demographic characteristics of the 3 groups formed from the PS. In addition to participants in the CSS, these groups include MM participants with PSs lower than any member of the LT group and LT participants with PSs higher than any member of the MM group. These groups differ in many indicators of UCD severity (Table 6), including type of UCD, counts of HAEs, counts of HAEs with LOS >8 days, and proportion of HAEs with ammonia levels of >399 or 150 to 399 μmol/L, as well as maximum HAE LOS and maximum HAE ammonia levels and frequency of coma during HAEs. By all indications, the nonoverlapping, high-PS LT group has considerably more severe UCDs than does the nonoverlapping, low-PS MM group. The high-PS LT group also tends to have lower parental education and is more likely to have birthdates between 2000 and 2009.

Table 5

General Characteristics of Participants Included and Not in the CSS.

Table 6

Specific UCD Severity-Related Characteristics of Participants in and Not in the CSS.

Evaluation of Comparability of MM and LT in Total and CSS Participants

Tables 7 and 8 compare MM and LT CSS participants with regard to indicators of UCD severity. As in Tables 5 and 6, the greatest differences among total eligible participants occurred in specific indicators of the frequency and severity of HAEs, especially in terms of the type of UCD, degree of ammonia elevation, and LOS. These highly statistically significant differences provide strong evidence of greater severity in the LT group than in the MM group. In the CSS, these differences are greatly reduced in magnitude and in statistical significance. Only the frequency of HA episodes remains a statistically significant difference. Level of ammonia elevation and the frequency at which different levels of elevation occur remain borderline different between MM and LT. These seem to be the strongest indicators of UCD severity. The CSS alleviates most of the differences in severity, but this level of balance comes at the price of sample size reduction, as the CSS (n = 109) excludes >40% of the total sample of 187, which compromises the power to detect all but large differences between treatment groups in study outcomes. In lieu of having the power to test aim 1 hypotheses, our results that represent fair, mainly unbiased comparisons are best used to generate inferences that will need to be tested in larger and comparably unbiased samples.

Table 7

Comparison of MM and LT Participants in the Total Sample and CSS.

Table 8

Comparison of Pre–Index Date UCD Severity Indicators in the Total Eligible Sample and CSS in 28-Day-Onset MM and LT Participants.

Analysis of the LT Procedure and Complications in Those Included and Not Included in the CSS

Table 9a (initial transplant) and Table 9b (second transplant) compare LT procedures and complications between those included and not included in the CSS. Of patients who underwent LT, 89% required a single transplant, 9.6% required 2 transplants, and 1.1% required 3 transplants. Except for the covariables highlighted below, those included and not included in the CSS were generally similar. There were borderline differences (P = .09) in the type of transplant, with cadaveric (orthotopic) being the most common and relatively balanced type, accounting for approximately 60% of those included and not included in the CSS. Those not included in the CSS were more likely to receive a living-donor liver or a transplant of unknown source. Those included in the CSS were more likely to receive split or orthotopic and split livers. In both the first and second transplants, infections were more common among those in the CSS. There were no complications reported for the third transplant of unknown source.

Table 9a

Procedures and Complications of the Initial LT by Inclusion Status in the CSS.

Table 9b

Procedures and Complications of the Second LT by Inclusion Status in the CSS.

Evaluation of Post–Index Date Metabolic Outcomes in the CSS

Among those of similar severity, the post–index date experience described in Table 10 differed between the LT and MM groups. The duration of follow-up differed between the groups, especially in those with <1 year of follow-up, which was more common in the MM group; 3 to 6 years of follow-up was more common in the LT group. Approximately 53% of both groups experienced follow-up for at least 6 years. Excluding those followed for <1 year, a period complicated by recovery from transplant surgery and not balanced between the groups, the clinical experiences of the 2 groups were divergent in every outcome related to UCDs. The MM group continued to experience HAEs, including 19% who experienced an average of >9 post–index date HAEs. This experience is in sharp contrast to that of the LT group, which experienced no HAEs and thus escaped continued exposure to the most harmful effects of ammonia elevation.

Table 10

Post–Index Date Metabolic Outcomes in the CSS.

Treatment-Related Effects on Mortality, QOL, and Neuropsychological Outcomes

Comparison of mortality in the CSS

There were 16 deaths in 109 persons followed collectively for 822 person-years (PY), for a total post–index date risk of mortality of 19.5 per 1000 PY in Table 11. The comparable crude mortality rate in those receiving LTs was 18.4/1000 PY, and it was 20.7/1000 PY in MM participants, for a relative risk of 0.81 (P = .88). Taking into account the time of death and adjusting for PS and index age differences as well as differences in the number of pre–index date HAEs, Cox regression analysis showed evidence of a difference in the risk of mortality. The estimate of the HR increased to 1.5 (Figure 9), suggestive of greater mortality in LT, but failed to reach statistical significance (P = .28). This estimate was largely unchanged with regard to the strength of association or statistical significance in an analysis stratified by the 3 PS strata. Subgroup analysis of the relationship in Cox regression models in the LT group between mortality and age at transplantation in LT showed little evidence of a meaningful impact of age at transplantation on the risk of mortality.

Table 11

Risk of Mortality in LT- and MM-Treated Patients in the CSS.

Figure 9

Survival Curve Based on Stratified PS and Covariate-Adjusted Cox Model.

Interpretation of covariate balance plots

In the estimation of ATTs for QOL and neuropsychological outcomes, balance plots adjacent to the results for each outcome show the success in achieving covariate balance (bias reduction) between LT and MM samples during comparisons. An orientation to plot interpretation is provided here using text and accompanying templates.

Figure 10 presents the layout of these 2-panel side-by-side box plots that are meant to show the degree of comparability and bias reduction, between treatment groups. Each panel shows the degree of MM (untreated) to LT (treated) overlap of PS distributions. The left panel with the heading “Raw” shows the raw CSS overlap, and the right panel shows the overlap after PS matching and controlling for key differences during the kmatch analysis to estimate the average treatment effect (ATT [LT − MM difference]) associated with each outcome. Each box plot includes the box itself, whose upper edge represents the 75th percentile and whose lower edge represents the 25th percentile; the horizontal line in the box represents the median (50th percentile). The whiskers that extend beyond the box represent the upper and lower extremes of the data, excluding outliers that are presented as dots above and below the whiskers. The degree of improvement in group overlap from the left panel to the right panel indicates the extent of bias reduction and the confidence that can be placed in the ATT results.

Figure 10

Distribution of the PSs in MM (Untreated) vs LT (Treated) Groups.

Comparison of QOL in the CSS

In the unadjusted results from the CSS (Table 12), among those individuals providing QOL assessments (39-76 of 109 patients), patients and parents reported patient QOL as modestly lower in the LT group than in the MM group, based on the total, psychosocial, and physical scores. None of these results excluded chance as the cause of the difference, but the direction of results was consistent. In contrast, parents reported that the family's QOL was higher with LT than with MM, and this difference cannot be fully explained by chance. Figure 11 compares the unadjusted, PS-adjusted, and the PS ridge-matched and adjusted QOL ATT results in the MM- and LT-treated groups. Ridge-matched adjustment variables included age at QOL assessment and index age. The highly comparable ridge-matched and adjusted results based on covariate balance plots indicate little to no difference in the total QOL and its components of psychosocial health and physical health. However, estimates suggest a sizable (15-30 point) advantage in family QOL score, but chance cannot be ruled out as the reason. The covariables included in PS generation were the same for all QOL analyses. The smaller sample size for family QOL assessments is due to its later inclusion in the QOL test battery, not due to family choice; thus, it cannot contribute to bias or lack of generalizability of the findings. The validity of these results depends not only on the comparability of treatment groups, which has been achieved through PS matching and covariate adjustment in the CSS, but also that participants not providing QOL assessments overall and by treatment group do not differ substantially from those providing assessments. Table 13 indicates that persons providing and not providing QOL assessments are generally similar in the main indicators of UCD severity, and there is little evidence of an interactive effect between group and missing assessments. Therefore, summary P values are presented, reflecting evidence of general differences in characteristics between those providing and not providing QOL assessments. These indicate evidence of differences by UCD type and somewhat by parental education, which should not interfere greatly with validity or generalizability of the results. Table 14 shows an evaluation of the relationship between age at LT and QOL score. Except for QOL associated with physical health, where the best QOL score occurs in patients undergoing LT before 1 year of age, there is little evidence that implementing LT at an earlier age improves the QOL score.

Table 12

Unadjusted QOL Results by Treatment Group in CSS.

Figure 11

Matched and Adjusted QOL Results in LT vs MM in CSS.

Table 13

Comparability of Persons Providing and Not Providing QOL Assessments in the CSS.

Table 14

Adjusted Relationship Between Age at LT and QOL in CSS.

Comparison of Neuropsychological Outcomes by Domain in the CSS

All neuropsychological outcomes are scaled to a mean of 100 and an SD of 15. The most complete neuropsychological outcome data are in the domains of global and language function, which can be assessed in those younger and older than 3 years. Except for language function in those <3 years (Table 15), the unadjusted results in the CSS indicate slightly higher scores and higher levels of function in the LT group. Additional analyses were conducted to further reduce sources of bias and confounding in LT vs MM comparisons in the CSS using PS matching and/or PS and covariable adjustment (Figures 12 and 13). Although none of the differences between MM and LT achieved statistical significance, these better-balanced analyses provide consistent results that favor the LT group in terms of global and language function. The estimated LT advantage in both outcomes tends to be greater in the more precise assessments conducted at older ages (>3 years) with a better PS balance between groups. The analyses also evaluated whether those not providing NPT results differed from those providing NPT results, which would affect validity and generalizability (Table 16). In the absence of evidence of differential effects by treatment group, those providing and not providing neuropsychological data presented borderline differences by type of UCD and percentage of HAEs with ammonia levels between 150 and 399 μmol/L. These differences do not negate group comparisons but raise caution regarding their validity and generalizability. Table 17 evaluates the relationship between age at LT in the transplanted group and global and language function. There is an indication, although not supported by statistical significance, of a general tendency toward better function at earlier ages of LT, especially during the first year of life.

Table 15

Unadjusted Global and Language Results by Treatment Group in the CSS.

Figure 12

PS-Matched and Adjusted Global Functioning Results in LT vs MM in the CSS.

Figure 13

Matched and Adjusted Language Results in LT Compared With MM in the CSS.

Table 16

Comparability of Participants Providing and Not Providing NP Assessments in CSS.

Table 17

Adjusted Relationship Between Age at LT and NP Assessment of Global and Language Functioning in the CSS.

Because of greater missingness, there is generally less information across the other neuropsychological domains, including visual, motor, attention/executive function, and emotional/behavioral domains, and there are also no statistically significant differences between LT and MM groups with regard to these (Appendix A, Tables A3-A12). In the following summary, except where noted, higher scores indicate better performance. There tended to be less consistency across tests measuring similar or overlapping attributes, likely due in part to poorer covariate balance from test to test. There were, however, some patterns that bear mention. Most consistent was a tendency toward higher scores that indicate worse emotional and behavioral outcomes in those receiving LT (Appendix A, Table A13) and a tendency toward better results (lower scores) with later age at LT in these outcomes (Appendix A, Table A14). Covariate-balanced results based on ridge matching and adjustment indicate that visual and motor function differed based on the test (Appendix A, Table A4). There was a tendency for performance IQ, which is a measure of visual reasoning, to be better in the LT group, but those receiving LT performed worse on direct assessments of visual-perceptual skills (Appendix A, Table A4). Earlier age at LT was associated with improved visual-perceptual skills (Appendix A, Table A5). Motor assessments were also inconsistent. Motor strength (grip strength), but not fine motor speed and dexterity (grooved pegboard), tended to be better in the LT group (Appendix A, Table A7) than in the MM group. The grooved pegboard results tended to improve with earlier LT, but the pattern was inconsistent for grip strength (Appendix A, Table A8). Attention/executive function did not produce consistent results by group or by age at LT (Appendix A, Tables A9-A11).

RS Matching Results With Imputation

LT generally occurs early in life (Figure 14), with 49% of surgeries occurring during the first year of life, 20% of surgeries during the second year of life, and 78% of surgeries by 3 years of age. In the eligibility set, those who received LT had more hospital admissions than did those receiving MM in the first year of life (Figure 15).

Figure 14

Cumulative Probability of Age at LT.

Figure 15

Overlap in Log Age at Hospitalization Between MM and LT.

The RS-matching method finds a comparable control group by first identifying the RS and then matching on important characteristics in that RS. A good diagnostic for whether the matched control group is comparable with the LT group is to examine balance after matching. One tuning parameter for matching, in our implementation, was the Euclidean distance caliper, where a larger caliper may lead to more control matches for each LT patient but worse balance, whereas a smaller caliper may lead to no matches for some LT patients but better balance. As expected, we see in Table 18 that a wide caliper for the Euclidean distance kept all those who received LT, but the control group was generally less severe (ie, fewer HAEs, lower peak ammonia level) than the LT group. Medical history variables were comparable but were higher in the LT group than in the matched control group. The balance was improved by narrowing the caliper, but in doing so, only 80% of the LT patients remained. Reducing the caliper further reduced the control group size and had a negligible impact on balance. Figure 16 shows a positive association between the total LOS and ammonia levels. Although the LT group and the control group cluster around the same values, the LT group tends to have longer hospital stays than does the control group. Thus, this method did not fully account for confounding by disease severity by matching, and we still adjusted for medical history variables in the outcome models.

Table 18

Cohort Baseline and Follow-up Characteristics by LT Status at Matching Time and After Matching (Narrow and Wide Calipers in RS-Matching Method).

Figure 16

Association Between Peak Ammonia Levels and Total Cumulative LOS for HAEs.

The results in Table 19 show little difference between the LT group and their matched controls for the outcome of mortality and QOL after accounting by confounding with RS matching and missing values by multiple imputations. The results for each imputed data set are shown in Appendix A (Figures 2A-6A).

Table 19

Effect of LT on Outcomes, RS Matching With Narrow Caliper.

Context of Limited Empirical Evidence

Interviews and focus groups with caregivers and providers captured the perception of limited outcomes data. Insufficient empirical evidence and ill-defined clinical guidelines hamper decision-making related to the choice between MM and LT for children with UCDs. Caregivers and providers alike described the challenges of treatment decision-making against a backdrop of high uncertainty, and detailed a decision-making experience largely defined within this context (Appendix B, Box 1 [1a-c]). We constructed the decision-making framework and its individual and interrelated components within this landscape. The framework represents the shared experience of families affected by UCDs who must pursue intervention in the absence of clear, evidence-based information.

UCD Treatment Choice Framework

The study team identified key themes through an analysis of caregiver and provider interviews and focus group data, which represent the shared decision-making experience of the study cohort. We positioned these themes within a holistic framework illustrating how elements of the decision-making experience interrelate and influence the decision between MM and LT (Figure 17).

Figure 17

Conceptual Framework Describing Key Factors Contributing to the Decision Between MM and LT Among Families of Children With UCDs, Within a Context of Limited Empirical Evidence and Poorly Defined Clinical Guidance.

The relative risks and benefits of MM vs LT were considered and were a central component of the decision-making experience. Caregiver participants described efforts to understand and analyze the risks and benefits of these treatment alternatives in an attempt to make informed treatment decisions (Appendix B, Box 2 [2a]). With limited empirical evidence informing the choice between MM and LT, caregivers and providers commonly discussed struggles in weighing treatment alternatives and the challenges of making a choice (Appendix B, Box 2 [2b-d]).

In the absence of uniform clinical guidance, families affected by UCDs often rely on their own experience and those of providers and peers as inputs of imperfect information guiding the choice between MM and LT. Participants discussed these inputs as an interrelated collection of complex clinical, personal, social, and system-level factors. These factors, found on the 4 cardinal points of the framework, informed each family's personal perception of the risks and benefits of MM vs LT, and ultimately, their decision to pursue or not pursue LT as a treatment for their child.

Phases of Childhood and Developmental Milestones

Interviews and focus groups highlighted the idea that changes during key developmental milestones throughout a child's life precipitated new or aggravated existing challenges associated with the MM of UCDs. Developmental milestones acted as a catalyst for caregivers to consider or reconsider LT as a viable treatment choice. As a child moves from infancy to early childhood, caregivers must contend with new feeding challenges, including a transition to solid foods (Appendix B, Box 3 [3a-b]). They may also face adherence issues once their child is able to refuse medications, formulas, or other forms of nutrition (Appendix B, Box 3 [3c]). Caregivers may have an increasingly difficult time protecting their child from viral exposures as they transition to preschool and grade-school age and become increasingly more independent and interactive with others (Appendix B, Box 3 [3d]). As children enter grade school age, caregivers described new challenges related to their child's transition to school, such as concerns over management of the disorder in a school setting and worries about their child's ability to forge peer relationships and participate in “normal” childhood activities (Appendix B, Box 3 [3e]). Caregivers whose children had transitioned into adolescence and early adulthood cited new adherence issues (Appendix B, Box 3 [3f]), questions about their child's long-term independence, and rising concern about their child's ability to manage their own medical needs (Appendix B, Box 3 [3g-h]). Data suggest that the clinical, personal, social, and system-level factors that influence treatment choice manifest differently across these key phases of childhood. Thus, we include these major developmental transitions and milestones on the “x-axis” and “y-axis” of the framework in Figure 17 to indicate that they may change priorities and reframe parents' perception of risks and benefits. These axes do not operate as a typical ordinate but rather are included to represent the impact of child development and life stages on the treatment decision landscape.

Tipping Point

Caregivers in interview and focus group cohorts who had chosen LT as a treatment for their child all reached a “tipping point” in their evaluation of the risks and benefits of LT vs MM. Ultimately, each of these families felt unable to continue the management of their child's disorder through diet and medication. This tipping point led them to pursue LT as a treatment alternative (Appendix B, Box 4 [4a]). Interviews with patients and providers highlighted variations in how families approached this decision and the conditions under which they would entertain LT as a viable treatment alternative for their child. If, when, how, and for what reason families reached this conclusion varied within the study cohort and among provider and participant panels. Among families affected by UCD, some described their tipping point being shortly after diagnosis. Some families never reached a point where they felt LT was a true consideration. Other families faced their tipping point only after several years of MM and key changes in circumstances (Appendix B, Box 4 [4b-d]). The family's overall tolerance for the uncertainty and ambiguity that accompanies MM seemed to factor into this timeline (Appendix B, Box 4 [4e]), as did the child's clinical status, the personal burden of disease on the family, social implications of the illness, and the patient's experience with the health care system. Together, these represent the landscape within which families affected by UCDs evaluated available treatment choices and reached or did not reach a tipping point in favor of LT over MM.

Clinical Factors

Personal Factors

Social Factors

System Factors

Table A1. Results of Balance and Bias Reduction Achieved per Each Propensity Score Variable (PDF, 139K)

Table A2. Stratum Mean Propensity Score Balance within Three Propensity Score Stata (PDF, 373K)

Table A3. Unadjusted Visual Skills Result by Treatment Group in Common Support Sample (PDF, 62K)

Table A4. Matched and Adjusted Visual Skills Result in LT Compared to MM (PDF, 329K)

Table A5. Relationship between Age at Liver Transplant And Visual Skills (PDF, 56K)

Table A6. Unadjusted Motor Skills Result by Treatment Group in Common Support Sample (PDF, 35K)

Table A7. Matched and Adjusted Motor Skills Result in LT Compared to MM (PDF, 317K)

Table A8. Relationship between Age at Liver Transplant And Motor Skills (PDF, 63K)

Table A9. Unadjusted Attention/Executive Function by Treatment Group in Common Support Sample (PDF, 59K)

Table A10. Matched and Adjusted Attention/Executive Function in LT Compared to MM (PDF, 394K)

Table A11. Relationship between Age at Liver Transplant And Attention/Executive Function (PDF, 67K)

Table A12. Unadjusted Emotional/Behavioral Function by Treatment Group in CSS (PDF, 31K)

Table A13. Matched and Adjusted Emotional/Behavioral Function in LT Compared to MM (PDF, 202K)

Table A14. Relationship Between Age at Liver Transplant And Emotional/Behavioral Function (PDF, 50K)

Table A15. Number and Percent of Missing Values Per Characteristic Among All Eligible (PDF, 88K)

Figure A1. Multiple Imputations to Impute Missing Characteristics (PDF, 116K)

Figure 2A. Estimated Average Treatment Effect +/− 95% CI by Imputation in LT for All-Cause Mortality (PDF, 104K)

Figure 3A. Estimated Average Treatment Effect ± 95% CI by Imputation in LT, Total Quality of Life (QOL) (PDF, 61K)

Figure 4A. Estimated Average Treatment Effect ±95% CI by Imputation in LT for Psychosocial Health QOL (PDF, 62K)

Figure 5A. Estimated Average Treatment Effect ±95% CI by Imputation in LT for Physical Health QOL (PDF, 64K)

Figure 6A. Estimated Average Treatment Effect ± 95% CI by Imputation in LT for Family QOL (PDF, 40K)

Box 1. Context of Limited Empirical Evidence (PDF, 90K)

Box 2. Weighing the Relative Risks and Benefits of Available Treatment Options (PDF, 51K)

Box 3. Phases of Childhood and Developmental Milestones (PDF, 77K)

Box 4. Tipping Point (PDF, 79K)

Box 5. Disease Severity (PDF, 98K)

Box 6. Disease Stability (PDF, 82K)

Box 7. Burden on Familial Unit (PDF, 61K)

Box 8. Burden on Child (PDF, 60K)

Box 9. Peer-to-Peer Interaction (PDF, 80K)

Box 10. Considerations for Child's Independence (PDF, 83K)

Box 11. Access to Quality Metabolic Care (PDF, 79K)

Box 12. Metabolic Physician Approach to Treatment and Guidance (PDF, 80K)

Box 13. Cost and Coverage of UCD (PDF, 86K)