An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Christopher B. Forrest, MD, PhD, Lisa J. Meltzer, PhD, Carole L. Marcus, MBBCh, Anna de la Motte, MS, Amy Kratchman, BA, Daniel J. Buysse, MD, Paul A. Pilkonis, PhD, Brandon D. Becker, PhD, and Katherine B. Bevans, PhD.
Author Information and AffiliationsDespite extensive evidence that supports sleep as an integral part of child health, no widely adopted self-reported measure of pediatric sleep exists. The NIH-sponsored PROMIS® initiative has produced adult sleep health measures, but pediatric companion instruments have not been developed.
To develop and evaluate the validity of pediatric versions of the PROMIS Sleep Disturbance and Sleep-Related Impairments scales.
This project adhered to PROMIS standards and relevant PCORI Methodological Standards for developing person-reported outcome measures. The pool of items that assess a child's sleep health developed from the item concepts and their expressions from the PROMIS adult sleep health item banks. Additional items were obtained through interviews with sleep medicine experts (n = 8), children aged 8 to 17 years (n = 18), parents of children aged 5 to 17 years (n = 33), and a systematic literature review of all existing pediatric sleep health instruments (n = 329).
Samples of 1104 children aged 8 to 17 and 1477 parents of children aged 5 to 17 years from the general population were obtained from a national internet panel. Statistical and psychometric testing and item response theory (IRT) item calibration were performed using data from these 2 samples. Clinical validity of the final measures was assessed in children with sleep problems recruited from a sleep referral clinic (n = 128) and 9 other chronic and neurodevelopmental condition samples (n = 491).
A total of 5015 participants—1773 children and 3242 parents—contributed to this project. Content validation of the resulting item pool resulted in 43 items. Psychometric evaluation and IRT analyses led to the deletion of 15 items. The final Sleep Disturbance item bank includes 15 items that assess difficulties with falling and staying asleep and with sleep quality. The Sleep-Related Impairment item bank includes 13 items that assess daytime sleepiness, waking up, and the impact of sleepiness on cognitive functioning, mood and behaviors, and daily activities. For each item bank, the researchers produced child self-report editions that can be used for children 8 to 17 years old and parent-proxy editions that can be used for children 5 to 17 years old. Validity of the item banks was supported by moderate correlations with existing measures of pediatric sleep health and significantly higher Sleep Disturbance and Sleep-Related Impairment scores for children with chronic and neurodevelopmental conditions and those with sleep problems.
The PROMIS Pediatric Sleep Disturbance and Sleep-Related Impairment item banks provide precise and valid assessments of a child's difficulties falling and staying asleep as well as level of daytime sleepiness and its impact on daytime functioning. They are ready for use in observational studies of sleep, in clinical trials for medications and devices that influence sleep, and in clinical practice.
Although the PROMIS sleep health item banks have excellent precision across a wide range of the latent variables, we have not evaluated their responsiveness to clinical change over time. Longitudinal studies that examine sleep health and its relationship to change in health status are a key direction for future research.
Patient-reported outcomes (PROs) provide assessments of individuals' lived experiences—that is, of how they feel, what they can or would like to do, and how they evaluate their health and lives. They are an important complement to clinical and objective evaluations of health, because they provide unique information that is directly obtained from the patients themselves. In 2004, NIH launched a program of research called PROMIS®. The goal of PROMIS was to develop and standardize PRO measures for clinical research and practice.1,2 NIH continues to fund ongoing dissemination of PROMIS measures via the healthmeasures.net website and the NIH (Pediatric Patient Reported Outcomes in Chronic Diseases) PEPR Consortium (peprconsortium.org), which is evaluating PROMIS measures in children with chronic conditions. (Note: In this report, we use the word child to refer to all individuals younger than 21 years.)
PROMIS uses a domain-specific rather than disease-specific measurement approach. Domains are clinically coherent, empirically unidimensional health concepts. The domain-specific approach is based on the perspective that many health concepts are universally experienced but that individuals with specific disorders may have characteristic profiles (patterns and severity of domain-specific scores).3 PROMIS tools are developed through a mixed methods process that includes domain conceptualization4; item concept elicitation through interviews of experts and participants and systematic review of existing measures5; content validation4,6; and psychometric testing and item response theory (IRT) calibration.7
PROMIS pediatric measures are developed with direct child and parent input.4,8 Semistructured interviews, focus groups, and cognitive interviews are conducted with children and parents to maximize the instrument's meaningfulness, developmental appropriateness, and understandability to respondents.2,6,9-11 PROMIS has produced more than 40 measures for adults and 20 for children; the latter include both child-reported and parent-proxy editions.
Each measure includes an item bank that can be administered as fixed-length questionnaires (usually 4 or 8 items) or as a computerized adaptive test.
There are 2 PROMIS sleep health instruments for adults: Sleep Disturbance (measures sleep quality and difficulties obtaining restful sleep) and Sleep-Related Impairment (measures wakefulness and sleepiness).12 One of the major gaps in the PROMIS portfolio is the absence of sleep health item banks for children. Healthy sleep is essential for children's cognitive functioning,13-15 emotional regulation,16-18 mood,18-20 immune function,21,22 school performance,23-25 and well-being.26,27 Sleep problems are common among children, particularly those with neurodevelopmental disorders and chronic conditions.
More than 300 sleep health PRO measures have been used in research on pediatric sleep.28,29 Based on a comprehensive review of these measures, Spruyt and Gozal concluded that just 2 (Sleep Disturbance Scale for Children30 and Sleep Disorders Inventory for Students31) have undergone adequate evaluation of their reliability and validity.29 One-third of existing measures were developed using factor analytic methods, and none was created using IRT, which provides information about a measure's precision for individuals with varying levels of severity. Additionally, nearly all measures rely on parental report despite discrepancies between child-reported and parent-reported sleep outcomes, particularly as children transition into adolescence.32-34 Whereas parents are accurate reporters of sleep-related behaviors that they observe (eg, child falls asleep quickly in the car), children may be better reporters of internal events (eg, daytime sleepiness) and covert experiences (eg, nighttime awakenings).
Pediatric PRO measures should assess concepts that are developmentally appropriate and meaningful to children and that use terminology that they understand.8,35-37 Indicators of optimal sleep health vary across the lifespan. For example, sleep timing and practices for younger children are primarily influenced by parents, while adolescent sleep timing is often determined by homework, extracurricular activity, social interactions, and early school start times.2 Item understandability is especially important for child-reported measures because the competencies needed to reliably and accurately self-report develop with age.38
Content validity is established through qualitative research that includes direct input from the target population.8,36 Of the 10 most commonly used PRO measures of children's sleep, only a single measure was developed with child input (Obstructive Sleep Apnea-18 Survey),39 and none was evaluated to assess respondents' understanding of the items.
The purpose of this report is to provide a summary document for PCORI peer review that addresses the 4 specific aims of the researchers project. Our overall goal was to develop pediatric sleep health measures according to PROMIS standards.
Specifically, our aims were the following:
We conducted this project in partnership with 20 representatives from 5 stakeholder groups:
We engaged youth (individuals between 13 and 24 years of age), parents, clinicians, and sleep health advocates to ensure that the final measures are relevant, understandable, and useful to future respondents and end-users. Four partners represented each stakeholder group to obtain a wide range of perspectives and ensure that no single group dominated decision making.
Researchers included the study principal investigator (PI), a co-PI, and 2 researchers who had previously developed the PROMIS sleep health measures for adults. Clinician partners specialized in clinical psychology, pediatric sleep disorders, and developmental-behavioral pediatrics. Advocates included professionals working toward sleep health-promotion public policies, such as later school start times. We identified parents and youth through the Children's Hospital of Philadelphia Family Partners program. Youth partners had significant histories of sleep problems and were aged 13, 14, 18, and 19 when the study began.
We structured the research partnership nonhierarchically to undermine the historic power stratification that separates researchers from other stakeholders. We invited youth, parents, clinicians, sleep health advocates, and researchers to join the partnership during the study planning phase (before submitting the project proposal to PCORI). At that time, we informed all potential partners of their role in the partnership, including expectations for their involvement and compensation. After agreeing to join, contributing stakeholders had the opportunity to review, edit, and vote on the partnership charter. The charter outlined the partnerships' guiding principles (respect, coproduction, quality, trust) and specified stakeholder expectations, communication methods, and decision-making procedures. A single leader from each stakeholder group served as a member on the research management team. Leaders were expected to consult with their groups and advocate on their behalf. Members of the research management team generally made decisions by consensus; on the few occasions when consensus could not be achieved, decisions were made democratically with each member having an equal vote.
Members of the research management team proactively contributed to the day-to-day study operations and decision making. Other stakeholder partners served as advisors, providing feedback on the study protocol and the sleep health measures. In addition to coproducing and following the partnership charter, other strategies for enhancing partner engagement included the following:
We aimed to develop content valid PROs that assessed children's sleep health. Content valid measures assess concepts that are appropriate, meaningful, and relevant to the target population in terminology that intended respondents can understand. Partners improved the measures' content validity by identifying missing content and offering suggestions for item revisions and deletions. For example, we developed a set of sleep practice items in response to clinician input that such a measure would have a high degree of clinical utility. Parents confirmed the importance of measuring sleep practice behaviors and ecology, and youth helped expand the concept to include items pertaining to technology use and other environmental factors that affect sleep. Partners also helped develop the study protocol for content validation procedures, including child and parent concept elicitation and cognitive interviews.
Youth, parent, clinician, and advocate partners helped devise participant recruitment and data collection methods. Partner feedback improved communication of the study's intent to potential participants, expedited IRB approval, and minimized the impact of recruitment and data collection on clinical workflow. In some contexts, clinical leaders and parents greatly enhanced participant accrual by expanding and refining recruitment methods (eg, use of social media platforms). Youth and parents suggested revisions to online surveys so that they were user-friendly. Clinician partners contributed to the selection of study measures. For example, clinicians helped refine the proposed use of polysomnography data and recommended alternative measures for characterizing children's sleep quality and chronic health conditions.
Research partners designed methods in accordance with rigorous PRO measure development and validation standards (eg, PCORI, COSMIN standards). Our team benefited from the expertise and experience of Drs Buysse and Pilkonis, who had previously developed PROMIS sleep health measures for adults. As described above, parent, youth, and clinician partners helped devise the study protocol. Partner recommendations improved our capacity to recruit harder-to-reach populations (eg, children undergoing sleep studies) and maximized the quality of data collected from children, parents, and other sources (eg, electronic health records). Partners helped interpret results of qualitative and psychometric analyses, which led to finalization of the measures.
Feedback from all stakeholder groups informed iterations of the PROs. Summary reports included stakeholder feedback and a list of final item selections, revisions, and deletions. All final decisions were explained by the researchers, which allowed partners to see their contributions and understand why, in some instances, their suggestions had not been used. The creation of these reports was the result of a suggestion made in the comments section of the second partnership evaluation.
The Interactive Autism Network and Rare Epilepsy Network administered the sleep health measures to a large sample of children and parents. We are currently finalizing aggregate-level reports for each of the clinical networks that field tested the measure; these reports compare clinical population scores to general population scores. We disseminated these reports to networks in August 2017. Members of our team presented the sleep health measures at 2 national conferences: the Sleep Research Network's annual meeting (October 2016) and Association of Professional Sleep Societies annual SLEEP conference (June 2017). Additional dissemination efforts include (1) presenting development/validation of the measures at the PROMIS Health Organization Conference (October 2017); (2) presenting validation of the measures at the Pediatric Sleep Medicine Conference (November 2017); (3) posting finalized sleep health measures on the PedSleep listserv (almost 400 pediatric sleep clinicians and researchers); and, (4) encouraging advocate partners (ie, Start School Later, National Sleep Foundation, Narcolepsy Network, MyApnea.org).
Two large NIH-funded research consortia have adopted the measures. The longitudinal construct validity and responsiveness of the measures are currently being tested in children with atopic dermatitis and chronic kidney disease (CKD) by research sites affiliated with the PEPR Consortium. In addition, the measures were integrated into the Environmental Influences on Child Health Outcomes (ECHO) program network-wide data collection protocol. Thus, the measures will be administered to many of the ECHO participants (projected to exceed 50 000 children).
Finally, public dissemination of the measures will be made possible when the measures are posted on the PROMIS Health Organization's healthmeasures.net website, which is the single repository for all PROMIS measures. We are completing our application for the review and quality assurance process that led to posting the measures in September 2017.
To develop the item banks, we adhered to the PROMIS methodological standards for person-reported outcome measure development, described in the Background section.2,40 PROMIS uses items (ie, questions) aggregated into item banks to measure a health domain. An item bank comprises items that measure various manifestations of the domain concept. The PROMIS approach for creating an item bank includes 6 major steps, listed below in the context of each of the project aims (Figure 1 displays the process followed for this project). For each step, we summarize the study populations and/or types of data collection involved.
Steps in the Development and Validation of the Sleep Health Item Banks.
PROMIS Steps: (1) Conceptual specification of the domain; (2) item pool construction; and (3) content validation
PROMIS Steps: (4) Survey administration to representative and clinical populations
PROMIS Steps: (5) Psychometric evaluation using classical test and IRT approaches
Children with sleep disorders—primarily insomnia and obstructive sleep apnea
Children with autism
PROMIS Steps: (6) Final product creation
The content validation of the PROMIS Pediatric Sleep Disturbance and Sleep-Related Impairment items followed a well-established set of methods.4,6,8 In this section, we discuss the methods used to generate the item pool and evaluate its content validity. Methods for content validation included a structured literature review, subject matter expert interviews, semistructured interviews with children and parents, cognitive interviews, a translatability review, and a reading-level analysis.
Figure 2 illustrates development of the child sleep health conceptual framework and item pool. With the goal of generating conceptually harmonized lifespan measures of sleep health, our initial set of sleep health concepts and expressions was informed by the PROMIS Adult Sleep-Wake measures.12 We generated a preliminary (version 1) child sleep health conceptual framework based on the 2 PROMIS Adult Sleep Health item banks: Sleep Disturbance includes perceptions of sleep quality, sleep depth, restoration associated with sleep, perceived difficulties and concerns with getting to sleep or staying sleep, and perceptions of the adequacy of and satisfaction with sleep; Sleep-Related Impairment includes perceptions of alertness, sleepiness, and tiredness during usual waking hours, and the perceived functional impairments during wakefulness associated with sleep problems or impaired alertness.12 We sorted the adult PROMIS Sleep Disturbance and Sleep-Related Impairment item concepts into distinct subcategories called facets and generated preliminary facet definitions. We further refined the sleep health concept framework based on expert (version 2), child, and parent interviews (version 3), and a systematic review of the literature (version 4).
PROMIS Pediatric Sleep Health Item Pool Development Process.
We turned item-level concepts into expressions. For example, we turned the item concept difficulty falling asleep into the following item expression: “In the past 7 days, how often did you have difficulty falling asleep?” In some cases, we developed more than 1 expression for a given item concept because we were unsure which wording would be best understood by children. For instance, we developed 2 items for the concept overall sleep quality: slept well and slept poorly. We iteratively refined the item expressions through cognitive interviews.
As part of the preliminary work for the grant application, we iteratively refined the initial pediatric sleep health conceptual framework through semistructured interviews conducted with 8 sleep medicine experts, each with a minimum of 3 peer reviewed publications in the area of children's sleep health. Interviewees represented the disciplines of developmental pediatrics, neurology, clinical psychology, neuropsychology, anthropology, and neuroscience. Four of the experts had experience with the development and validation of existing pediatric sleep measures. We asked the experts to review the PROMIS Adult Sleep Disturbance and Sleep-Related Impairment domain definitions and items before the interview. During the interview, we asked experts to define healthy sleep for children, identify common antecedents and consequences of poor sleep in children, and assess applicability of the PROMIS adult sleep domain definitions and items to children. Expert interviews were audio recorded, transcribed, and analyzed for themes. We added original themes not included in the initial framework (based on the adult sleep health item banks).
We designed child and parent interviews to elicit sleep health concepts—that is, their experiences sleeping and the effects of feeling sleepy or refreshed on daytime activities. We asked interviewees to describe instances when they (children) or their child (parents) had a “hard” and “easy” time sleeping. Specific probes elicited information about children's sleep and wake routines, factors that affect sleep, and the impact of sleep on child health and functioning. This part of the project was essential to ensuring that the sleep health concepts reflect the experiences of children with diverse health- and sleep-related experiences. Accordingly, we interviewed children and parents recruited from 3 patient care settings: primary care clinics (n = 7 children, 11 parents), a tertiary care specialty sleep clinic (n = 7 children, 8 parents), and a nephrology clinic (n = 14 children, 14 parents). Children recruited from primary care settings had no diagnosed sleep problems. Those recruited from the sleep clinic were undergoing diagnostic testing or treatment of sleep disorders. Children recruited from the nephrology clinic were being treated for CKD, which is associated with a high prevalence of sleep problems.41,42 No children with CKD were on dialysis. Interviews were audio recorded, transcribed, and analyzed for themes. We explored interview topics until saturation was achieved, separately for children and parents. Saturation was achieved when a sleep health concept was elicited in at least 1 but not the last interview and if enough information was derived from interview transcripts to fully understand the meaning and importance of the concept to children.8
We conducted a systematic literature review to identify child-report or parent-reported measures of pediatric sleep health. We conducted the search in MEDLINE, CINHAL, PsychINFO, and HaPI (Health and Psychosocial Instruments). We selected search terms to capture constructs of the sleep experience (eg, sleep disorders, wakefulness), self-report instruments (eg, self-report, self-assessment), and measurement (eg, questionnaires, health survey) within each of the databases. We limited searches to research involving children (< 18 years of age) and articles published in English. Two investigators reviewed the articles to identify sleep health PRO measures. We collected the 10 most commonly used measures from published works or instrument authors and catalogued the instruments' item-level concepts.
The purpose of the cognitive interviews was to identify problems with item comprehension, recall, and other cognitive processes that could be remediated through question rewording, reordering, or more extensive instrument revisions.6,43,44 Interviewees completed a paper-based questionnaire of 25 randomly selected items. After completing the questionnaire, they were asked to read each item aloud, restate the item in their own words, and explain their response. We conducted cognitive interviews with 32 children aged 8 to 17 years and 21 parents of children aged 5 to 17 years recruited from primary care clinics and a tertiary care specialty sleep clinic. We initially tested all items with 4 to 5 children (at least 2 of whom were 8-11 years old) and 3 parents. Cognitive interviews were audio recorded, transcribed, and coded based on the degree to which interviewees' understanding of the item was consistent with its intended meaning (1 = poor/different than intended; 2 = partial; 3 = fully consistent). Items with average ratings of less than 2 were removed or iteratively revised and retested using the same cognitive interview procedures until these items were adequately understood.
An expert in translation reviewed each item to identify idiomatic expressions, complex sentences, and concepts that are not easily translated. We removed from the item pool items deemed poorly translatable.
We reviewed items for readability by calculating the Flesch-Kincaid Grade Level equivalent using Microsoft Word software. The readability score analyzed and rated text on a US grade-school level, based on the average number of syllables per word and words per sentence.45 For example, a score of 8.0 means that an eighth grader would be expected to understand the text.
After the completion of Aim 1, we had an item pool composed of 43 items that were expressions of 6 Sleep Disturbance conceptual facets (ie, sleep onset, sleep continuity, dreams, breathing, parasomnias, and sleep quality) and 6 facets for Sleep-Related Impairment (daytime sleepiness; energy; sleep offset; and the impact of sleep on cognitive functioning, affect and behavior, and daily activities). Each facet included multiple item expressions for relevant concepts. Consistent with our prior pediatric PRO work, we used the following frequency-based response options (1 = never, 2 = almost never, 3 = sometimes, 4 = almost always, 5 = always). This ordinal scale is well understood by children. The item expressions used a 7-day recall period. Parent-proxy versions replaced the pronoun “I” with “my child.”
Aim 2 methods included a general population survey and clinical sample surveys. We expanded the original planned clinical validation approach, limited to children with autism spectrum disorder (ASD) and sleep disorders, to include additional clinical populations. We added data collection for the originally proposed ASD population as well as 9 additional clinical populations toward the end of the project. One of our clinician partners added atopic dermatitis and asthma populations. Through our work in PCORnet, we further expanded the number of clinical populations by collaborating with interested patient-powered research networks. These included Interactive Autism Network (autism); ImproveCareNow (inflammatory bowel disease); Genetic Alliance (rare genetic conditions, including WAGR syndrome, Cornelia de Lange syndrome, and RASopathies); NephCure Kidney Network (nephrotic syndrome and other kidney conditions); and Rare Epilepsy Network (seizure disorders). Expansion of these data collection efforts greatly exceeded the original recruitment targets and provided a unique opportunity to evaluate self-reported sleep health among children with chronic and neurodevelopmental conditions. Each survey sample is described below. All questionnaires were self-administered via computer, either home computers or in-clinic tablets.
We gave the final item pool, produced after removing items that were poorly understood or not translatable, to a sample of 1104 children aged 8 to 17 years and 1477 parents of children aged 5 to 17 years. We recruited these individuals from GFK Knowledge Panel, a dual-frame (random-digit dial and address-based) online probability panel46,47 (see Table 1 for sociodemographic and clinical characteristics). Parents of children aged 5 to 17 years were emailed a link to the online enrollment questions and the survey content. The enrollment questionnaire included a question on whether the child had an “intellectual or developmental delay” that would prevent him or her from completing a questionnaire. If the parents responded affirmatively to this question, they were deemed ineligible for the study. We used this same question for all the clinical samples as well.
General Population and Sleep Referral Center Samples.
When finished with their part of the questionnaire, parents asked their children to complete their part of the questionnaire on their own. We asked children if they were indeed [name of child inserted], and then we asked them to record their name. We used these questions to ensure that the child himself or herself was responding to the questionnaire. To reduce the burden on any single respondent, we created 4 questionnaire forms; none of the forms contained all the items we tested, but substantial overlap existed across the forms. For example, we included both Sleep Disturbance and Sleep-Related Impairment items on 3 of the 4 forms. We presented 1 item per screen to focus a respondent's attention on each question. We used weights to render the study population a national sample of the US population. The initial weights adjusted for oversampling of individuals living in minority communities and Spanish language–dominant areas and nonresponse. We adjusted the weights until the distributions of gender, census region (Northeast, Midwest, South, West), metropolitan area (Metro, Non-Metro), and household income (less than $25 000, $25 000-$49 999, $50 000-$74 999, ≥$75 000) matched those for children aged 5 to 17 years in the 2015 Current Population Survey.48
We examined the validity of the item banks in analyses completed with the general population sample, and additional data collected from children seen at the Children's Hospital of Philadelphia (CHOP) Sleep Center, which provides diagnostic and management services for children with sleep problems. We used weekly chart reviews to identify potentially eligible children (same eligibility criteria as the general population). Parents were either approached during their child's scheduled appointment in clinic or the polysomnography sleep laboratory, or sent an email including study information and a link to the online questionnaire.
Using the study criteria (parents of children with ASD aged 5-17 years and children with ASD aged 8-17), coordinators of the autismMatch online panel identified potentially eligible families and sent a total of 2 emails to the parent listed in the registry. The emails described the study and provided a link to the study website through which parents—and, if applicable, their children— completed the sleep health measures. Prior to enrollment, parents completed a questionnaire to assess their/their child's eligibility for study participation.
We also conducted a survey in the Interactive Autism Network (IAN). The IAN registry was designed to involve interested families in ASD-related research projects. Families enroll in the registry online or by mail and provide consent to be contacted about research studies. IAN personnel assembled a list of potentially eligible parents and families (ie, parents of children with ASD aged 5-17 years and children with ASD aged 8-17 years) from the IAN registry. CHOP contacted parents of potentially eligible subjects by email. The emails described the study and provided a link to the study website through which parents—and, if applicable, their children—completed the sleep health measures. Before enrollment, parents completed a questionnaire to assess eligibility for study participation.
Patients treated at National Jewish Health are asked to indicate their interest in serving as participants in future research studies. If patients and their families indicate interest, they are entered into a National Jewish Health research database. Study staff periodically pulled potential participants from the database using the study criteria (ie, parents of children with asthma and/or atopic dermatitis aged 5 to 17 years and children with asthma and/or atopic dermatitis aged 8-17 years). Parents were approached during their child's scheduled appointment in clinic or sent an email including study information and a link to the online questionnaire.
Staff at each network identified potentially eligible participants (ie, parents of children aged 5-17 and children aged 8-17 years) from registries. Potentially eligible parents and guardians were sent an email that described the study and provided a link to the online questionnaire. Before enrollment, parents completed a questionnaire to assess their/their child's eligibility for study participation.
This aim describes the psychometric evaluation using classical test and IRT approaches.
We performed descriptive statistics for all survey samples and included assessments of each item's mean, SD, skewness, and response distribution. At the scale level, we examined the range of scores and the percentages of individuals at the floor and ceiling of these scores.
Using data from the general population sample, we evaluated reliability with an IRT-based estimate of Cronbach's alpha called marginal reliability (range 0 low to 1 high).49
Using data from the general population sample, we tested the assumption that the item pool would have the same 2 factors, 1 for Sleep Disturbance and 1 for Sleep-Related Impairment, as the PROMIS adult measures on which the pediatric versions were based in exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). We performed both EFA and CFA with the weighted least squares means and variance-adjusted estimator and a Promax rotation using Mplus 6.1. Following the guidance of the PROMIS scientific standards,40 we evaluated the comparative fit index ([CFI] > 0.95 for good fit), the Tucker-Lewis Index ([TLI] > 0.95 for good fit), and the root mean square error of approximation ([RMSEA] < 0.06 for good fit). To test the assumption of 2 factors, we fit a CFA with the full item pool (expected poor fit) and a 2-factor EFA (items expected to load on their hypothesized factor). We considered items locally dependent if their residual correlations (the correlation between items that remains after removing their common correlation with the underlying latent variable) were ≥ 0.20 in the single-factor CFA,50 and when present, 1 of the correlated items was removed.
Using data from the general population sample, we developed graphs of item mean scores conditional on the total test scale score minus the item score were to examine item monotonicity (ie, the probability of item endorsement should increase as the measured trait increases). We removed nonmonotonic items.
We calibrated the final item pool using an IRT model called Samejima's graded response model.51 Data for IRT modeling were from the general population sample. In IRT, calibration refers to estimating discrimination and threshold parameters for each item using a sample's item responses. The discrimination statistic (also referred to as slope and designated by a) measures the ability of responses to the item to differentiate respondents by their level of the latent variable (eg, sleep disturbance, sleep-related impairment). The IRT model produces threshold parameters (referred to as item difficulty, and designated as b), which correspond to the difficulty of endorsing item responses. Thresholds indicate the point on the latent variable scale, called theta with a mean of 0 and SD of 1, where a respondent is more likely than not to respond in (at least) the next category. For an item with 5 response options, the IRT model produces 4-item threshold statistics. Because we calibrated items using a weighted representative sample of the general US pediatric population, scale scores can be interpreted relative to the general US pediatric population aged 5 to 17 years. We implemented these analyses using Samejima's cumulative logit graded response model in Stata 14.2 with maximum likelihood estimation.52 We estimated the parameters with mean and variance adaptive Gauss– Hermite quadrature using 7 quadrature points to compute the log likelihood.
DIF refers to the possibility that 2 individuals with equivalent levels of a latent variable (eg, sleep disturbance) answer questions differently as a function of another variable (eg, age). Failure to account for DIF can bias scale scores and lead to spurious study conclusions. To identify item bias by child age, sex, race, and ethnicity, we performed DIF analyses using the Lordif package in R and using data from the general population sample.53 Lordif regresses an item's ordered responses on an IRT-derived scale score; the indicator variable, such as age or sex; and the interaction of the 2 variables to test for both uniform and nonuniform differential DIF. Items that showed a 1% change in the McFadden pseudo R² measure were considered to demonstrate differential item functioning54 and, if present, were removed from the item pool.
PROMIS measures are scored in the direction of their concept's name, so higher Sleep Disturbance and Sleep-Related Impairment scores indicate poorer sleep health. After finalizing item parameters, we used Firestar Version 1.3.2, an R-based simulation software,55 and estimated full item bank and short-form scores using Bayesian Expected A Posteriori (EAP) estimation.56 EAP scoring uses an individual's pattern of responses and the model's parameters to estimate an individual's theta score, which is set to a mean of 0 with a SD of 1. The theta scores are linearly transformed to T-scores by multiplying by 10 and adding 50. A score of 50 represents the average level of the sleep health latent variable for children in the national sample used for calibration and centering of scores, and a score of 40, for example, is 1 SD below the national average, which can also be expressed using T-distribution tables as the 16th percentile. This type of scoring provides an intuitive and easy-to-use metric.
We performed known group validation of the sleep health measures to evaluate differences in scale scores among subgroups that are hypothesized to have clinically meaningful differences. It is an essential first step in the evaluation of the validity of any health measure. In the general population, we expected poorer sleep health scores to be associated with older age, parent-reported sleep problem (an affirmative response to the question “Do you think your child has a sleep problem?”), poorer general health, obesity (computed from parent-reported child height and weight converted to body mass index age-sex percentiles and defined as ≥ 95th percentile), and presence of a doctor-diagnosed chronic and neurodevelopmental conditions.32-34 Because they were referred to the clinic for sleep problems, we expected children from the sleep center to have markedly poorer sleep health scores than those of the general population.
We performed all known-group validity analyses using Cohen's d statistic to produce an effect size estimate,57 computed as the difference in means divided by the SD of the sample. We considered Cohen's d values of 0.2 to < 0.5 to be small effects, 0.5 to < 0.8 moderately sized, and 0.8 and above large effects.
We further examined the validity of the item banks using data collected from 10 condition-specific populations: ASD (recruited through autismMatch, an online registry run by the Center for Autism Research at the Children's Hospital of Philadelphia, and the Interactive Autism Network, an online national and international network/registry for families of children with ASD); asthma and atopic dermatitis (recruited at the National Jewish Health Pediatric Asthma and Allergy Clinic); rare epilepsy conditions (recruited through the Rare Epilepsy Network, an online national and international network for families of children with rare epilepsy conditions); inflammatory bowel disease (IBD; recruited through ImproveCareNow, an online national and international network for families of children with IBD); nephrotic syndrome (recruited through the NephCure Kidney Network, an online national and international network for families of children with Nephrotic Syndrome); WAGR (Wilms tumour-aniridia) syndrome (recruited through the International WAGR Syndrome Association, an online national and international association for families of children with WAGR syndrome); Cornelia de Lange syndrome (recruited through the Cornelia de Lange Syndrome Foundation, an online national and international association for families of children with Cornelia de Lange syndrome); Phelan-McDermid syndrome (recruited through the Phelan-McDermid Syndrome Data Network, an online national and international foundation for families of children with Phelan-McDermid Syndrome); and RASopathies (recruited through the RASopathies Network, an online national and international network for families of children with RASopathies). We expected children from these chronic and neurodevelopmental research networks to have poorer sleep health scores than those of the general population.
We also evaluated the correlations between the newly developed sleep measures and existing measures of sleep problems, daytime sleepiness, and fatigue. To evaluate sleep-related impairments, we gave the Children's Sleep Habits Questionnaire Daytime Sleepiness Subscale (minus the open-response question regarding child wake time), which is a 9-item scale that assesses parent-reported sleep offset time, difficulties waking in the morning (which can be a sign of poor quality or deficient sleep), and daytime sleepiness during specific activities (eg, watching TV, riding in car).58 The School Sleep Habits Survey is a self-reported measure of adolescent sleep patterns, including descriptions of school-night and weekend-night sleep habits. A 10-item Sleep/Wake Problems Behavior Subscale measuring inconsistent or delayed sleep onset and offset times, difficulties with sleep onset or sleep offset, and daytime sleepiness was given to all child participants. To assess fatigue, we used a subset of the Pediatric Fatigue Short Form 10a items.59 The items addressed general fatigue experiences (“I felt weak”; “I got tired easily”) and fatigue impact (“I was so tired it was hard for me to pay attention”; “Being tired made it hard for me to play or go out with friends”).60,61
Fixed-length short forms (typically 4 or 8 items) can be constructed from an item bank. Regardless of the specific items administered, the scores from any short form are comparable, because they are calibrated to the same IRT-based metric. For each domain, PROMIS has produced recommended short forms that minimize respondent burden but maintain measurement precision for people with a wide range of the measured concept.1 Items from item banks may also be given as a computerized adaptive test, which uses algorithms to select which items to administer based on the respondent's prior responses.62,63
The CHOP IRB reviewed and approved study procedures for the qualitative phase of the project (CHOP IRB protocol number 15-011759). This protocol included the cognitive interviews conducted during the funding period.
The CHOP Institutional Review Board reviewed and approved study procedures for the collection of general population (15-012503) and clinical sample (16-013083) data. Parents provided informed consent for children, and children gave their assent.
An amendment adding 9 additional populations to the clinical validation protocol (16-013083) was submitted and approved by CHOP's IRB. The additional clinical populations are described above in Section E2.b.
Analyses were included under IRB protocols 15-012503 and 16-013083.
Analyses were included under IRB protocols 15-012503 and 16-013083.
The PROMIS Adult Sleep Disturbance and Sleep-Related Impairment item banks comprised 27 and 16 items, respectively. We derived 4 facets from the Sleep Disturbance item bank: sleep depth (sleeping soundly, light and feed sleep); sleep quality (perceptions of the adequacy of and satisfaction with sleep); sleep onset (falling asleep); and sleep continuity (staying asleep). We derived an additional 4 facets from the adult Sleep-Related Impairment item bank: daytime sleepiness (feeling sleepy or drowsy); energy/tiredness (having enough energy vs feeling tired); sleep offset (waking up); and perceived functional impairments associated with sleep problems (difficulty completing daily life activities). The initial (version 1) child sleep health conceptual framework included these 8 facets (Figure 2).
All experts recognized the need for improved PRO measures of pediatric sleep health. They acknowledged that child-report measures are needed to assess sleep-related experiences that are individually perceived or displayed in contexts that parents cannot observe. However, expert opinions differed regarding the age at which children can reliably and accurately report on their own sleep. Whereas some experts argued for reliance on parent report until adolescence, others felt that children as young as 8 years of age can provide valid responses to developmentally appropriate questions. Particularly, several experts noted that younger children might have difficulty accurately attributing their health and functioning to sleep quality and duration.
In general, experts thought the PROMIS adult domain definitions captured the critical elements of sleep health for children, but that a few critical pediatric sleep health concepts were missing. For example, experts felt items that assess the impact of sleep on “daytime activities” and the capacity to “get things done” may not fully represent the impact of sleep on a child's functioning. Experts suggested adding items that better reflect the social role functions of childhood, including peer interactions and leisure activities. They also recommended expanding the set of items that assess the impact of sleep on children's cognitive functions (eg, understanding/following directions) and emotional regulation. A few experts identified hyperactivity, or “trouble sitting still,” as an important indicator of poor sleep in children that was missing from the PROMIS adult measures. Based on expert input, we replaced the more general functional impairment facet with 3 facets representing the impact of poor sleep and sleepiness on children's cognition (alertness, concentration, and learning); affect/behavior (irritability and emotional/behavioral regulation); and daily life activities (task completion and social role fulfillment). Experts identified 2 additional aspects of sleep health that are not included in the PROMIS measures for adults: breathing problems (including sleep apnea and snoring) and parasomnias (abnormal movements, behaviors, emotions, and perceptions while falling asleep, sleeping, or awakening). We added these facets to the framework. Thus, after expert interviews, the sleep health conceptual framework (version 2) included 12 facets (Figure 2). Table 2 shows the frequency with which experts identified sleep health concepts representative of the final facets.
PROMIS Pediatric Sleep Health Facets: Summary of Qualitative Item Development and Review.
Child interviewees were 39% male, 39% were aged 8 to 11 years, and 61% were aged 12 to 17 years. The children of parent interviewees were 52% male, 12% were aged 5 to 7 years, 36% were aged 8 to 11 years, and 52% were aged 12 to 17 years. Both children and parents identified a wide range of child sleep health experiences and impacts that confirmed the importance of concepts derived from the PROMIS adult measures and expert interviews. Table 2 shows the frequency with which interviewees from each clinical context referenced the final sleep health facets. Difficulty falling asleep was the most commonly identified sleep disturbance. Most children from the sleep clinic (86%), all children with CKD, and almost half of children from primary care (43%) referenced having trouble falling asleep. Most parents indicated that sleep onset was a problem for their child. In contrast, only 2 parents and no children referenced sleep depth in describing their child's sleep. As expected, parents of children treated in the sleep clinic were most likely to identify breathing problems (eg, snoring, pauses in breathing during sleep) and parasomnias (eg, talking in sleep, grinding teeth). However, only a single child (from the sleep clinic) identified a parasomnia and no children referenced breathing problems. Both children and parents talked about dreams and nightmares, a feature of sleep that was not derived from the PROMIS adult measures and expert interviews. We added a dreams (having dreams and nightmares) facet to the conceptual framework based on child and parent concept elicitation interviews.
Almost two-thirds of children from primary care and sleep clinics (71%) and one-third of children with CKD (29%) reported experiencing daytime sleepiness. Parents of children from the sleep clinic (63%) and nephrology clinic (50%) were more likely than parents of children from primary care (9%) to recognize this problem. Parents were more likely than children to reference the impact of sleep on children's daily activities, affect, and behavior. In particular, parents identified negative mood and temper loss as common consequences of poor sleep. Upon completion of the child and parent interviews, the sleep health conceptual framework (version 3) included 13 facets.
The literature search yielded 2490 citations. Of them, 634 articles described the development or application of 329 unique PRO measures of pediatric sleep. Between 6 and 53 articles included descriptions of the 10 most commonly used measures (mean, 17.3; SD, 14.5).39,58,60,64-67,69-71 We sorted the 172 item concepts abstracted from these measures into the 13 facet categories. We eliminated redundancies and generated new item concepts for facets not fully represented by the existing measures. We translated item concepts into item expressions to generate preliminary versions of pools comprising 92 parent-reported items and 82 child-report items.
The phrase “tried hard” in the item “I tried hard to fall asleep” is an American expression that connotes struggle or that substantial and purposeful effort is required to achieve a goal (eg, falling asleep). This concept is not applicable to sleep in other languages and was deleted. It was the only item we deleted because of the translatability review. The phrase “took a nap” in the item “I took a nap during the day” is difficult to translate in some languages. We did not include this item in the final bank. Other comments involved minor adjustments that will be made in the final item translation (eg, “I slept through the night” will be translated as “I slept all night”).
On average, items were written at a fourth-grade reading level (mean, 2.7; SD, 2.7). Items ranged in estimated grade equivalent from 0.0 to 7.0.
Child interviewees were 50% male, 63% were aged 8 to 11 years, 37% were aged 12 to 17 years, 62% were White, 13% were African American/Black, 3% were Asian, 22% were other/more than 1 race, 97% were non-Hispanic, and 3% were Hispanic. The children of parent interviewees were 43% male, 10% were aged 5 to 7 years, 57% were aged 8 to 11 years, 33% were aged 12 to 17 years, 57% were White, 29% were African American/Black, 5% were Asian, 9% were other/more than 1 race, 95% were non-Hispanic, and 5% were Hispanic. Of the 82 child-report items tested in cognitive interviews, 35 (43%) were retained without revision, 8 (10%) were revised and retained, and 39 (47%) were eliminated. Of the 92 parent-reported items tested, 41 (44%) were retained without revision, 8 (9%) were revised and retained, and 43 (47%) were eliminated. We eliminated or revised items that contained words or phrases that interviewees did not understand. For example, all sleep depth items were eliminated because the terms deep sleep, light sleep, and sleeping soundly were unfamiliar to children. Both children and parents confused the terms restful and restless. Most children did not understand the term alert. We eliminated items that included these terms. Notably, interviewees understood several breathing and parasomnia items in ways that differed from their intended meaning. The item “my child stopped breathing during sleep” was meant to represent intermittent pauses in breathing associated with sleep apnea, but parents understood the item as ceasing to breath without spontaneous recovery. Children interpreted items indicative of restless leg syndrome (eg, my legs bothered me at bedtime) to mean muscle soreness resulting from sports or play. We eliminated other items to reduce redundancy in the item pool. For example, we removed “I felt stressed at bedtime” because it was assigned the same meaning by children and parents as “I felt worried at bedtime.” Last, we removed a few items after cognitive interviews revealed that they lacked a clear valence. For example, “I fell asleep quickly” was interpreted by some interviewees as an indicator of excessive sleepiness and by others as healthy sleep onset (eg, no difficulty falling asleep).
The final item pool included 43 child-report items and 49 parent-proxy items (Table 3). Both versions contained 23 adult PROMIS items.
Final PROMIS Pediatric Sleep Health Item Pool by Facet.
For the general population sample, we sent email invitations to 4419 parent participants in the internet panel who had a child between the ages of 5 and 17. Of those, 2461 parents accepted the invitation to learn more about the study. Of these, 1614 were eligible; 1477 completed the parent survey (92% participation rate). Of the 1234 eligible children (ie, had an eligible parent and were between the ages of 8 and 17), 1104 completed the child survey (89% participation rate).
For the clinical samples, potentially eligible participants were either recruited in-clinic (Sleep Center, National Jewish Health Asthma and Allergy clinic), or sent an email including a link to the online survey (Sleep Center and PPRN clinical samples). Of the 160 eligible children recruited from the sleep center, 128 completed the survey (80% participation rate). Of the 309 eligible parents recruited from the sleep center, 270 completed the survey (87% participation rate). Of the 128 eligible children recruited from autismMatch, 96 completed the survey (75% participation rate). Of the 223 eligible parents recruited from autismMatch, 196 completed the survey (88% participation rate). In-person and email recruitment of the added clinical samples resulted in the following participant numbers: Cornelia de Lange syndrome (child = 3, parent = 41); ASD (recruited through the Interactive Autism Network; child = 182, parent = 718); inflammatory bowel disease (child = 10, parent = 21); nephrotic syndrome (child = 19, parent = 46); Phelan-McDermid syndrome (child = 0, parent = 30); RASopathies (child = 8, parent = 20); seizure disorders (child = 0, parent = 109); WAGR syndrome (child = 2, parent = 29); atopic dermatitis (child = 14, parent = 34); asthma (child = 88, parent = 117); atopic dermatitis and asthma (child = 59, parent = 81) (Tables 4-7). The low number of children participating in the Phelan-McDermid, RASopathies, seizure disorders, and WAGR samples was due to ineligibility that resulted from intellectual delay.
Sleep Disturbance Child Report SF8.
Sleep-Related Impairment Child Report SF8.
Sleep Disturbance Proxy Report SF8.
Sleep-Related Impairment Proxy Report SF8.
In the general sample, a total of 1614 parents were deemed eligible. Ineligible parents included those who did not provide consent (n = 59), did not enter the survey after consenting (n = 81), were not able to participate because the quota for their child's age had already been met (n = 344), or did not meet eligibility criteria (38 reported a cognitive limitation or developmental delay that would prevent their child from completing a survey, and 111 parents did not provide permission for their child to participate).
In the sleep center sample, a total of 309 parents were deemed eligible. One parent was deemed ineligible because she indicated that she did not speak English. Of the 309 eligible parents, 160 of their children were also deemed eligible. Ineligible children included those whose parents did not provide permission for them to complete the survey (n = 49), were not able to read or speak English (n = 5), were not between the ages of 8 and 17 (n = 112), or had an intellectual disability or developmental delay that prevented them from answering survey questions (n = 13).
In the autismMatch sample, a total of 223 parents were deemed eligible. Of the 223 eligible parents, 128 of their children were also deemed eligible. Ineligible children included those whose parents did not provide permission for them to complete the survey (n = 41), were not able to read or speak English (n = 4), were not between the ages of 8 and 17 (n = 36), or had an intellectual disability or developmental delay that prevented them from answering survey questions (n = 27).
Item bank development began with a pool of 43 items that were previously produced using qualitative methods. Although we hypothesized that there would be 2 factors, 1 for Sleep Disturbance and another for Sleep-Related Impairment items, initial testing examined these items as having a single underlying dimension. Internal consistency reliability was high, consistent with the large number of items (child-report 0.97; parent-proxy 0.96). However, confirmatory factor analysis indicated poor model fit, treating all 43 items as a single scale (child-report CFI 0.90, TLI 0.89, RMSEA 0.09; parent-proxy CFI 0.89, TLI 0.88, RMSEA 0.07).
In a 2-factor exploratory factor analysis, we identified 6 items that were positively worded and did not correlate with their hypothesized factor (ie, they had low “factor loadings”). Each measured satisfaction with sleep: got enough sleep; felt refreshed when woke up; satisfied with sleep; happy with sleep; slept well; and woke up feeling ready to start the day. We excluded these 6 items. All of the other items were reflective of a disturbance in sleep, daytime sleepiness, or an impairment resulting from poor sleep. When we repeated the 2-factor exploratory factor analysis, the remaining 37 items (20 Sleep Disturbance items and 17 Sleep-Related Impairment items) loaded on their hypothesized factor.
We deleted additional sleep disturbance items because of low factor loadings (< 0.60) from single-factor confirmatory factor analyses: wet the bed (0.27) and take medicine to fall asleep (0.58). We deleted other items because they had high residual correlation (> 0.20) with 1 or more items: scary dreams, bad dreams, and hard time falling asleep because felt upset. These item deletions resulted in a final set of 15 sleep disturbance items, which had adequate model fit: child-report, CFI 0.96, TLI 0.95, RMSEA 0.12; parent-proxy, CFI 0.95, TLI 0.95, and RMSEA 0.12. Of these 15 items, 9 had the same wording as parallel items in the PROMIS Adult Sleep Disturbance item bank.
We excluded 2 Sleep-Related Impairment items before psychometric analysis because they are included in the PROMIS Pediatric Fatigue item bank: felt tired and had enough energy. The item difficulty waking up had a high residual correlation with felt sleepy when woke up, so we deleted the former. Finally, slept during the day had nonmonotonic thresholds and low factor loading (0.58), so we deleted it, leaving a total of 13 items. Model fit for these 13 items was adequate: child-report, CFI 0.97, TLI 0.96, and RMSEA 0.12; parent-proxy, CFI 0.98, TLI 0.97, and RMSEA 0.11. Of these 13 items, 7 had the same wording as parallel items PROMIS Adult Sleep-Related Impairment item bank. Results from the final CFAS for the item banks, child-report and parent-proxy editions, item-level descriptive statistics, and a 2-factor exploratory factor analysis completed in the final set of items are included in Appendix C.
This project did not evaluate specific health outcomes. The final products produced from this project are self-reported health measures that can be used as one approach to assessing sleep health outcomes.
See table 1 for sociodemographic characteristics of the general population sample.
In IRT, calibration refers to estimating discrimination and threshold parameters for each item using a sample's item responses. The discrimination statistic (also referred to as a slope and designated by a) measures the capacity of responses to the item to differentiate respondents by their level of the latent variable (eg, sleep disturbance). The IRT model produces threshold parameters (referred to as item difficulty, and designated as b), which correspond to the difficulty of endorsing the item. Thresholds indicate the point on the latent variable where a respondent is more likely than not to respond in (at least) the next category. Lower thresholds detect lower severity levels and vice versa. For an item with 5 response options, the IRT model results in 4-item threshold statistics. We used the graded response model, which is a type of IRT model, and fitted it to each of the item pools separately (Tables 8 and 9).
Item Calibrations for the PROMIS Pediatric Sleep Disturbance Item Banks, Child-Report and Parent-Proxy Editions, and Assignments to SF8 and SF4.
Item Calibrations for the PROMIS Pediatric Sleep-Related Impairment Item Banks, Child-Report and Parent-Proxy Editions, and Assignments to SF8 and SF4.
The Sleep Disturbance discrimination parameters ranged for child-report 1.7 to 4.5 and 1.6 to 6.2 for parent-proxy, while the Sleep-Related Impairment discrimination parameters ranged from 1.7 to 5.0 (child report) and 1.5 to 6.0 (parent-proxy).
The Sleep Disturbance threshold parameters ranged for child-report from −0.7 to 3.8 and for parent-proxy −0.5 to 4.6, a span of 4.5 and 5.1 SDs. The Sleep-Related Impairment threshold parameters ranged for child-report from −1.2 to 3.1 and −0.8 to 3.9, a span of 4.3 and 4.7 SDs.
Tables 8 and 9 also indicate which items were selected for the short forms, and which of the pediatric items are also included in the parallel PROMIS adult item banks.
In comparison with the general sample rank of 50th percentile, the Sleep Disturbance mean percentile ranks by both child reported and parent-proxy for the clinical samples were higher than for the general population. For example, the mean percentile rank for child report Sleep Disturbance among children in autismMatch was 84th, which is 34 percentile points higher than with the general sample. Sleep-Related Impairment followed a similar trend to Sleep Disturbance, with the exception of Cornelia de Lange syndrome and WAGR syndrome by child report, where the mean percentile rank was in an unexpected direction, most likely due to small sample size. Tables 4 to 7 display the sample size, mean T-score, SD, and T-score percentile rank (relative to the US general population).
Short forms for both child and proxy report Sleep Disturbance and Sleep-Related Impairment are being submitted to healthmeasures.net for adoption into the PROMIS suite of instruments.
The marginal reliabilities for both item banks were 0.92 for child-report and 0.90 for parent-proxy (Table 10), which indicates excellent precision that is high enough to support individual-level analyses in research and clinical practice. The marginal reliability by T-score estimates is shown for the child-report editions of both item banks in Figure 3. The precision of the estimates was high (marginal reliability > 0.80) from T-scores in the range of 35 to a high of 80. Reliability declined more precipitously at the tails for the 4-item short form compared with the 8-item short form.
Evaluation of the Validity of the Sleep Disturbance and Sleep-Related Impairment SF8 Scale Scores for the General Population Sample.
Marginal Reliability of the Sleep Health Item Banks, Child Report Editions.
The general population sample mean was set to 50 (50th percentile) with a SD of 10. For the clinical sample, we converted T-scores to percentile ranks using T-distribution tables. The Sleep Disturbance means for the sleep center clinical sample were substantively higher than the general population: child-report 58.4 (80th percentile) and parent-proxy 62.4 (89th percentile). The Sleep-Related Impairment means were child-report 56.9 (75th percentile) and parent-proxy 61.8 (88th percentile).
In the general population sample, we evaluated sociodemographic and clinical known group differences (Table 10). Sleep disturbance and sleep-related impairment increased with age, financial strain, and food insecurity, and, as expected, was no different by gender, race, or ethnicity. Although the sociodemographic differences were small, there were moderate to large effects observed for parent-reported presence of any chronic or neurodevelopmental condition, attention deficit hyperactivity disorder, autism, insomnia, obstructive sleep apnea, current use of melatonin for sleep, current use of other medications for sleep, and presence of a sleep problem.
The correlations (Table 11) between the parent-proxy and child self-reported editions were 0.76 for Sleep Disturbance and 0.72 for Sleep-Related Impairment. Fatigue correlated more strongly with Sleep-Related Impairment than Sleep Disturbance. The Children's Sleep Habits Questionnaire Morning Waking/Daytime Sleepiness Subscale was weakly correlated with the Sleep Disturbance and moderately correlated with the Sleep-Related Impairment. The School Sleep Habits Survey was moderately correlated with both Sleep Disturbance and Sleep-Related Impairment. These results provide evidence in support of the validity of the newly developed measures.
Correlations of the Sleep Disturbance and Sleep-Related Impairment Scale Scores With Other Measures for the Sleep Referral Center Clinical Sample.
We did not detect any differential item functioning by child age, sex, race, or ethnicity.
To develop the PROMIS Pediatric Sleep Health items, we applied best-practice qualitative methods for PRO instrument development and content validation. Sleep health concepts were initially derived from the PROMIS sleep measures for adults, because one of our goals was to enable development of a lifespan sleep health measurement system. We supplemented and refined underlying sleep health concepts based on expert, child, and parent input to maximize the tools' developmental appropriateness and meaningfulness for children. We verified understandability of items based on child and parent cognitive interviews. It is noteworthy that we deleted nearly half of the items that we tested in cognitive interviews because of poor understanding. To our knowledge, no other pediatric PRO sleep health measure has been developed with comparable attention to content validity. As such, the PROMIS pediatric sleep health measures have greater potential to assess relevant and important aspects of children's sleep health in a developmentally appropriate manner.36
The final item pools contain 43 child-report items and 49 parent-report items. Both versions assess children's capacity to fall and stay asleep; perceptions of sleep quality; occurrence of bad and scary dreams; bed wetting; sleepiness during waking hours; low energy; difficulty waking up; and the impact of sleep problems and sleepiness on cognition, affect, behavior, and participation in daily activities. Trouble falling asleep was the most commonly referenced sleep problem in child and parent concept elicitation interviews. Yet only 3 of the 10 most commonly used pediatric sleep health PRO measures assess sleep onset. Likewise, most children and parents identified waking up in the night as a concern, but only 2 of the 10 frequently used measures assess sleep continuity. Thus, many existing sleep health PRO measures fail to adequately capture sleep problems most commonly identified by children and parents. Many of the measures assess specific breathing problems and parasomnias, which we found to be infrequent except among children being treated in a tertiary care sleep clinic. Cognitive interviews indicated that parents may misinterpret items that assess symptoms of sleep apnea (eg, stop breathing) and children and parents alike may misinterpret items that assess restless leg symptoms. Respondents' lack of familiarity with these symptoms may reduce the accuracy of their self-report. It is also worth noting that these measures do not assess presence of sleep disorders; rather they are applicable to all children and assessment of the quality of their sleep and insomnia-type symptoms (Sleep Disturbance scale) and daytime sleepiness and its impact on functioning (Sleep-Related Impairment scale).
Across clinical contexts in this study, most children reported experiencing daytime sleepiness. Fewer parents identified this concern. Many children described sleepiness at school, a setting that parents are unlikely to observe. Thus, parent-reported measures may underestimate this commonly experienced and impactful sleep problem. Conversely, parents were more likely than children to identify the ways in which poor sleep impacts their children's alertness, learning, ability to follow directions, mood, emotional regulation, behavior, and social functioning. Although most parents identified children's negative mood and temper loss as important consequences of poor sleep, only 4 of the 10 most commonly used pediatric sleep PRO measures assess these concepts. As noted by experts, it may be difficult for children to accurately attribute changes in their health and functioning to sleep. Cognitive interviews indicated that children adequately understood final versions of the sleep impact questions. However, in responding to the items, they may underestimate the consequences of sleep problems or sleepiness.
About half of the final items were derived from the PROMIS sleep health measures for adults. Because they have items in common, it may be possible to statistically link pediatric and adult versions of the measures, scoring the 2 versions on the same scales.72-75 Importantly, both pediatric and adult versions have distinct items that characterize sleep experiences and impacts that are uniquely relevant to that life stage. For example, items that assess “wetting the bed” and “trouble sitting still” are unique to the pediatric item pool. The inclusion of both common and developmentally specific items optimizes the measures' content validity for both children and adults, while also enabling score comparisons across the groups.
Content validation of the resulting item pool produced a final pool of 43 items that were expressions of children's lived experiences of sleep disturbance and sleep-related impairments and were well understood by children themselves. This item pool underwent extensive psychometric evaluation and construct validation in this study. The psychometric testing, IRT item calibration, and the evaluation of the measures' validity were performed in a general population (1100 children and 1400 parents) and sleep referral clinic (134 children and 286 parents) samples. Additional clinical populations, described above, were also evaluated.
The final Sleep Disturbance item bank includes 15 items that assess difficulties with sleep onset, sleep continuity, and sleep quality. The Sleep-Related Impairment item bank includes 13 items that assess daytime sleepiness; sleep offset; and the impact of sleepiness on cognitive functioning, affect and behaviors, and daily activities. For each bank, we created 8- and 4-item short forms. Both provide excellent precision across a large range of the latent variables, but the 4-item short form has more error associated with scores at the tails of the distribution. In addition, both banks include a child self-reported edition that can be used for children 8 to 17 years old and a parent-proxy edition that can be used for children 5 to 17 years old.
Of the final 28 items in the 2 item banks, 16 are also included in the parallel adult PROMIS item banks. Furthermore, the pediatric short forms have substantial overlap with their adult counterparts (Tables 8 and 9). Because the pediatric and adult item banks are conceptually harmonized and share several of the same items, it is possible in future work to statistically link the pediatric and adult versions, creating a common metric (both item banks scored on the same scale) and a life course sleep health measurement system from ages 5 to 85 years. Methods for linking alternative forms (ie, instruments) for the same test (ie, sleep health construct) are well established in the field of educational measurement,76-81 where, for example, they have been used to equate different forms of standardized tests across grade levels. More recently linkage methods have been applied to patient self-reported health status measures, such as work that has taken alternative measures for depressive symptoms and linked them to the same common metric.82
We used multiple, rigorous qualitative methods to develop the pediatric sleep health items. Each method provided unique and complementary information about the child sleep health conceptual framework and items. Nonetheless, our approach has several limitations. The final item pool assesses a wide range of sleep health outcomes, but they are not exhaustive. For example, they do not assess sleep timing and duration or behaviors and environmental conditions that promote or prevent healthy sleep. We developed the item pools in accordance with the PROMIS domain-specific approach; they are intended to be universally applicable (not condition specific).
The general population sample enrolled participants from a large, national internet panel. Advantages of internet panels include their efficiency with which large amounts of data can be collected, the accessibility of diverse populations, and standardization of the data collection process.83 Because not all individuals and families have access to home computers and participants of internet panels tend to have a higher socioeconomic status than the general US population,84 we cannot say that the study samples were nationally representative; rather, it is fair to say that the PROMIS pediatric sleep health measures have been standardized to a national, highly diverse sample.
Although the PROMIS sleep health item banks have excellent precision across a wide range of the latent variables, we have not evaluated their responsiveness to longitudinal changes in children's health conditions.
We included children with CKD and their parents in concept elicitation and cognitive interviews to maximize the tools' content validity for children with a chronic condition known to disrupt sleep.41,42 However, the tools' applicability to children with other chronic conditions and neurodevelopmental disorders should be evaluated; this is being done in ongoing studies that are part of the NIH PEPR Consortium. The translatability review was conducted focusing on translation into German and Spanish because PRO measures are increasingly used for research and clinical care in several European nations. We recognize that other translations conducted outside of the United States and western Europe may reveal more significant cultural and linguistic differences in children's sleep health.
Although the PROMIS sleep health item banks have excellent precision across a wide range of the latent variables, we have not evaluated their responsiveness to clinical change.
Longitudinal studies that examine sleep health and its relationship to change in health status is a direction for future research. Another line of investigation is examining the association of these self-reported subjective measures of sleep health with objective assessments using actigraphy and polysomnography.
The PROMIS Pediatric Sleep Disturbance and Sleep-Related Impairment item banks provide efficient, precise, and valid short forms that can be used to assess a child's difficulties falling and staying asleep as well as daytime sleepiness and its impact on daytime functioning. We used rigorous qualitative procedures to generate developmentally appropriate, content-valid, and translatable child- and parent-reported sleep health item pools. Experts, children, and parents had direct input into item content. We conducted child and parent cognitive interviews to ensure the appropriateness and understandability of items. The scales have excellent precision across a wide range of the latent variables, and strong evidence for their construct validity. The child-report edition can be used for children 8 to 17 years old, and a parent-proxy edition is available for children aged 5 to 7 years old. They are ready for use in observational studies of sleep, clinical trials for medications that influence sleep, and in clinical practice.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#ME-1403-12211). Available at: https://www.pcori.org/research-results/2014/developing-sets-survey-questions-measure-childrens-sleep-health
Description of Adherence to PCORI's Methodology Standards (PDF, 100K)
Sleep Health Project Products (PDF, 206K)
Scoring Manuals (PDF, 1.0M)
Forrest CB, Meltzer LJ, Marcus CL, et al. (2018). Developing Sets of Survey Questions to Measure Children's Sleep Health. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/9.2018.ME.140312211
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
