Objective
To examine the psychometric robustness of tools used to measure change in EDSD in young children with EDSD.
Methods
Outcome measurement tools were primarily identified through the updates of the three published systematic reviews of interventions and mapping review, as well as the first round of focus groups and the survey (see Chapters 3–5 and 7). The list of 43 named tools focused on the evaluation of EDSD-related outcomes of interventions (see Appendix 7). Tools mentioned in the literature, or by participants in the focus groups and survey, that measured other important outcomes (e.g. Parent stress and Child quality of life) were not included in the listing and review of measurement properties.
Searches were conducted to specifically identify papers that examined the measurement properties of the named tools. The search strategy included terms for neurodisability or feeding disorders and children, and included a COnsensus-based Standards for the Selection of health status Measurement INstruments (COSMIN) filter (www.cosmin.nl; accessed 11 November 2019) (see Appendix 8). The COSMIN system was developed by an international group of experts to standardise assessment of the methodological quality of measurement studies. Papers identified from searching are examined in terms of the quality of the research study examining a tool’s measurement properties (e.g. whether the study had sufficient numbers of participants, clear hypotheses stated and a robust approach to the conduct of factor analysis); the paper is rated as ‘inadequate’, ‘doubtful’, ‘adequate’ or ‘very good’ on each property. Next, the data presented in each paper on reliability, validity and responsiveness to change are extracted and judged on COSMIN criteria (e.g. a cut-off point for a good reported level of inter-rater reliability) as being of a ‘sufficient’, ‘insufficient’ or ‘indeterminate’ level. Finally, the evidence is synthesised, following standard COSMIN criteria, to determine the strength of the evidence on each measurement property across all papers that examine any one particular measurement tool.
The following databases were searched: MEDLINE, CINAHL, PsycINFO and WoS. Following the initial searches of MEDLINE and PsycINFO on the 43 listed tools, further named tools of potential relevance were identified in the texts of the measurement properties papers, for example as criterion reference tools or revised versions (n = 21) [e.g. a paper on the Parent Mealtime Action Scale (PMAS) concerned development of a revised scale, the Parent Mealtime Action Scale – Revised (PMAS–R)]. The searches were updated to include the new tool names; thus, in total, we searched for papers on the measurement properties of 64 named tools.
Two reviewers (HM, a clinical psychologist who is experienced in systematic reviewing including use of the COSMIN approach, and CU, systematic reviewer in health research) sifted search results by title and abstract for likely relevance, using definitions and criteria agreed with the research team (see Appendix 9); uncertainty was resolved by requesting the full text. Papers were examined for inclusion at the full-text stage separately by both reviewers, with any disagreement settled by referral to a third reviewer (LP).
At the full-text stage, and before data extraction, a strategy was developed to focus on the most promising outcome measurement tools for application in a future trial. The criteria for exclusion comprised the age range covered (e.g. excluded the Neonatal Oral Motor Assessment Scale); not being about EDSD once further information was available from papers (e.g. Motivation Assessment Scale); cost of training and/or poor availability of training (e.g. Dysphagia Disorders Survey); precision of measurement [e.g. excluding the TOMs and the Eating and Drinking Ability Classification System for individuals with cerebral palsy (EDACS) and other similar classifications of function on 4- to 7-point scales]; or having been poorly rated on measurement properties in a published systematic review of measures of oropharyngeal dysphagia for preschool children with cerebral palsy and neurodevelopmental disabilities by Benfer et al.65 Based on these criteria, 20 tools were not considered further.
Data extraction followed the COSMIN risk-of-bias checklist on reliability and validity of the evidence,66 with additional information noted on acceptability, feasibility, precision and interpretability from Fitzpatrick et al.67 The COSMIN criteria for judgement of good measurement properties were taken from Prinsen.66 The two reviewers trained together on papers and established reliability, before proceeding with data extraction (in addition, HM checked all data extracted by CU). Data were not extracted from papers with a sample of < 10 participants, with a non-relevant sample (e.g. a feeding clinic sample not further described) or when the paper was not about a measurement property.
Results
Results of searches
Papers from the first searches of PsycINFO and MEDLINE were sifted by title and abstract; 51 out of 560 papers and 18 out of 196 papers, respectively, were taken forward to sifting at full text. The reviewers checked 11% of articles (i.e. 88 articles) with 92% agreement on ‘get full text’/exclusion.
In the second search, CINAHL and WoS were added, as well as searching for the 21 additional named tools identified in the previous search. The searches of all four databases yielded 888 references, from which a total of 111 went forward to be sifted at full text. The two reviewers double checked 17.3% of articles, with 94% agreement on ‘get full text’/exclusion.
After de-duplication, a total of 127 papers were sifted as full text from the first and second searches. Of these, 86 papers were excluded and, therefore, 41 papers were included for data extraction ().
Results of data extraction
The 41 papers provided evidence on measurement properties relating to 22 tools used for children with neurodisability: 12 tools measuring child behaviours (three with parent domains also), five tools measuring parent strategies and five tools measuring oral motor skills. For the remaining 22 tools of the 44 considered in this review, no papers were found studying measurement properties with this population.
The information extracted for each paper in terms of the quality of the evidence about measurement properties is presented in Appendix 10, and the evidence for robust measurement is presented in Appendix 11. (Note that some papers provided evidence on more than one tool.)
The evidence is synthesised for each tool in – and a description of each tool is presented in – . The types of tools are divided into three categories: mainly assessing child behaviours in eating and drinking and at mealtimes, parent strategies for feeding their child and reported difficulties in managing mealtime situations or the child’s observable oral motor skills.
Evidence summary on tool measurement properties for parent/carer report on child behaviours
Evidence summary on tool measurement properties for professional observation/assessment of oral motor skills
Information on tools for parent/carer report on child behaviours
Information on tools for professional observation/assessment of oral motor skills
Evidence summary on tool measurement properties for parent/carer report on parent strategies
Information on tools for parent/carer report on parent strategies
There was patchy evidence of variable quality on measurement properties for most tools, with only one study providing any evidence of responsiveness to change (see –). Much of the evidence found related to hypothesis-testing of convergent/divergent validity or differences between known groups. The available evidence showed that hardly any tools met the COSMIN criteria for a sufficient quality of structural validity.
Conclusions
The strongest evidence for robust measurement properties of a tool measuring child behaviours was for the Paediatric Eating Assessment Tool (PediEAT),71,72 a 78-item parent questionnaire that is used with children aged 6 months to 7 years (see –). Evidence for the measurement properties of the Brief Autism Mealtime Behaviour Inventory (BAMBI),50 an 18-item parent questionnaire (or 15 items51) for children aged 2–11 years, is more mixed. The BAMBI was subsequently reworked for all children with feeding problems as the Brief Assessment of Mealtime Behaviour in Children (BAMBIC),68 a 10-item parent-reported scale relating to children aged 18 months to 17 years, which also had mixed evidence of robustness of its measurement properties. The evidence relating to seven further tools measuring child behaviours was limited and poor; therefore, these tools will not be considered further.
In addition, there were two types of tool used to measure children’s intake in this category (i.e. food frequency and food preferences) (see –). The Food Frequency Questionnaire (FFQ)61 evaluated was based on the youth/adolescent version of a list developed at Harvard University.86,87 It has 131 items, and parents indicate foods refused and the number of times per day foods are eaten. Evidence was found only in relation to hypothesis testing of convergent validity and discrimination between groups.
In regard to parent strategies tools, the evidence of measurement property robustness is sparse and poor (see –). Among the child behaviours tools, the Behavioural Paediatric Feeding Assessment Scale (BPFAS),33 Child’s Eating Behaviour Inventory35 and Meals in Our Household70 also have parent domains. The BPFAS has a little more evidence of robustness for use with this population. The BPFAS has 25 items on child behaviours and 10 items on parent attitudes and strategies, relevant for those children aged 2–6 years.
The systematic reviews of parent-reported measures of feeding difficulties by Sanchez et al.88 and Jaafar et al.89 supported the BPFAS as the most robust measurement tool for children with feeding problems aged 2–5 years. In studies identified by our measurement properties review, it was used with children with a range of conditions, including ASD.90,91
Regarding oral motor skills, the strongest evidence is for the Schedule of Oral Motor Assessment (SOMA).45,92 The SOMA involves a structured assessment of children’s ability with a range of food textures and trained observers to rate the video-taped session. The strength of this tool was also concluded by the systematic review by Benfer et al.,65 although more recent reviews by Barton et al.93 and Speyer et al.94 more cautiously reported the difficulty of reaching any conclusion on the basis of the patchy evidence.
Consultation
As the evidence from the review became available, three of the scales (PediEAT, BAMBIC and BPFAS) with the more robust properties were presented to the PAG in September 2018. For child behaviours, parents commented that the PediEAT was long. The BAMBIC was thought to be relatively easy to fill in, but short. The parents in the advisory group considered that some child behaviour items of the BPFAS were difficult to answer for particular situations, for example ‘eats junky snack foods but will not eat at mealtime’ as a child may eat junky snack foods as a meal at mealtime. The parent co-investigators also commented on the FFQ, with reservations expressed about the affordability of some of the range of foods included.
Summary of findings
The review of the papers on measurement properties of tools relevant to EDSD revealed the patchiness of the available evidence. A similar conclusion was reported by other recent reviews examining parts of this topic.95,96 The review enabled the recommendation of candidate tools to be included in the design of outcome measurement in any future trial of interventions for EDSD in children with neurodisability. The PediEAT is a recently developed parent-report tool that measures child feeding difficulties and mealtime behaviours, and had the most evidence of robust measurement properties. For child intake, a FFQ might be acceptable. The BPFAS combines child and parent subscales, and had marginally stronger measurement properties than other parent strategy scales. The SOMA is the strongest measure of oral motor skills, and requires training of the assessors and observers.
Strengths and limitations of the measurement properties review
The identification of tools to be included in the review was comprehensive as it drew on a range of sources, including the mapping review. The inclusion of further tools identified within papers of measurement properties increased the breadth of the review. The pragmatic decision to focus data extraction on tools that were most likely to be candidates for use in evaluation in any future trial of interventions for EDSD was a limitation, but also an expedient best use of resources. The two reviewers worked closely together, to a detailed set of definitions, with access to another experienced reviewer to resolve discrepancies. Conclusions about the robustness of tools were checked against similar systematic reviews. The COSMIN approach to evaluate measurement properties is not applicable to the way in which some of the most valued outcomes are measured (e.g. Growth and Nutrition), which implies that a further review of approaches to their measurement will be required.
Patient and public involvement in the measurement properties review
Members of the PAG were presented with three parent-reported tools with the strongest evidence of their measurement properties, and commented on wording, layout and ease of use.
How did the measurement properties review inform the next step?
The three parent-reported tools that captured child behaviours, child intake of food and parent strategies were presented to groups at the stakeholder consultation workshops for further discussion (see Chapter 11). The process of the review also informed the thinking of the research team around how to conceptualise categories of outcomes valued by parents and professionals as elicited in the focus groups (see Chapter 9) and in the surveys (see Chapters 7 and ). The tools reviewed for their measurement properties belong mostly to the intermediate category of child behaviours and parent strategies relating to EDSD, with child oral motor skills being grouped with proximal outcomes, such as Nutrition and Growth (see Chapter 9).
