NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Chesnut RM, Carney N, Maynard H, et al. Rehabilitation for Traumatic Brain Injury. Rockville (MD): Agency for Health Care Policy and Research (US); 1999 Feb. (Evidence Reports/Technology Assessments, No. 2.)
This publication is provided for historical reference only and the information may be out of date.
Topic Assessment and Refinement
Two panels of experts worked with the research team to identify key questions in the rehabilitation and survivor phases for adults with TBI. The first panel was composed of two physiatrists, a survivor of TBI, the wife of a survivor of TBI, a State vocational rehabilitation counselor, a neuropsychologist, a psychologist, a clinical coordinator of an outpatient TBI rehabilitation program, and a rehabilitation clinical nurse specialist, all from the Portland, OR, area. The second panel was composed of nationally recognized experts in rehabilitation.
The local panel met twice to establish the scope of the literature review, develop a common understanding of the main concepts bearing on questions of effectiveness in rehabilitation, and identify key questions for investigation. Prior to the first local panel meeting, members were sent a document describing the prevalence, incidence, and burden of illness of TBI and a list of treat-ment settings and techniques. At the first meeting, one of the investigators explained the use of a causal pathway diagram (Woolf, Battista, Anderson, et al., 1990) to enumerate causal links and identify key clinical questions about an intervention. Causal pathways highlight the role of intermediate measures of outcomes, which are often used as proxies for health outcomes in clinical studies and in practice. The format of the meetings was semistructured to promote free interaction aimed at identifying issues with respect to TBI rehabilitation among representatives of the various disciplines. Proposed questions and elements of causal pathways were documented and synthesized for review by the national panel.
Prior to the first meeting of the national panel, members were provided the same briefing given the local panel members, along with a summary of material generated from the local panel's meetings. During the national panel meeting, members were asked to consider the proposed questions, add to or delete from the list, and elaborate on the revised list of questions. The research team synthesized this second level of input and distributed draft questions for study to both panels. Panel members submitted revisions to the task order manager, who coordinated input and continued to distribute new iterations of the questions until consensus was reached.
The panels formulated five questions pertaining to the phases of recovery described in Figure 2. These questions addressed the effectiveness of (1) early rehabilitation in the acute care setting (timing), (2) intensity of rehabilitation, (3) cognitive rehabilitation, (4) supported employment, and (5) care coordination (case management). For each of these questions, members of the research team worked with panelists to write a brief rationale for the question, define key terms, and specify the relevant patient populations, interventions, and outcome measures that should be examined in the literature review (Appendix 1). Detailed information for each question is provided in Chapter 3 (Results) of this report.
Literature Retrieval
Literature was searched using MEDLINE, CINAHL, HealthSTAR, and PsycINFO. In addition, the Cochrane Collaboration made available a database of about 500 articles on brain and spinal cord injury. Four MEDLINE search strategies were written. A single strategy was designed to seek references regarding the timing and intensity of acute care rehabilitation (questions 1 and 2). Three additional search strings were used to find articles on the remaining questions (3, 4, and 5). The full MEDLINE database was searched for randomized controlled trials; otherwise, MEDLINE was searched from 1976 to 1997. One additional search strategy, designed to capture literature for all five questions, was written for each of the remaining databases used (CINAHL from 1982 to 1997, HealthSTAR from 1995 to 1997, and PsycINFO from 1984 to 1997). The search strings are given in Appendix 2. Finally, we searched the Cochrane database for articles about rehabilitation. We referred to the Current Contents database on a monthly basis between November 1997 and May 1998 to ensure we did not miss new literature that might be relevant.
Figure 3 shows the results of the search. The MEDLINE search retrieved a total of 2,271 references. The CINAHL search retrieved 431 articles, HealthSTAR 55, and PsycINFO 339. Members of the research team read the abstracts from CINAHL, HealthSTAR, and PsycINFO and assigned them to one or more of the five questions. Two Cochrane references were also retrieved (the Cochrane database contained primarily pharmaceutical studies). After removal of duplicates, 2,536 citations remained.
Figure
Figure 3. Chart of search and selection of articles for review. Articles reviewed and cited
Two members of the research team read the 2,536 abstracts and applied the eligibility criteria, which are listed in Table 4. The criteria were designed to be broad and generally inclusive of studies with data from at least a series of people undergoing rehabilitation for traumatic brain injury. When the two reviewers disagreed, a third reviewer read the abstract and cast the deciding vote on whether to include it. For citations without abstracts or when the title of the abstract was not sufficient to determine its status, the full article was retrieved and reviewed to determine its eligibility. The reviewers examined each abstract and indicated whether it met the inclusion criteria and, if not, the reason for exclusion. Appendix 3 shows the results of a reliability test between reviewers.
Table
Table 4. Exclusion criteria for review of abstracts.
Eighty-seven articles pertaining to questions 1 and 2, 114 for question 3, 93 for question 4, and 69 for question 5 passed the eligibility screen. The full text of these articles was retrieved, and the articles were reviewed using the data abstraction instrument described below. Each article was assigned one or more categories, such as "longitudinal," "effectiveness," "interventions," etc. (Table 5). Categories were noted in the database for each article to allow for electronic search of subsets of articles containing specific information.
Table
Table 5. Topics addressed by reviewed articles.
Literature Synthesis
Data Abstraction
We designed an instrument to record data abstracted from each eligible article (Appendix 4). The instrument includes items for patient characteristics, interventions, cointerventions, outcomes, study methods, relevance to the specific research questions, and results of the study. The instrument has two components: the first three pages of the instrument apply to all articles specified for inclusion in the study. The remaining pages are individual instruments that apply to one of the five questions. To abstract an article, a reader used the initial abstraction instrument plus one or more of the five question-specific instruments.
The first few questions of the initial abstraction instrument allowed the reviewer to determine if the article actually met the eligibility criteria listed in Table 4. If an article was determined to be ineligible, it was passed to a second reader for confirmation. As shown in Figure 3, 143 articles were eliminated at this phase based on initial eligibility criteria. The remaining 220 articles, plus an additional 67 articles recommended by technical experts or found in the reference lists of reviewed articles (total 287), were subjected to the full abstraction protocol.
Managing the Database of Articles
Retrieved articles were kept in an electronic database (EndNotes PlusTM). User-defined fields were used to record results of the eligibility screen, review for inclusion, and number of reviews performed. Paper copies of the articles were maintained in a master file. Throughout the literature review process, any articles identified from sources such as bibliographies or reference lists, manual searches of journals not contained in electronic databases, or references recommended by outside parties were added to the electronic databases.
Specification of Level of Evidence
We used a three-level system to rate individual studies:
- Class I
- Properly designed randomized controlled trials.
- Class II
- Class II(a): Randomized controlled trials that contain design flaws preventing a specification of Class I. An example of a design flaw is failure to blind raters or lack of followup data.Class II(a): Multicenter or population-based longitudinal (cohort) studies.Class II(b): Controlled trials that were not randomized.Class II(b): Case control studies.Class II(b): Case series with adequate description of the patient population, interventions, and outcomes measured.
- Class III
- Descriptive studies (uncontrolled case series).Expert opinion.Case reports.Clinical experience.
A well-done, prospective, multicenter or population-based cohort study can provide valuable information that, in some ways, is more representative of results in actual practice than are data from a randomized trial done in a highly selected sample. However, an uncontrolled case series is generally classified as Class III, indicating a low level of confidence for inferences about effectiveness.
Why is a control group needed to make inferences about effectiveness? Information about the long-term natural history of TBI provides too little certainty about the outcome to forgo the use of a control group. If the consequences of brain injury in the absence of rehabilitation could be predicted accurately, a control group would not be needed. Some evidence suggests a natural course of recovery from TBI in both basic and complex functions (Dikmen, Reitan, and Temkin, 1983), especially in the first 6 months after injury (Bond, 1979; Levin, Benton, and Grossman, 1982; Levin, Gary, Eisenberg, et al., 1990). After 6 months, survivors of TBI recognize their disabilities as they stabilize and often maximize their functional status, but major deficits in social and leisure activities tend to persist (Oddy and Humphrey, 1980).
A "gray zone" exists between Class II and definite Class III articles. Much of the research in rehabilitation uses quasiexperimental designs. In these observational study designs, control groups are sometimes identified from a separate population of people with TBI. One study compared patients undergoing inpatient rehabilitation with a sample of people with TBI who had been treated in a region of the country where formal inpatient TBI rehabilitation was not available (Aronow, 1987). This was an entirely separate patient group, and all the data except outcome measures came from an independent database.
The main difficulty with the quasi-experimental design is lack of control over the constitution of the compared groups. Since there is no randomization and generally no control over the details of the selection process through which the study participants received their separate therapies, the groups are likely to differ in the frequency of characteristics that are associated with the outcomes of interest. Even when significant efforts are made to match the experimental and the quasicontrol groups, groups still are likely to differ significantly.
Quasiexperimental designs rely heavily on multivariate statistical analysis and matching to counter this problem. While this is common practice, making inferences about effectiveness from statistically controlled data is controversial because these methods do not necessarily guarantee that the results are reliable or that serious bias and confounding have been eliminated. There are at least three major barriers to accuracy in the practice of statistical adjustment for risks. First, all of the confounding variables must be recognized and be associated with accurate and reliable measurement instruments. Second, data on all of these confounding variables must be available in comparable formats in the compared groups. For example, in the study mentioned earlier, baseline data, such as admission functional status scores, may be available for patients who entered inpatient rehabilitation but missing for patients who did not. Third, statistical methods must be selected and applied properly.
Since the first two barriers are virtually never overcome and the third remains somewhat controversial, the execution of a statistically valid quasiexperimental study is a daunting and generally unrealized task. Most studies using retrospective data suffer from the absence of well-measured data on a number of important confounding variables. The few prospective studies generally suffer from a lack of consensus on the proper measurement instruments for many of these variables. Finally, while the standard of practice for use of statistical methods to remove confounding is not completely defined, many studies do not use, or fail to report, available methods to improve the reliability and robustness of these methods.
Much of the literature relevant to the five questions addressed in this report falls into the "gray zone" between Class II and Class III. For this reason, critical appraisal of key studies plays a particularly important role in this review. A number of characteristics of these studies were considered relevant to all rehabilitation questions and were recorded in the data abstraction form. Evaluation of the following factors played a major role in critically appraising these articles:
- Prospective collection of data.
- Complete description of parent patient population.
- Large study population size (driven by hypothesis, power, type I error threshold).
- Study setting--a single center, many centers, or population-based.
- Descriptions of reasons for referral to service being studied.
- Methods described completely enough to allow study replication.
- Complete description of rehabilitation technique in question (independent variable).
- Complete and adequate description of differences between "control" and "experimental" groups.
- Conditions determining whether study subjects did or did not receive the rehabilitation technique in question.
- Information about potential confounders, including types and severity of injury; age; and others (including, in some cases, economic status, educational level, lack of family support).
- Measurement of confounding variables using instruments validated as accurate, sensitive, and reliable.
- Payer group.
- Choice of outcome variables that are meaningful to survivors as well as caregivers.
- Use of functional status and other health outcomes rather than surrogate intermediate outcomes.
- Measurement of outcome variables using instruments validated as accurate, sensitive, and reliable.
- Timing of outcome measurements.
- Assessment of survivor characteristics and outcomes by blinded observer.
- Use of multivariate statistical analysis: Were interactions sought and controlled for? Were risk estimates calibrated? Were all relevant confounders included as candidate variables?
The criteria used to classify articles and the features to be considered in critically appraising them were discussed at the subcommittee, committee, national expert panel, and Aspen Neurobehavioral Conference levels with the goal of maintaining consensus on at least the relative stratification of individual articles.
Construction of Evidence Tables
Evidence tables were constructed to summarize the best evidence about effectiveness pertaining to each question. For questions 1 and 2, there were no randomized trials and only a few quasiexperimental studies. There were many relevant observational studies of important relationships (for example, the relation of survivor characteristics to outcome); while we discuss these results, we chose not to summarize studies that concerned individual causal links or relationships in evidence tables. For question 3, addressing cognitive rehabilitation, 15 randomized controlled trials and comparative studies that met specified inclusion criteria (see the section on question 3) were placed into evidence tables. All comparative studies located for the last two questions, which addressed supported employment and care coordination, were included in evidence tables.
Critical Appraisal of Key Articles
For each of the five questions, we formed subcommittees of one to two members of the research team and one to two members of the local technical panel. A member of the research team chaired each subcommittee. The principal investigator also led subcommittees consisting of members of the national expert panel. All members of the subcommittees reviewed key articles relevant to the assigned question. These reviews were discussed among the various members of the subcommittees, and the results were summarized by the chair. This was an attempt to ensure that the summary statements on the research questions reflected the expertise and experience of a variety of technical experts with relevant skills and training. These interpretive efforts addressed the methods and results of individual studies, their ratings, and their scientific importance.
The principal investigator read all of the critical articles for the five questions. Summaries were presented and discussed with national experts at the Aspen Neurobehavioral Conference in April 1998 (see Appendix 6).
- Methods - Rehabilitation for Traumatic Brain InjuryMethods - Rehabilitation for Traumatic Brain Injury
Your browsing activity is empty.
Activity recording is turned off.
See more...