Factor Analysis for Data Reduction

Factor analysis has been used in studies of Gulf War veterans, initially to see whether a unique “Gulf War syndrome” could be identified and later to inform case definitions of CMI. In attempting to reduce the amount of data that would be gathered (the large and varied number of symptoms), researchers used factor analysis so that a structure that would include substantially fewer factors than symptoms could be proposed.

Typically in CMI studies, a survey that includes many individual items or symptoms is administered. For example, a survey administered by Knoke et al. (2000) included 98 individual items or symptoms, but the factor-analytic results suggested a data structure of five factors. In that example, one factor included 27 symptoms. Factor scores estimate people's relative levels of given factors. By reducing large sets of symptom data into their structural components, factor analysis can simplify comparisons of symptoms that are potentially related to Gulf War deployment.

The application of the data-reduction capabilities of factor analysis has been attractive for the study of Gulf War veterans' symptoms, but, as will be discussed below, studies' findings have been inconsistent. Different studies have identified different numbers of factors and assigned different names to common groups of symptoms. Some lack of consistency in findings of factor-analytic studies is expected because of differences in methods and questionnaires and because of random variation. Furthermore, when factor analyses are conducted without an a priori hypothesis, the subjective labeling of factors can be controversial, so factor labels should be critically reviewed (Ismail and Lewis, 2006). The factors posited in a given study will depend on the statistical approach used in the factor analysis, including decisions about how factors are “extracted” and “rotated.” In addition, researchers must subjectively assign a name that they believe represents the items of a factor. To compare studies, it is necessary to understand which symptoms were associated with each factor, rather than depending on the label that researchers assigned to each factor.

Factor Analysis for Case Definitions

Although factor-analytic studies have facilitated and clarified comparisons of symptom prevalence and severity in deployed and nondeployed people, they have been less useful in specifying a case definition of CMI. The results of factor analyses do not differentiate among groups of people and cannot create a case definition. That fact has been obscured because investigators often operationalize a case definition by dichotomizing factor scores obtained from a factor-analytic model. However, people do not have factors. As explained above, everyone will have a score on every factor, but dichotomization of factor scores to define a “case” is a postprocessing decision made by investigators and not a direct result of the factor-analytic model.

Factor analysis can be used to evaluate whether the structure of symptom data is different in different populations (such as deployed and nondeployed), but this is not the same question as whether populations have higher symptom levels or greater symptom severity. The unique question that can be asked in the context of a factor-analytic study is whether the factor structure varies among compared populations. That question is most appropriately posited as a formal statistical test, in which the probability of observing the differences between the factor structures in the samples is estimated under the null hypothesis that the factor structures are the same in the two populations. For example, Ismail and colleagues (1999) compared three UK military cohorts: veterans of the Bosnia conflict, those deployed to the Gulf War, and Gulf War veterans not deployed to the Gulf. In addition to applying exploratory factor analysis, the investigators used a particular application of confirmatory factor analyses for which they generated statistics to estimate the goodness of fit of the factor-analytic model. They also tested a series of three models with different constraints: (1) factor correlations are equal in Gulf War–deployed and era veterans, (2) correlations between factors and factor loadings are equal in the two groups, and (3) all parameters are equal. Those models provided a direct, thorough, and hypothesis-based test of whether the factor structure differed in the two groups. That the constrained models did not fit significantly better than the unconstrained model indicates congruence in factor structure in the Gulf War–deployed and Gulf War–era veterans. However, most of the studies of factor-structure differences have failed to test the hypothesis directly, and none has used structural equation models. Instead, investigators have commonly relied on hypotheses related to factor scores or on descriptive comparisons of factor scores, factor loadings, and factor correlations.

There are two ways of comparing factor scores. In one, factor scores are generated for all members of reference and comparison groups. The scores are derived on the basis of a single factor-analytic model. The scores of deployed and nondeployed persons are compared to ascertain whether the presence or severity of the symptoms that define a factor differ by deployment status. No comparison of factor structure is made, because a single modeled structure is used to generate all scores. The other approach is to conduct separate factor analyses in the reference and comparison groups, derive factor scores for everyone on the basis of both factor-analytic models, and then compare the resulting scores; this is not an accepted method of comparing factor structures.

The studies that have used factor analysis to investigate symptoms in Gulf War veterans have used several analytic strategies. Some studies used statistical testing of the hypothesis that the factor structures of deployed and nondeployed veteran populations are significantly different (that is, testing the null hypothesis that the structures are the same) (Ismail et al., 1999), as discussed below and in Chapter 3. Other studies relied exclusively on descriptive statistical techniques, such as correlations among factors, factor scores, or factor loadings (Doebbeling et al., 2000; Kang et al., 2002). Finally, a number of studies used “visual inspection” to discern differences between the factor structures of deployed and nondeployed groups (Knoke et al., 2000; Nisenbaum et al., 2004; Shapiro et al., 2002).

Department of Veterans Affairs

The nationally representative Department of Veterans Affairs (VA) study searched for potential new syndromes through factor analysis (Kang et al., 2002). Data were from a sample of 15,000 deployed and 15,000 nondeployed active-duty, reserve, National Guard, and retired service members in all four branches. Through questionnaures, the authors inquired about 47 symptoms on a three-point ordinal scale. On the basis of judgments of factor interpretability, they chose a five-factor solution for the nondeployed and a six-factor solution for the deployed sample. By inspection, the investigators judged that the first five factors were “very similar” in the two groups. The six factors were fatigue and depression,² musculoskeletal and rheumatologic,³ gastrointestinal,⁴ pulmonary,⁵ upper respiratory,⁶ and neurologic.⁷ However, the last factor extracted contained symptoms consistent with neurologic impairment in the Gulf War group but not the non–Gulf War group. It should be noted that each successive factor that is extracted in a factor analysis accounts for less of the variance than the previous one. In the deployed sample, the sixth factor, labeled neurologic impairment, accounted for only 3% of the total variance, compared with 79% for the first factor. The neurologic factor was not extracted in the nondeployed group but accounted for 4% of the variance. In the neurologic factor, four symptoms—loss of balance or dizziness, speech difficulty, blurred vision, and tremors or shaking—loaded for the deployed but not for the nondeployed group. The authors indicated that a group of 277 deployed veterans (2.4%) and a group of 43 nondeployed veterans (0.45%) had all four of those symptoms.

The authors interpreted their findings as suggesting a possible neurologic syndrome related to Gulf War deployment that would require objective supporting clinical evidence. It is possible, however, that this is an overinterpretation of the data inasmuch as the factor accounts for a small amount of the total variance and the nonextracted sixth factor in the nondeployed group accounted for more variance than it did in the deployed group.

The Iowa Study

The Iowa study (Iowa Persian Gulf Study Group, 1997) grouped symptoms into categories suggestive of existing syndromes or disorders, such as fibromyalgia or depression. Its finding of a considerably higher prevalence of symptom groups suggestive of fibromyalgia, depression, and cognitive dysfunction in Gulf War veterans motivated the first applications of factor analysis to grouping and classifying veterans' symptoms. Several years later, the same team of Iowa investigators performed a factor analysis on the Iowa cohort (Doebbeling et al., 2000). They studied the frequency and severity of 137 self-reported symptoms in 1,896 Gulf War veterans and 1,799 era veterans. Doebbeling et al. (2000) applied factor analysis in a sample of veterans who had been deployed and in a sample of nondeployed era controls. The deployed sample was divided into a training sample and a validation sample to evaluate the reproducibility of the factor solution for the group. Comparisons were made by correlating both factor loadings and factor scores in the deployed vs nondeployed samples. The authors identified three symptom factors in deployed veterans in the derivative sample that accounted for 35% of the variance: somatic distress (joint stiffness, myalgia, polyarthralgia, numbness or tingling, headache, and nausea), psychologic distress (feeling nervous, worrying, feeling distant or cut off; depression; and anxiety), and panic (anxiety attacks; a racing, skipping, or pounding heart; attacks of chest pain or pressure; and attacks of sweating). The researchers also conducted factor analysis in the nondeployed group and found the same three factors, which accounted for 29% of the variance. The authors concluded that their analyses did not support the existence of a new syndrome.

Oregon and Washington Veteran Studies

Investigators studied clusters of unexplained symptoms in a study of Portland area veterans by creating a new case definition of unexplained illness. Cases were identified on the basis of meeting a threshold number and combination of symptoms (cognitive and psychologic, and musculoskeletal) and on the basis of the duration of fatigue. Veterans whose symptom clusters remained unexplained at clinical examination (after exclusion of established diagnoses) were defined as constituting cases. Controls were those who at the time of clinical examination had no history of case-defining symptoms during or after their service in the Gulf War (Bourdette et al., 2001; Storzbach et al., 2000). The researchers undertook a factor analysis and then re-examined 48 symptoms in a second factor analysis. Three factors—cognitive and psychologic, mixed somatic, and musculoskeletal—were retained for followup factor analysis and accounted for 34.2% of the common variance. The authors used their three-factor solution to test the validity of their a priori case definition, which was composed of 35 symptoms encompassing musculoskeletal pain, cognitive and psychologic changes, gastrointestinal complaints, skin or mucous membrane lesions, and unexplained fatigue (discussed in greater detail in Chapter 5).

There were two major findings when the researchers compared the three-factor solution with the a priori case definition of Gulf War unexplained illnesses. First, their three factors did not include any symptoms related to the gastrointestinal system, the skin, or mucous membranes. Second, three of the symptoms—numbness in fingers or toes, clumsiness, and dizziness—were not included in the case definition. A limitation of the case-control design eliminated the possibility of examining differences between deployed and nondeployed veterans in that the study population by definition comprised only Gulf War veterans.

United Kingdom Veteran Studies

Australian Cohort

In a population-based study of all Australian Gulf War veterans, Forbes et al. (2004) applied factor analysis to findings from a 62-item symptom questionnaire that included measures of severity (“none,” “mild,” “moderate,” and “severe”). They found three factors that accounted for 47.1% of the variance: psychophysiologic distress (23 symptoms), cognitive distress (20 symptoms), and arthroneuromuscular distress (six symptoms). Those were broadly similar to factors extracted in previous analyses and were the same as factors that were based on data collected from a sample of nondeployed Australian veterans. However, although the prevalence was similar among deployed and nondeployed veterans, factor scores were higher among the deployed than among the nondeployed; the authors noted that this indicated a greater severity of symptoms. They concluded that there was no evidence of a unique pattern of self-reported symptoms in deployed veterans.

Seabee Studies

Haley et al. (1997) studied a battalion of 249 naval reservists called to active duty for the Gulf War. More than half the battalion had left the military by the time of the study; 41% of the battalion participated in the study. Of those participating, 70% reported having had a serious health problem since returning from the Gulf War. The study was the first to examine groupings of symptoms in Gulf War veterans with factor analysis. Through standardized symptom questionnaires and a two-stage exploratory factor analysis, the investigators defined what they considered to be either six syndromes or six variants of a single syndrome, which they labeled impaired cognition, confusion–ataxia, arthromyoneuropathy, phobia–apraxia, fever–adenopathy, and weakness–incontinence. One-fourth of the veterans in the study (63) were classified as having one of the six syndromes. The study was limited by its lack of a comparison group; the authors were unable to comment on the uniqueness of the factors in relation to other groups of veterans. Haley et al. (2001) attempted to replicate their factor-analysis findings in a validation cohort, which was separate from their original cohort of Seabees. The validation cohort of 335 consisted of veterans who were living in north Texas and who had registered with a VA clinic in Dallas or were recruited by advertising.

In the 2001 study, a more detailed questionnaire was used than in the earlier Seabee cohort in an effort to replicate the 1997 findings. Haley et al. (2001) undertook a series of analyses to test whether the factor structure that they found in the earlier cohort could be replicated in the larger and more representative cohort. In their confirmatory factor analysis, they imposed an additional constraint by allowing only four symptoms per factor for each of five models and compared the results of the five models with their earlier findings by using structure-equation models. The five models had either 12 or 16 measured variables, which loaded onto three first-order factors and zero or one higher-order factor. In two models, the four additional variables (or symptom factors) were allowed to load onto the primary or higher-order factors. The three primary syndrome factors were impaired cognition, confusion–ataxia, and central pain (termed arthromyoneuropathy in the original study); and the four additional variables or secondary symptom factors were chronic watery diarrhea, chronic fatigue involving excessive muscle weakness, chronic fever and night sweats, and middle and terminal insomnia. The higher-order factor was the presence of an underlying single Gulf War syndrome posited to explain all variance and covariance among the three first-order factors. Overall, 29% of participants had one or more of the three first-order factors, defined by dichotomizing the syndrome factor scale at 1.5, as in the original study. Haley et al. (2001) interpreted the results as confirming a three-factor solution, originally extracted in the Seabee cohort (Model 1). They also concluded that the three syndrome factors probably represented a higher-order syndrome, such as a single Gulf War syndrome, and that some additional symptoms (the four secondary symptom factors) appeared in all three syndrome variants. The researchers suggested that the confusion–ataxia syndrome may represent a more severe form of a single Gulf War syndrome of which impaired cognition and central pain variants (the other two syndrome factors) were less severe forms.

Knoke et al. (2000) applied factor analysis to data from a population of active-duty Seabees in response to the factor analysis conducted by Haley et al. (1997). The study population was drawn from US Navy construction-battalion personnel (Seabees) who were on active duty in 1990 and remained on active duty in 1994, when the study was conducted. The instrument contained 98 symptom questions. Among the 524 Gulf War veterans and 935 nondeployed Seabees, Knoke et al. (2000) performed three factor analyses: the first on the deployed Seabees, the second on the nondeployed Seabees, and the third on both. The three factor analyses accounted for 80%, 89%, and 93% of the total variance, and each extracted five factors. The factors were labeled insecurity or minor depression (27 symptoms), somatization (13 symptoms), depression (10 symptoms), obsessive-compulsive (seven symptoms), and malaise (seven symptoms). Knoke et al. (2000) derived standardized factor scores that were based on each solution and then evaluated whether the scores differed. Scores among the three analyses were similar for insecurity or minor depression; higher in Gulf War veterans for somatization, depression and obsessive–compulsive; and higher in nondeployed Seabees for malaise.⁹ Somatization, depression, and obsessive–compulsive affected an excess of about 20% of Gulf War veterans. This indirect method of testing the differences in factor structure led them to conclude that deployed and nondeployed veterans report “more of the same symptoms and illnesses” and that “identifying a new syndrome such as the putative Gulf War syndrome is a difficult task and is unlikely to be accomplished by factor analysis, or any other statistical methodology, performed on a small, selected group of Gulf War Veterans.” The authors also conducted a discriminant analysis to test the ability of the factors to discriminate between Gulf War–deployed and nondeployed veterans: the probability of misclassification was 7.4% in nondeployed veterans and 76.5% in Gulf War–deployed veterans. The findings were similar to those of Doebbeling et al. (2000), Fukuda et al. (1998), and Ismail et al. (1999). They concluded that there was no evidence of a unique spectrum of neurologic injury.

Iannacchione et al. (2011) conducted a validation study of the Haley et al. (1997) case definition in a larger population of Gulf War veterans. The study is discussed in Chapter 5.

Pennsylvania Air National Guard Study

In response to a request from the Department of Defense, VA, and Pennsylvania, Fukuda et al. (1998) used factor analysis and other methods to assess the health status of Gulf War Air Force veterans. The objective was to assess the prevalence and causes of an unexplained illness in members of one Air National Guard unit compared with three comparison Air Force units. The investigators aimed to organize symptoms into a case definition and to carry out clinical evaluations of members of an Air National Guard unit (the index unit). They administered a 35-item symptom inventory that included symptom severity (mild, moderate, or severe) and duration (less than 6 months or 6 months or longer) and divided the 3,255 participants who had answered all symptom questions into two subsamples of 1,631 and 1,624. The authors conducted a principal components analysis of the first subsample, extracting 10 components with eigenvalues¹⁰ greater than 1.0; three of the components accounted for 39.1% of the total variance. (See Appendix A for a discussion of the relationship between principal components analysis and factor analysis.) When the three components were examined in a confirmatory factor analysis in the second subsample, two were confirmed. The first, labeled mood–cognition–fatigue, consisted of these symptoms: feeling depressed, feeling anxious, feeling moody, difficulty in remembering or concentrating, trouble in finding words, difficulty in sleeping, and fatigue. The second, labeled musculoskeletal, consisted of these symptoms: joint stiffness, joint pain, and muscle pain. Fukuda et al. (1998) used 10 symptoms associated with the two factors from their confirmatory factor analysis to develop a preliminary case definition.

Department of Veterans Affairs Gulf War Health Registry

Hallman et al. (2003) examined patterns of reported symptoms in participants in the VA Gulf War Health Registry. The study sample consisted of a state-based random sample of 2,011 veteran registry members who resided in Delaware, Illinois, New Jersey, New York, North Carolina, Ohio, or Pennsylvania and who were not participating in other studies. Questionnaires included 48 symptoms, which were rated on a three-point ordinal scale, and were returned by 1,161 veterans (58% of the sample). The investigators divided the participants into two groups and conducted five factor analyses in each group to examine replicability. They identified four factors that accounted for 50.2% of the variance. The factors were mood–memory–fatigue (depression, anxiety, sudden mood changes, problems in concentrating and remembering, unexplained weakness, sleep problems, and unexplained fatigue), musculoskeletal (pain or numbness in joints or muscles), gastrointestinal (abdominal pain and gas, diarrhea, nausea, and vomiting), and throat and breathing (difficulty in swallowing, swollen glands, nose or sinus problems, coughing, difficulty in breathing, and difficulty in tasting). Like Cherry et al. (2001), Hallman et al. (2003) conducted a cluster analysis (see below) to examine consistency between the two statistical methods. The principal limitation of the study is the lack of a nondeployed control group, which limits its ability to identify factors that may have been peculiar to exposure in the Gulf War. By starting with presumably the most symptomatic subset of Gulf War veterans (those who had left the service and registered with the Gulf War Health Registry), the authors might have had the chance of identifying clusters unique to Gulf War veterans if they existed. However, the four factors that they identified were largely similar to factors identified by other Gulf War investigators (Fukuda et al., 1998).

United Kingdom Veteran Studies

Several groups of researchers have examined symptoms in UK veterans. Two conducted cluster analyses.

SUMMARY AND CONCLUSIONS

The studies described in this chapter, despite their methodologic differences, have findings that are similar, that is, similar groups of symptoms were identified as falling roughly into factors associated with fatigue, pain, and neurocognitive symptoms (see Table 4.1). Less commonly reported are factors that involve gastrointestinal and respiratory symptoms. Taken together, the studies' findings do not support a unique syndrome, although they do highlight more frequent and severe symptoms in Gulf War–deployed than in nondeployed.

Well-conducted factor analyses should have high participation rates and include representative samples. Some of the studies fall short on those two criteria. For example, several studies included members of only one branch of service (Fukuda et al., 1998; Haley et al., 1997; Knoke et al., 2000), collected small samples (Haley et al., 1997), or drew samples only from symptomatic groups of veterans (Haley et al., 1997; Hallman et al., 2003). Another problem is the lack of a comparison group in some of the studies, which limits investigators' ability to compare factor structure in deployed and other groups (such as nondeployed and deployed elsewhere) (Bourdette et al., 2001; Haley et al., 1997). Although results of those studies are valuable and add rich detail to the epidemiologic literature surrounding Gulf War veterans, other studies are more representative, and their results therefore more generalizable (Cherry et al., 2001; Doebbeling et al., 2000; Ismail et al., 1999; Kang et al., 2002). Findings of the more representative studies were quite similar, broadly describing neurologic, psychologic, cognitive, fatigue, and musculoskeletal symptoms. One exception to the congruence of those findings is the extraction by Kang et al. (2002) of factors that represented symptoms labeled as gastrointestinal, pulmonary, and upper respiratory.

Three studies compared factors in representative deployed and nondeployed groups (Cherry et al., 2001; Doebbeling et al., 2000; Ismail et al., 1999). The factors extracted in those studies were remarkably similar between the deployed and nondeployed groups, and the findings did not suggest a unique complex of symptoms in the deployed group. In each of the studies, as in many of the less generalizable studies, symptoms were more frequent and more severe (in papers that reported severity) in the deployed than in the nondeployed groups. The three studies that used cluster analysis had broadly similar findings. The two studies that included nondeployed comparison groups (Cherry et al., 2001; Everitt et al., 2002) failed to identify a cluster of people that presented with a unique syndrome. Each study identified a highly symptomatic cluster of people that contained a higher proportion of Gulf War veterans than of non–Gulf War veterans, ranging from 14% in Cherry et al. (2001) to 22.2% in Everitt et al. (2002). The cluster analyses were consistent with the findings of the most representative factor-analysis studies: although they did not support a unique symptom complex in Gulf War veterans, they found that these veterans were more symptomatic than their nondeployed counterparts.

Factor analysis and cluster analysis may be useful methods for making sense of the large number of symptoms potentially associated with CMI. However, the findings obtained with these methods must be validated against other observed variables. The choice of variables to include in a model is critical, and omission of key symptoms will result in models that do not capture the most salient features of CMI. In most of the studies, the percentage of variance explained is not great. The heterogeneity of the survey questions makes it difficult to account for a lot of the variance with only a few factors. Moreover, the validity of factor analysis or cluster analysis depends on the quality of data. Methodologic flaws in such studies can bias their results (Ismail and Lewis, 2006). The committee notes that neither factor analysis nor cluster analysis alone can directly produce a case definition; such definitions are the product of postprocessing of factoranalytic model results (for example, dichotomization of factor scores to operationalize a case definition).

Given the historical dependence on factor-analytic methods in CMI studies and the methodologic flaws associated with many of them, the following are suggested practices for future factor-analytic studies. More details and definitions are available in Appendix A. For interested readers, excellent published resources provide systematic and extensive descriptions of factor-analytic methods; see, for example, Brown et al. (2012), Kline (2000), Norman and Streiner (2003), Pett et al. (2003), Rummel (1970), and Stewart (1981).

• Describe the factor-analytic process in sufficient detail to allow replication, including the wording of each item, the methods used to decide the number of factors, the method of factor extraction, the method of factor rotation, and any additional postprocessing conducted by the authors.
• Because factor analysis is a family of methods rather than a single method, select a factor-analytic approach that is aligned with the research question.
• In deciding how many factors to extract in an exploratory factor analysis, consider parallel analysis—a simulation-based approach in which factors are retained only if their eigenvalues exceed what would be obtained in random samples with no underlying factors.
• Factor analysis is not ideal for application to dichotomous (such as yes–no) data, so symptom survey items should include multiple response categories for collecting symptom data.
• Account for the measurement level of the data when conducting factor analysis; for example, use a polychoric correlation matrix, rather than a Pearson correlation matrix, to account for ordinal-level data. See Appendix A for further discussion of these matrices.
• Select a factor rotation method that is consistent with expectations regarding the relationships among factors (for example, choose an oblique rotation when factors are expected to correlate with each other).
• Explicitly test whether factor-analytic results are reproducible (for example, compare with a “holdout” sample and replicate in an independent sample).
• Apply a confirmatory factor model only when there is an a priori hypothesis regarding data structure.
• To evaluate whether the factor structure of symptom data differs in different populations (for example, in deployed vs nondeployed), posit and test the question as a formal statistical hypothesis.