ADMINISTRATION OF COGNITIVE AND NEUROPSYCHOLOGICAL TESTS TO EVALUATE COGNITIVE IMPAIRMENT

In contrast to testing that relies on self-report, as outlined in the preceding chapter, evaluating cognitive functioning relies on measures of task performance to establish the severity of cognitive impairments. Such tests are commonly used in clinical neuropsychological evaluations in which the goal is to identify a patient's pattern of strengths and weaknesses across a variety of cognitive domains. These performance-based measures are standardized instruments with population-based normative data that allow the examiner to compare an individual's performance with an appropriate comparison group (e.g., those of the same age group, sex, education level, and/or race/ethnicity).

Cognitive testing is the primary way to establish severity of cognitive impairment and is therefore a necessary component in a neuropsychological assessment. Clinical interviews alone are not sufficient to establish the severity of cognitive impairments, for two reasons: (1) patients are known to be poor reporters of their own cognitive functioning (Edmonds et al., 2014; Farias et al., 2005; Moritz et al., 2004; Schacter, 1990) and (2) clinicians relying solely on clinical interviews in the absence of neuropsychological test results are known to be poor judges of patients' cognitive functioning (Moritz et al., 2004). There is a long history of neuropsychological research linking specific cognitive impairments with specific brain lesion locations, and before the advent of neuroimaging, neuropsychological evaluation was the primary way to localize brain lesions; even today, neuropsychological evaluation is critical for identifying brain-related impairments that neuroimaging cannot identify (Lezak et al., 2012). In the context of the SSA disability determination process, cognitive testing for claimants alleging cognitive impairments could be helpful in establishing a medically determinable impairment, functional limitations, and/or residual functional capacity.

The use of standardized psychological and neuropsychological measures to assess residual cognitive functioning in individuals applying for disability will increase the credibility, reliability, and validity of determinations on the basis of these claims. A typical psychological or neuropsychological evaluation is multifaceted and may include cognitive and non-cognitive assessment tools. Evaluations typically consist of a (1) clinical interview, (2) administration of standardized cognitive or non-cognitive psychological tests, and (3) professional time for interpretation and integration of data. Some neuropsychological tests are computer administered, but the majority of tests in use today are paper-and-pencil tests.

The length of an evaluation will vary depending on the purpose of the evaluation and, more specifically, the type or degree of psychological and/or cognitive impairments that need to be evaluated. A national professional survey of 1,658 neuropsychologists from the membership of American Academy of Clinical Neuropsychology (AACN), Division 40 of American Psychological Association (APA), and the National Academy of Neuropsychologists (NAN) indicated that a typical neuropsychological evaluation takes approximately 6 hours, with a range from 0.5 to 25 hours (Sweet et al., 2011). The survey also identified a number of reasons for why the duration of an evaluation varies, including reason for referral, the type or degree of psychological and/ or cognitive impairments, or factors specific to the individual.

The most important aspect of administration of cognitive and neuropsychological tests is selection of the appropriate tests to be administered. That is, selection of measures is dependent on examination of the normative data collected with each measure and consideration of the population on which the test was normed. Normative data are typically gathered on generally healthy individuals who are free from significant cognitive impairments, developmental disorders, or neurological illnesses that could compromise cognitive skills. Data are generally gathered on samples that reflect the broad demographic characteristics of the United States including factors such as age, gender, and educational status. There are some measures that also provide specific comparison data on the basis of race and ethnicity.

As discussed in detail in Chapter 3, as part of the development of any psychometrically sound measure, explicit methods and procedures by which tasks should be administered are determined and clearly spelled out. All examiners use such methods and procedures during the process of collecting the normative data, and such procedures normally should be used in any other administration. Typical standardized administration procedures or expectations include (1) a quiet, relatively distraction-free environment; (2) precise reading of scripted instructions; and (3) provision of necessary tools or stimuli. Use of standardized administration procedures enables application of normative data to the individual being evaluated (Lezak et al., 2012). Without standardized administration, the individual's performance may not accurately reflect his or her ability. An individual's abilities may be overestimated if the examiner provides additional information or guidance than what is outlined in the test administration manual. Conversely, a claimant's abilities may be underestimated if appropriate instructions, examples, or prompts are not presented.

PSYCHOMETRICS AND TESTING NORMS FOR COGNITIVE TESTS

Once a test has been administered, assuming it has been done so according to standardized protocol, the test-taker's performance can be scored. In most instances, an individual's raw score, that is the number of items on which he or she responded correctly, is translated into a standard score based on the normative data for the specific measure. In this manner, an individual's performance can be characterized by its position on the distribution curve of normal performances.

The majority of cognitive tests have normative data from groups of people who mirror the broad demographic characteristics of the population of the United States based on census data. As a result, the normative data for most measures reflect the racial, ethnic, socioeconomic, and educational attainment of the population majorities. Unfortunately, that means that there are some individuals for whom these normative data are not clearly and specifically applicable. This does not mean that testing should not be done with these individuals, but rather that careful consideration of normative limitations should be made in interpretation of results.

Selection of appropriate measures and assessment of applicability of normative data vary depending on the purpose of the evaluation. Cognitive tests can be used to identify acquired or developmental cognitive impairment, to determine the level of functioning of an individual relative to typically functioning same-aged peers, or to assess an individual's functional capacity for everyday tasks (Freedman and Manly, 2015). Clearly, each of these purposes could be relevant for SSA disability determinations. However, each of these instances requires different interpretation and application of normative data.

When attempting to identify a change in functioning secondary to neurological injury or illness, it is most appropriate to compare an individual's postinjury performance to his or her premorbid level of functioning. Unfortunately, it is rare that an individual has a formal assessment of his or her premorbid cognitive functioning. Thus, comparison of the postinjury performance to demographically matched normative data provides the best comparison to assess a change in functioning (Freedman and Manly, 2015; Heaton et al., 2001; Manly and Echemendia, 2007). For example, assessment of a change in language functioning in a Spanish-speaking individual from Mexico who has sustained a stroke will be more accurate if the individual's performance is compared to norms collected from other Spanish-speaking individuals from Mexico rather than English speakers from the United States or even Spanish-speaking individuals from Puerto Rico. In many instances, this type of data is provided in alternative normative data sets rather than the published population-based norms provided by the test publisher.

In contrast, the population-based norms are more appropriate when the purpose of the evaluation is to describe an individual's level of functioning relative to same-aged peers (Busch, 2006; Freedman and Manly, 2015). A typical example of this would be in instances when the purpose of the evaluation is to determine an individual's overall level of intellectual (i.e., IQ) or even academic functioning. In this situation, it is more relevant to compare that individual's performance to that of the broader population in which he or she is expected to function in order to quantify his or her functional capabilities. Thus, for determination of functional disability, demographically or ethnically corrected normative data are inappropriate and may actually underestimate an individual's degree of disability (Freedman and Manly, 2015). In this situation, use of otherwise appropriate standardized and psychometrically sound performance-based or cognitive tests is appropriate.

Determination of an individual's everyday functioning or vocational capacity is perhaps the evaluation goal most relevant to the SSA disability determination process. To make this determination, the most appropriate comparison group for any individual would be other individuals who are currently completing the expected vocational tasks without limitations or disability (Freedman and Manly, 2015). Unfortunately, there are few standardized measures of skills necessary to complete specific vocational tasks and, therefore, also no vocational-specific normative data at this time. This type of functional capacity is best measured by evaluation techniques that recreate specific vocational settings and monitor an individual's completion of related tasks.

Until such specific vocational functioning measures exist and are readily available for use in disability determinations, objective assessment of cognitive skills that are presumed to underlie specific functions will be necessary to quantify an individual's functional limitations. Despite limitations in normative data as outlined in Freedman and Manly (2015), formal psychometric assessment can be completed with individuals of various ethnic, racial, gender, educational, and functional backgrounds. However, the authors note that “limited research suggests that demographic adjustments reduce the power of cognitive test scores to predict every-day abilities” (e.g., Barrash et al., 2010; Higginson et al., 2013; Silverberg and Millis, 2009). In fact, they go on to state “the normative standard for daily functioning should not include adjustments for age, education, sex, ethnicity, or other demographic variables” (p. 9). Use of appropriate standardized measures by appropriately qualified evaluators as outlined in the following sections further mitigates the impact of normative limitations.

INTERPRETATION AND REPORTING OF TEST RESULTS

Interpretation of results is more than simply reporting the raw scores an individual achieves. Interpretation requires assigning some meaning to the standardized score within the individual context of the specific test-taker. There are several methods or levels of interpretation that can be used, and a combination of all is necessary to fully consider and understand the results of any evaluation (Lezak et al., 2012). This section is meant to provide a brief overview; although a full discussion of all approaches and nuances of interpretation is beyond the scope of this report, interested readers are referred to various textbooks (e.g., Groth-Marnat, 2009; Lezak et al, 2012).

ASSESSING VALIDITY OF COGNITIVE TEST PERFORMANCE

Neuropsychological tests assessing cognitive, motor, sensory, or behavioral abilities require actual performance of tasks, and they provide quantitative assessments of an individual's functioning within and across cognitive domains. The standardization of neuropsychological tests allows for comparability across test administrations. However, interpretation of an individual's performance presumes that the individual has put forth full and sustained effort while completing the tests; that is, accurate interpretation of neuropsychological performance can only proceed when the test-taker puts forth his or her best effort on the testing. If a test-taker is not able to give his or her best effort, for whatever reason, the test results cannot be interpreted as accurately reflecting the test-taker's ability level. As discussed in detail in Chapter 2, a number of studies have examined potential for malingering when there is a financial incentive for appearing impaired, suggesting anywhere from 19 to 68 percent of SSA disability applicants may be performing below their capability on cognitive tests or inaccurately reporting their symptoms (Chafetz, 2008; Chafetz et al., 2007; Griffin et al., 1996; Mittenberg et al., 2002). For a summary of reported base rates of “malingering,” see Table 2-2 of this report and the ensuing discussion. However, an individual may put forth less than optimal effort due to a variety of factors other than malingering, such as pain, fatigue, medication use, and psychiatric symptomatology (Lezak et al., 2012).

For these reasons, analysis of the entire cognitive profile for consistency is generally recommended. Specific patterns that increase confidence in the validity of a test battery and overall assessment include

• Consistency between test behavior or self-reported symptoms and incidental behavior;
• Consistency between test behavior or self-reported symptoms and what is known about brain functioning and the type and severity of injury/illness claimed;
• Consistency between test behavior or self-reported symptoms and known patterns of performance (e.g., passing easy items and failing more difficult items; better performance on cued recall and recognition tests than free recall tests; intact memory requires intact attention);
• Consistency between test behavior or self-reported symptoms and reliable collateral reports or other background information, such as medical documentation;
• Consistency between self-reported history and reliable collateral history or medical documentation; and
• Consistency across tests measuring the same cognitive domain or across tests administered at different times.

Specific tests have also been designed especially to aid in the examination of performance validity. The development of and research on these PVTs has increased rapidly during the past two decades. There have been attempts to formally quantify performance validity during testing since the mid-1900s (Rey, 1964), with much of the initial focus on examining the consistency of an individual's responses across a battery of testing, with the suggestion that inconsistency may indicate variable effort. However, a significant push for specific formal measures came in response to the increased use of neuropsychological and cognitive testing in forensic contexts, including personal injury litigation, workers compensation, and criminal proceedings in the 1980s and 1990s (Bianchini et al., 2001; Larrabee, 2012a). Given the nature of these evaluations, there was often a clear incentive for an individual to exaggerate his or her impairment or to put forth less than optimal effort during testing, and neuropsychologists were being called upon to provide statements related to the validity of test results (Slick et al., 1999). Several studies documented that use of clinical judgment and interpretation of performance inconsistencies alone was an inadequate methodology for detection of poor effort or intentionally poor performance (Faust et al., 1988; Heaton et al., 1978; van Gorp et al., 1999). As such, the need for formal standardized measures of effort and means for interpretation of these measures emerged.

PVTs are measures that assess the extent to which an individual is providing valid responses during cognitive or neuropsychological testing. PVTs are typically simple tasks that are easier than they appear to be and on which an almost perfect performance is expected based on the fact that even individuals with severe brain injury have been found capable of good performance (Larrabee, 2012b). On the basis of that expectation, each measure has a performance cut-off defined by an acceptable number of errors designed to keep the false-positive rate low. Performances below these cutoff points are interpreted as demonstrating invalid test performance.

Types of PVTs

PVTs may be designed as such and embedded within other cognitive tests, later derived from standard cognitive tests, or designed as stand-alone measures. Examples of each type of measure are discussed below.

Embedded and Derived Measures

Embedded and derived PVTs are similar in that a specific score or assessment of response bias is determined from an individual's performance on an aspect of a preexisting standard cognitive measure. The primary difference is that embedded measures consist of indices specifically created to assess validity of performance in a cognitive test, whereas derived measures typically use novel calculations of performance discrepancies rather than simply examining the pattern of performance on already established indices. The rationale for this type of PVT is that it does not require administration of any additional tasks and therefore does not result in any added time or cost. Additionally, development of these types of PVTs can allow for retrospective consideration or examination of effort in batteries in which specific stand-alone measures of effort were not administered (Solomon et al., 2010).

The forced-choice condition of the California Verbal Learning Test—second edition (CVLT-II) (Delis et al., 2000) is an example of an embedded PVT. Following learning, recall, and recognition trials involving a 16-item word list, the test-taker is presented with pairs of words and asked to identify which one was on the list. More than 92 percent of the normative population, including individuals in their eighties, scored 100 percent on this test. Scores below the published cut-off are unusually low and indicative of potential noncredible performance. Scores below chance are considered to reflect purposeful noncredible performance, in that the test-taker knew the correct answer but purposely chose the wrong answer.

Reliable Digit Span, based on the Digit Span subtest of the Wechsler Adult Intelligence Scale, is an example of a measure that was derived based on research following test publication. The Digit Span subtest requires test-takers to repeat strings of digits in forward order (forward digit span), as well as in reverse order (backward digit span). To calculate Reliable Digit Span, the maximum forward and backward span are summed, and scores below the cut-off point are associated with noncredible performance (Greiffenstein et al., 1994). A full list of embedded and derived PVTs is provided in Table 5-1.

TABLE 5-1

Embedded and Derived PVTs.

Stand-Alone Measures

A stand-alone PVT is a measure that was developed specifically to assess a test-taker's effort or consistency of responses. That is, although the measure may appear to assess some other cognitive function (e.g., memory), it was actually developed to be so simple that even an individual with severe impairments in that function would be able to perform adequately. Such measures may be forced choice or non-forced choice (Boone and Lu, 2007; Grote and Hook, 2007).

The Test of Memory Malingering (TOMM) (Tombaugh and Tombaugh, 1996), the Word Memory Test (WMT) (Green et al., 1996), and the Rey Memory for Fifteen Items Test (RMFIT) (Rey, 1941) are examples of standalone measures of performance validity. As with many stand-alone measures, the TOMM, WMT, and RMFIT are memory tests that appear more difficult than they really are. The TOMM and WMT use a forced-choice method to identify noncredible performance in which the test-taker is asked to identify which of two stimuli was previously presented. Accuracy scores are compared to chance level performance (i.e., 50 percent correct), as well as performance by normative groups of head-injured and cognitively impaired individuals, with cut-offs set to minimize false-positive errors. Alternatively, the RMFIT uses a non-forced-choice method in which the test-taker is presented with a group of items and then asked to reproduce as many of the items as possible.

Forced-Choice PVTs

As noted above, some PVTs are forced-choice measures on which performance significantly below chance has been suggested to be evidence of intentionally poor performance based on application of the binomial theorem (Larrabee, 2012a). For example, if there are two choices, it would be expected that purely random guessing would result in 50 percent of items correct. Scores deviating from 50 percent in either direction indicate nonchance-level performance. The most probable explanation for substantially below-chance PVT scores is that the test-taker knew the correct answer but purposely selected the wrong answer. The Slick and colleagues (1999) criteria for malingered neurocognitive dysfunction include below chance performance (P < 0.05) on one or more forced-choice measures of performance validity as indicative of malingering, and the authors state that “short of confession,” below-chance performance on performance validity testing is “closest to an evidentiary ‘gold standard’ for malingering.” Though below-chance performance on forced-choice PVTs implies intent, the committee believes it does not necessarily imply malingering, because the motivation of the performance may not be known; however, it does mean that the remainder of the test battery cannot be interpreted. A list of forced-choice PVTs can be found in Table 5-2.

TABLE 5-2

Forced-Choice PVTs.

APPLICANT POPULATIONS FOR WHOM PERFORMANCE-BASED TESTS SHOULD BE CONSIDERED OR USED

As suggested above, there are many applicants for whom administration of cognitive or neuropsychological testing would be beneficial to improve the standardization and credibility of determinations based on allegations of disability on the basis of cognitive impairment. The discussion below should not be considered all-inclusive, but rather as an attempt to highlight categories of disability applicants in which cognitive or performance-based testing would be appropriate.

CONCLUSION

The results of standardized cognitive tests that are appropriately administered, interpreted, and validated can provide objective evidence to help identify and document the presence and severity of medically determinable mental impairments at Step 2 of SSA's disability determination process. In addition, such tests can provide objective evidence to help identify and assess the severity of work-related cognitive functional impairment relevant to disability evaluations at the listing level (Step 3) and to mental residual functional capacity (Steps 4 and 5).Therefore, standardized cognitive test results are essential to the determination of all cases in which an applicant's allegation of cognitive impairment is not accompanied by objective medical evidence.

The results of cognitive tests are affected by the effort put forth by the test-taker. If an individual has not given his or her best effort in taking the test, the results will not provide an accurate picture of the person's neuropsychological or cognitive functioning. Performance validity indicators, which include PVTs, analysis of internal data consistency, and other corroborative evidence, help the evaluator to interpret the validity of an individual's neuropsychological or cognitive test results. For this reason, it is important to include an assessment of performance validity at the time cognitive testing is administered. It also is important that validity be assessed throughout the cognitive evaluation.

PVTs provide information about the validity of cognitive test results when administered as part of the test or test battery and are an important addition to the medical evidence of record for specific groups of applicants. It is important that PVTs only be administered in the context of a larger test battery and only be used to interpret information from that battery. Evidence of invalid performance based on PVT results pertains only to the cognitive test results obtained and does not provide information about whether or not the individual is, in fact, disabled. A lack of validity on PVTs alone is insufficient grounds for denying a disability claim.

REFERENCES

• AACN (American Academy of Clinical Neuropsychology). AACN practice guidelines for neuropsychological assessment and consultation. Clinical Neuropsychology. 2007;21(2):209–231. [PubMed: 17455014]
• Allen LM III, Conder RL, Green P, Cox DR. CARB ‘97: Manual for the computerized assessment of response bias. Durham, NC: Cognisyst; 1997.
• American Psychiatric Association. The diagnostic and statistical manual of mental disorders: DSM-5. Washington, DC: American Psychiatric Association; 2013.
• APA (American Psychological Association). Guidelines and principles for accreditation of programs in professional psychology: Quick reference guide to doctoral programs. 2015. [January 20, 2015]. http://www​.apa.org/ed​/accreditation/about/policies/doctoral​.aspx .
• Barrash J, Stillman A, Anderson SW, Uc Y, Dawson JD, Rizzo M. Predicition of driving ability with neuropsychological tests: Demographic adjustments diminish accuracy. Journal of the International Neuropsychological Society. 2010;16(04):679–686. [PMC free article: PMC3152745] [PubMed: 20441682]
• Benedict RH. Brief visuospatial memory test—revised: Professional manual. Lutz, FL: Psychological Assessment Resources; 1997.
• Benedict RH, Schretlen D, Groninger L, Brandt J. Hopkins Verbal Learning Test–Revised: Normative data and analysis of inter-form and test-retest reliability. The Clinical Neuropsychologist. 1998;12(1):43–55.
• Benton AL, de Hamsher KS, Varney NR, Spreen O. Contributions to neuropsychological assessment: A clinical manual. New York: Oxford University Press; 1983.
• Benton L, de Hamsher K, Sivan A. Multilingual Aphasia Examination. 1994a. Controlled oral word association test; p. 3.
• Benton AL, de Hamsher KS, Varney NR, Spreen O. Contributions to neuropsychological assessment: A clinical manual—second edition. New York: Oxford University Press; 1994b.
• Berthelson L, Mulchan SS, Odland AP, Miller LJ, Mittenberg W. False positive diagnosis of malingering due to the use of multiple effort tests. Brain Injury. 2013;27(7-8):909–916. [PubMed: 23782260]
• Bianchini KJ, Mathias CW, Greve KW. Symptom validity testing: A critical review. The Clinical Neuropsychologist. 2001;15(1):19–45. [PubMed: 11778576]
• Bigler ED. Symptom validity testing, effort, and neuropsychological assessment. Journal of the International Neuropsychological Society. 2012;18(4):632–642. [PubMed: 23057080]
• Bigler ED. Limitations with symptom validity, performance validity, and effort tests; Presentation to IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration; June 25, 2014; Washington, DC. 2014.
• Bigler ED. Use of symptom validity tests and performance validity tests in disability determinations; Paper commissioned by the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations; 2015. [April 9, 2015]. http://www​.iom.edu/psychtestingpaperEB .
• Bilder RM, Sugar CA, Hellemann GS. Cumulative false positive rates given multiple performance validity tests: Commentary on Davis and Millis (2014) and Larrabee (2014). The Clinical Neuropsychologist. 2014;28(8):1212–1223. [PMC free article: PMC4331348] [PubMed: 25490983]
• Binder LM. Portland Digit Recognition Test manual—second edition. Portland, OR: Private Publication; 1993.
• Binder LM, Willis SC. Assessment of motivation after financially compensable minor head trauma. Psychological Assessment. 1991;3(2):175–181.
• Binder LM, Villanueva MR, Howieson D, Moore RT. The Rey AVLT recognition memory task measures motivational impairment after mild head trauma. Archives of Clinical Neuropsychology. 1993;8:137–147. [PubMed: 14589671]
• Binder LM, Iverson GL, Brooks BL. To err is human: “Abnormal” neuropsychological scores and variability are commin in healthy adults. Archives of Clinical Neuropsychology. 2009;24:31–46. [PubMed: 19395355]
• Boone KB. Assessment of feigned cognitive impairment: A neuropsychological perspective. New York: Guilford Press; 2007.
• Boone KB. The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations. The Clinical Neuropsychologist. 2009;23(4):729–741. [PubMed: 18949583]
• Boone KB. Selection and use of multiple performance validity tests (PVTs); Presentation to IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration; June 25, 2014; Washington, DC. 2014.
• Boone KB, Lu P. Non-forced-choice effort measures. In: Larrabee GJ, editor. Assessment of malingered neurocognitive deficits. New York: Oxford University Press; 2007. pp. 27–43.
• Boone KB, Lu P, Back C, King C, Lee A, Philpott L, Shamieh E, Warner-Chacon K. Sensitivity and specificity of the Rey Dot Counting Test in patients with suspect effort and various clinical samples. Archives of Clinical Neuropsychology. 2002a;17(7):625–642. [PubMed: 14591847]
• Boone KB, Lu PH, Herzberg D. The B Test manual. Los Angeles: Western Psychological Services; 2002b.
• Boone KB, Lu P, Wen J. Comparison of various RAVLT scores in the detection of non-credible memory performance. Archives of Clinical Neuropsychology. 2005;20:301–319. [PubMed: 15797167]
• Brandt J, Benedict RH. Hopkins Verbal Learning Test, Revised: Professional manual. Lutz, FL: Psychological Assessment Resources; 2001.
• Brandt J, van Gorp W. American Academy of Clinical Neuropsychology policy on the use of non-doctoral-level personnel in conducting clinical neuropsychological evaluations. The Clinical Neuropsychologist. 1999;13(4):385.
• Busch RM, Chelune GJ, Suchy Y. Using norms in neuropsychological assessment of the elderly. In: Attix DK, Welsh-Bohmer KA, editors. Geriatric neuropsychology: Assessment and intervention. New York: Guilford Press; 2006.
• Bush SS, Ruff RM, Tröster AI, Barth JT, Koffler SP, Pliskin NH, Reynolds CR, Silver CH. Symptom validity assessment: Practice issues and medical necessity. NAN policy & planning committee; Archives of Clinical Neuropsychology. 2005;20(4):419–426. [PubMed: 15896556]
• Carone DA. Children with moderate/severe brain damage/dysfunction outperform adults with mild-to-no brain damage on the Medical Symptom Validity Test. Brain Injury. 2008;22(12):960–971. [PubMed: 19005888]
• Carrow-Woolfolk E. CASL: Comprehensive Assessment of Spoken Language. Circle Pines, MN: American Guidance Services; 1999.
• Chafetz MD. Malingering on the Social Security disability consultative exam: Predictors and base rates. The Clinical Neuropsychologist. 2008;22(3):529–546. [PubMed: 17853151]
• Chafetz MD. The psychological consultative examination for Social Security disability. Psychological Injury and Law. 2011;4(3-4):235–244.
• Chafetz MD, Underhill J. Estimated costs of malingered disability. Archives of Clinical Neuropsychology. 2013;28(7):633–639. [PubMed: 23800432]
• Chafetz MD, Abrahams JP, Kohlmaier J. Malingering on the Social Security disability consultative exam: A new rating scale. Archives of Clinical Neuropsychology. 2007;22(1):1–14. [PubMed: 17097263]
• Conder R, Allen L, Cox D. Computerized Assessment of Response Bias test manual. Durham, NC: Cognisyst; 1992.
• Davis JJ, Millis SR. Examination of performance validity test failure in relation to number of tests administered. The Clinical Neuropsychologist. 2014;28(2):199–214. [PubMed: 24528190]
• Dean AC, Victor TL, Boone KB, Arnold G. The relationship of IQ to effort test performance. The Clinical Neuropsychologist. 2008;22(4):705–722. [PubMed: 17853124]
• Delis DC. CVLT-C, California Verbal Learning Test: Children's version: Manual. San Antonio, TX: The Psychological Corporation; 1994.
• Delis DC, Kramer JH, Kaplan E. California Verbal Learning Test: CVLT-II; adult version; manual. San Antonio, TX: The Psychological Corporation; 2000.
• Delis D, Kaplan E, Kramer J. Delis-Kaplan executive function system. San Antonio, TX: The Psychological Corporation; 2001.
• DeRight J, Carone DA. Assessment of effort in children: A systematic review. Child Neuropsychology. 2015;21(1):1–24. [PubMed: 24344790]
• Edmonds EC, Delano-Wood L, Galasko DR, Salmon DP, Bondi MW. Subjective cognitive complaints contribute to misdiagnosis of mild cognitive impairment. Journal of the International Neuropsychological Society. 2014;20(8):836–847. [PMC free article: PMC4172502] [PubMed: 25156329]
• Elliott R. Executive functions and their disorders. British Medical Bulletin. 2003;65:49–59. [PubMed: 12697616]
• Etherton JL, Bianchini KJ, Ciota MA, Greve KW. Reliable Digit Span is unaffected by laboratory-induced pain: Implications for clinical use. Assessment. 2005a;12(1):101–106. [PubMed: 15695748]
• Etherton JL, Bianchini KJ, Greve KW, Ciota MA. Test of Memory Malingering performance is unaffected by laboratory-induced pain: Implications for clinical use. Archives of Clinical Neuropsychology. 2005b;20(3):375–384. [PubMed: 15797173]
• Etkin A, Gyurak A, O'Hara R. A neurobiological approach to the cognitive deficits of psychiatric disorders. Dialogues in Clinical Neuroscience. 2013;15(4):419. [PMC free article: PMC3898680] [PubMed: 24459409]
• Farias ST, Mungas D, Jagust W. Degree of discrepancy between self and other reported everyday functioning by cognitive status: Dementia, mild cognitive impairment, and healthy elders. International Journal of Geriatric Psychiatry. 2005;20(9):827–834. [PMC free article: PMC2872134] [PubMed: 16116577]
• Faust D, Hart K, Guilmette T, Arkes H. Neuropsychologists' capacity to detect adolescent malingerers. Professional Psychology: Research and Practice. 1988;19:508–515.
• Frederick RI. Validity indicator profile manual. Minnetonka, MN: NCS Assessments; 1997.
• Frederick RI, Foster HG. Multiple measures of malingering on a forced-choice test of cognitive ability. Psychological Assessment. 1991;3(4):596–602.
• Freedman D, Manly J. Use of normative data and measures of performance validity and symptom validity in assessment of cognitive function; Paper commissioned by the IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration Disability Determinations; 2015. [April 9, 2015]. http://www​.iom.edu/psychtestingpapersDFJM .
• Funahashi S. Neuronal mechanisms of executive control by the prefrontal cortex. Neuroscience Research. 2001;39:147–165. [PubMed: 11223461]
• Gast J, Hart KJ. The performance of juvenile offenders on the Test of Memory Malingering. Journal of Forensic Psychology Practice. 2010;10(1):53–68.
• Gervais RO, Rohling ML, Green P, Ford W. A comparison of WMT, CARB, and TOMM failure rates in non-head injury disability claimants. Archives of Clinical Neuropsychology. 2004;19(4):475–487. [PubMed: 15163449]
• Goodglass H, Kaplan E. Boston diagnostic aphasia examination. Philadelphia: Lea & Febiger; 1983.
• Graue LO, Berry DT, Clark JA, Sollman MJ, Cardi M, Hopkins J, Werline D. Identification of feigned mental retardation using the new generation of malingering detection instruments: Preliminary findings. The Clinical Neuropsychologist. 2007;21(6):929–942. [PubMed: 17886151]
• Green P. Green's Memory Complaints Inventory (MCI). Edmonton, Alberta, Canada: Green's; 2004.
• Green P. Green's Word Memory Test for Window's: User's manual. Edmonton, Alberta, Canada: Green's; 2005.
• Green P. Manual for Nonverbal Medical Symptom Validity Test. Edmonton, Alberta, Canada: Green's; 2008.
• Green P, Flaro L. Word Memory Test performance in children. Child Neuropsychology. 2003;9(3):189–207. [PubMed: 13680409]
• Green P, Allen L, Astner K. The Word Memory Test: A user's guide to the oral and computer-administered forms, U.S. version 1.1. Durham, NC: CogniSyst; 1996.
• Greiffenstein MF, Baker WJ, Gola T. Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment. 1994;6(3):218–224.
• Greiffenstein M, Gervais R, Baker WJ, Artiola L, Smith H. Symptom validity testing in medically unexplained pain: A chronic regional pain syndrome type 1 case series. The Clinical Neuropsychologist. 2013;27(1):138–147. [PubMed: 23062188]
• Greve KW, Bianchini KJ. Setting empirical cutoffs on psychometric indicators of negative response bias: A methodological commentary with recommendations. Archives of Clinical Neuropsychology. 2004;19(4):533–541. [PubMed: 15163454]
• Griffin GA, Normington J, May R, Glassmire D. Assessing dissimulation among Social Security disability income claimants. Journal of Consulting Clinical Psychology. 1996;64(6):1425–1430. [PubMed: 8991329]
• Gronwall D. Paced auditory serial-addition task: A measure of recovery from concussion. Perceptual and Motor Skills. 1977;44(2):367–373. [PubMed: 866038]
• Grote LG, Hook JN. Forced-choice recognition tests of malingering. In: Larrabee GJ, editor. Assessment of malingered neurocognitive deficits. New York: Oxford University Press; 2007. pp. 27–43.
• Groth-Marnat G. Handbook of psychological assessment. Hoboken, NJ: John Wiley & Sons; 2009.
• Hammill DD, Larsen SC. Test of written language: Examiner's manual. 4th. Austin, TX: Pro-Ed.; 2009.
• Hampson NE, Kemp S, Coughlan AK, Moulin CJ, Bhakta BB. Applied Neuropsychology: Adult. ahead-of-print. 2013. Effort test performance in clinical acute brain injury, community brain injury, and epilepsy populations; pp. 1–12. [PubMed: 25084843]
• Heaton RK. Wisconsin Card Sorting Test: Computer version 2. Odessa, FL: Psychological Assessment Resources; 1993.
• Heaton RK, Smith HH, Lehman RA, Vogt AT. Prospects for faking believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology. 1978;46(5):892. [PubMed: 701568]
• Heaton RK, Grant I, Matthews CG. Comprehensive norms for an expanded Halstead-Reitan Battery: Demographic corrections, research findings, and clinical applications. Odessa, FL: Psychological Assessment Resources; 1991.
• Heaton RK, Taylor M, Manly J. Demographic effects and demographically corrected norms with the WAIS-III and WMS-III. In: Tulsky D, Heaton RK, Chelune GJ, Ivnik I, Bornstein RA, Prifitera A, Ledbetter M, editors. Clinical interpretations of the WAIS-II and WMS-III. San Diego, CA: Academic Press; 2001. pp. 181–210.
• Heilbronner RL, Sweet JJ, Morgan JE, Larrabee GJ, Millis SR. Conference Participants. American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist. 2009;23(7):1093–1129. [PubMed: 19735055]
• Higginson CI, Lanni K, Sigvardt KA, Disbrow EA. The contribution of trail making to the prediction of performance-based instrumental activities of daily living in Parkinson's disease without dementia. Journal of Clinical and Experimental Neuropsychology. 2013;35(5):530–539. [PMC free article: PMC3674142] [PubMed: 23663116]
• Hiscock M, Hiscock CK. Refining the forced-choice method for the detection of malingering. Journal of Clinical and Experimental Neuropsychology. 1989;11(6):967–974. [PubMed: 2531755]
• HNS (Houston Neuropsychological Society). The Houston Conference on Specialty Education and Training in Clinical Neuropsychology policy statement. 2003. [November 25, 2014]. http://www​.uh.edu/hns/hc.html .
• Holdnack JA, Drozdick LW. Advanced clinical solutions for WAIS-IV and WMS-IV: Clinical and interpretive manual. San Antonio, TX: Pearson; 2009.
• Hurley KE, Deal WP. Assessment instruments measuring malingering used with individuals who have mental retardation: Potential problems and issues. Mental Retardation. 2006;44(2):112–119. [PubMed: 16689611]
• Iverson GL, Franzen MD. Using multiple objective memory procedures to detect simulated malingering. Journal of Clinical and Experimental Neuropsychology. 1996;18(1):38–51. [PubMed: 8926295]
• Jelicic M, Merckelbach H, Candel I, Geraets E. Detection of feigned cognitive dysfunction using special malinger tests: A simulation study in naïve and coached malingerers. The International Journal of Neuroscience. 2007;117(8):1185–1192. [PubMed: 17613120]
• Johnson-Greene D, Brooks L, Ference T. Relationship between performance validity testing, disability status, and somatic complaints in patients with fibromyalgia. The Clinical Neuropsychologist. 2013;27(1):148–158. [PubMed: 23121595]
• Kaplan E, Goodglass H, Weintraub S. Boston Naming Test. Austin, TX: Pro-Ed.; 2001.
• Killgore WD, DellaPietra L. Using the WMS-III to detect malingering: Empirical validation of the rarely missed index (RMI). Journal of Clinical and Experimental Neuropsychology. 2000;22:761–771. [PubMed: 11320434]
• Kirkwood M. Validity testing in pediatric populations; Presentation to IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration; June 25, 2014; Washington, DC. 2014.
• Kirkwood MW, Yeates KO, Randolph C, Kirk JW. The implications of symptom validity test failure for ability-based test performance in a pediatric sample. Psychological Assessment. 2012;24(1):36–45. [PubMed: 21767023]
• Larrabee GJ. Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist. 2003;17(3):410–425. [PubMed: 14704892]
• Larrabee GJ. Introduction: Malingering, research designs, and base rates. In: Larrabee GJ, editor. Assessment of malingered neuropsychological deficits. New York: Oxford University Press; 2007.
• Larrabee GJ. Assessment of malingering. In: Larrabee GJ, editor. Forensic neuropsychology: A scientific approach. New York: Oxford University Press; 2012a.
• Larrabee GJ. Performance validity and symptom validity in neuropsychological assessment. Journal of the International Neuropsychological Society. 2012b;18(4):625–630. [PubMed: 23057079]
• Larrabee GJ. False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology. 2014a;29(4):364–373. [PubMed: 24769887]
• Larrabee GJ. Performance and Symptom Validity; Presentation to IOM Committee on Psychological Testing, Including Validity Testing, for Social Security Administration; June 25, 2014; Washington, DC. 2014b.
• Lewis RF. Digit Vigilance Test. Lutz, FL: Psychological Assessment Resources; 1990.
• Lezak M, Howieson D, Bigler E, Tranel D. Neuropsychological assessment. 5th. New York: Oxford University Press; 2012.
• Lu PH, Boone KB, Cozolino L, Mitchell C. Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers Recognition Trial in the detection of suspect effort. The Clinical Neuropsychologist. 2003;17:426–440. [PubMed: 14704893]
• MacAllister WS, Nakhutina L, Bender HA, Karantzoulis S, Carlson C. Assessing effort during neuropsychological evaluation with the TOMM in children and adolescents with epilepsy. Child Neuropsychology. 2009;15(6):521–531. [PubMed: 19424879]
• Manly J, Echemendia R. Race-specific norms: Using the model of hypertension to understand issues of race, culture, and education in neuropsychology. Archives of Clinical Neuropsychology. 2007;22(3):319–325. [PubMed: 17350797]
• McCrea M, Kelly JP, Randolph C, Cisler R, Berger L. Immediate neurocognitive effects of concussion. Neurosurgery. 2002;50(5):1032–1042. [PubMed: 11950406]
• McCrea M, Guskiewicz KM, Marshall SW, Barr W, Randolph C, Cantu RC, Onate JA, Yang J, Kelly JP. Acute effects and recovery time following concussion in collegiate football players: The NCAA concussion study. JAMA. 2003;290(19):2556–2563. [PubMed: 14625332]
• Meyers JE, Volbrecht M. Detection of malingers using the Rey Complex Figure and Recognition Trial. Applied Neuropsychology. 1999;6:201–207. [PubMed: 10635434]
• Mittenberg W, Patton C, Canyock EM, Condit DC. Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology. 2002;24(8):1094–1102. [PubMed: 12650234]
• Mittenberg W, Patton C, Legler W. Identification of malingered head injury on the Wechsler Memory Scale—Third Edition; Paper presented at the annual conference of the National Academy of Neuropsychology; Dallas, TX. 2003.
• Moritz S, Ferahli S, Naber D. Memory and attention performance in psychiatric patients: Lack of correspondence between clinician-rated and patient-rated functioning with neuropsychological test results. Journal of the International Neuropsychological Society. 2004;10(4):623–633. [PubMed: 15327740]
• NAN (National Academy of Neuropsychology). NAN definition of a clinical neuropsychologist: Official position of the National Academy of Neuropsychology. 2001. [November 25, 2014]. https://www​.nanonline​.org/docs/PAIC/PDFs​/NANPositionDefNeuro.pdf .
• Niccolls R, Bolter JF. Multi-Digit Memory Test. San Luis Obispo, CA: Wang Neuropsychological Laboratories; 1991.
• NIH (National Institutes of Health). NIH toolbox: Processing speed. n.d. [October 15, 2014]. http://www​.nihtoolbox​.org/WhatAndWhy/Cognition​/ProcessingSpeed/Pages/default​.aspx .
• OIDAP (Occupational Information Development Advisory Panel). Mental cognitive subcommittee: Content model and classification recommendations. 2009. [October 6, 2014]. http://www​.ssa.gov/oidap​/Documents/AppendixC.pdf .
• Paulhus DL. Paulhus Deception Scales (PDS). Toronto: Multi-Health Systems; 1998.
• Randolph C. Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). San Antonio, TX: Psychological Corporation; 1998. [PubMed: 9845158]
• Rao SM. Neuropsychology of multiple sclerosis: A critical review. Journal of Clinical and Experimental Neuropsychology. 1986;8(5):503–542. [PubMed: 3805250]
• Reitan RM. Trail Making Test: Manual for administration and scoring. Mesa, AZ: Reitan Neuropsychology Laboratory; 1992.
• Reitan RM, Wolfson D. The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation—second edition. Tucson: Neuropsychology Press; 1993.
• Rey A. L'examen psychologique dans les cas d'encéphalopathie traumatique (les problems). Archives de Psychologie. 1941;28:286–340.
• Rey A. The clinical examination in psychology. Paris, France: Presses Universitaires de France; 1964.
• Roberson CJ, Boone KB, Goldberg H, Miora D, Cottingham M, Victor T, Ziegler E, Zeller M, Wright M. Cross validation of the B test in a large known groups sample. The Clinical Neuropsychologist. 2013;27(3):495–508. [PubMed: 23157695]
• Ruben RJ. Redefining the survival of the fittest: Communication disorders in the 21st century. International Journal of Pediatric Otorhinolaryngology. 1999;49:S37–S38. [PubMed: 10577772]
• Salazar XF, Lu PH, Wen J, Boone KB. The use of effort tests in ethnic minorities and in non-English-speaking and English as a second language populations. In: Boone KB, editor. Assessment of feigned cognitive impairment: A neuropsychological perspective. New York: Guilford Press; 2007. pp. 405–427.
• Salekin KL, Doane BM. Malingering intellectual disability: The value of available measures and methods. Applied Neuropsychology. 2009;16(2):105–113. [PubMed: 19430992]
• Schacter DL. Toward a cognitive neuropsychology of awareness: Implicit knowledge and anosognosia. Journal of Clinical and Experimental Neuropsychology. 1990;12(1):155–178. [PubMed: 2406281]
• Schmidt M. Rey Auditory Verbal Learning Test: RAVLT: A handbook. Los Angeles: Western Psychological Services; 1996.
• Schretlen DJ, Testa S, Winicki JM, Pearlson GD, Gordon B. Frequency and bases of abnormal performance by healthy adults on neuropsychological testing. Journal of the International Neuropsychological Society. 2008;14(3):436–445. [PubMed: 18419842]
• Semel E, Wiig E, Secord W. Clinical evaluation of language fundamentals: Examiners manual. 4th. San Antonio, TX: The Psychological Corporation; 2003.
• Shandera AL, Berry DT, Clark JA, Schipper LJ, Graue LO, Harp JP. Detection of malingered mental retardation. Psychological Assessment. 2010;22(1):50–56. [PubMed: 20230151]
• Sheslow D, Adams W. Wide range assessment of memory and learning second edition administration and technical manual. Lutz, FL: Psychological Assessment Resources; 2003.
• Silverberg ND, Millis SR. Impairment versus deficiency in neuropsychological assessment: Implications for ecological validity. Journal of the International Neuropsychological Society. 2009;15(1):94–102. [PubMed: 19128532]
• Silverton L. Malingering Probability Scale (MPS) manual. Los Angeles, CA: Western Psychological Services; 1999.
• Slick DJ, Hopp G, Strauss E, Thompson GB. Victoria Symptom Validity Test: Professional manual. Odessa, FL: Psychological Assessment Resources; 1997.
• Slick DJ, Sherman EMS, Iverson GL. Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist (Neuropsychology, Development and Cognition: Section D). 1999;13(4):545–561. [PubMed: 10806468]
• Sollman MJ, Berry DT. Detection of inadequate effort on neuropsychological testing: A meta-analytic update and extension. Archives of Clinical Neuropsychology. 2011;26(8):774–789. [PubMed: 22100969]
• Solomon RE, Boone KB, Miora D, Skidmore S, Cottingham M, Victor T, Ziegler E, Zeller M. Use of the WAIS-III picture completion subtest as an embedded measure of response bias. The Clinical Neuropsychologist. 2010;24(7):1243–1256. [PubMed: 20924983]
• Spreen O, Strauss E. Controlled oral word association (word fluency). In: Spreen O, Strauss E, editors. A compendium of neuropsychological tests. Oxford, UK: Oxford University Press; 1991. pp. 219–227.
• SSA (Social Security Administration). Disability evaluation under social security—Part III: Listing of impairments—Adult listings (Part A)—section 12.00 mental disorders. n.d.-a. [November 14, 2014]. http://www​.ssa.gov/disability​/professionals/bluebook/12​.00-MentalDisorders-Adult.htm .
• SSA. Disability evaluation under Social Security: Part I—general information. n.d.-b. [November 14, 2014]. http://www​.ssa.gov/disability​/professionals​/bluebook/general-info.htm .
• Stevens A, Schneider K, Liske B, Hermle L, Huber H, Hetzel G. Is subnormal cognitive performance in schizophrenia due to lack of effort or to cognitive impairment? German Journal of Psychiatry. 2014;17(1):9.
• Strauss E, Sherman EM, Spreen O. A compendium of neuropsychological tests: Administration, norms, and commentary. Oxford, UK: Oxford University Press; 2006.
• Suchy Y, Chelune G, Franchow EI, Thorgusen SR. Confronting patients about insufficient effort: The impact on subsequent symptom validity and memory performance. The Clinical Neuropsychologist. 2012;26(8):1296–1311. [PubMed: 23061472]
• Suhr JA, Boyer D. Use of the Wisconsin Card Sorting Test in the detection of malingering in student simulator and patient samples. Journal of Clinical and Experimental Neuropsychology. 1999;21:701–708. [PubMed: 10572288]
• Sweet JJ, Meyer DG, Nelson NW, Moberg PJ. The TCN/AACN 2010 “salary survey”: Professional practices, beliefs, and incomes of U.S. neuropsychologists. The Clinical Neuropsychologist. 2011;25(1):12–61. [PubMed: 21253962]
• Tombaugh TN, Tombaugh PW. Test of Memory Malingering: TOMM. North Tonawanda, NY: Multi-Health Systems; 1996.
• Trahan DE, Larrabee GJ. Continuous Visual Memory Test. Odessa, FL: Psychological Assessment Resources; 1988.
• van Gorp WG, Humphrey LA, Kalechstein A, Brumm VL, McMullen WJ, Stoddard M, Pachana NA. How well do standard clinical neuropsychological tests identify malingering?: A preliminary analysis. Journal of Clinical and Experimental Neuropsychology. 1999;21(2):245–250. [PubMed: 10425521]
• Victor TL, Boone KB. Identification of feigned mental retardation. In: Boone K, editor. Assessment of feigned cognitive impairment. New York: Guilford Press; 2007. pp. 310–345.
• Victor TL, Boone K, Serpa JG, Buehler J, Ziegler E. Interpreting the meaning of multiple symptom validity test failure. The Clinical Neuropsychologist. 2009;23(2):297–313. [PubMed: 18821138]
• Warrington E. Recognition Memory Test manual. Windsor: Nfer-Nelson; 1984.
• Wechsler D. Wechsler Adult Intelligence Scale (WAIS-III): Administration and scoring manual—3rd edition. San Antonio, TX: The Psychological Corporation; 1997a.
• Wechsler D. WMS-III: Wechsler Memory Scale administration and scoring manual. San Antonio, TX: The Psychological Corporation; 1997b.
• Wechsler D. Wechsler Intelligence Scale for Children—fourth edition (WISC-IV). San Antonio, TX: The Psychological Corporation; 2003.
• Wechsler D. Wechsler Adult Intelligence Scale—fourth edition (WAIS-IV). San Antonio, TX: NCS Pearson; 2008.
• Wechsler D. WMS-IV: Wechsler Memory Scale—Administration and scoring manual. San Antonio, TX: The Psychological Corporation; 2009.
• WHO (World Health Organization). International classification of functioning, disability, and health (ICF). Geneva, Switzerland: WHO; 2001.
• Young G. Resource material for ethical psychological assessment of symptom and performance validity, including malingering. Psychological Injury and Law. 2014;7(3):206–235.

Footnotes

1

As documented in Chapters 1 and 2, 57 percent of claims fall under mental disorders other than intellectual disability and/or connective tissue disorders.

2

Public comments are currently under review and a final rule has yet to be published as of the publication of this report.

3

At the committee's second meeting, Drs. Bianchini, Boone, and Larrabee all expressed great concern about the susceptibility of PVTs to coaching and stressed the importance of ensuring test security, as disclosure of test materials adversely affects the reliability and validity of psychological test results.

4

The exception being a single below-chance failure on a forced-choice PVT is sufficient to render scores invalid.