Aim

To summarize the validity of the following outcome measures used in the RADIANCE study.

• Visual acuity measurement: Snellen charts and the Early Treatment Diabetic Retinopathy Study (ETDRS) Letters score
• National Eye Institute Visual Function Questionnaire 25 (NEI-VFQ-25)
• EuroQoL (Quality of Life)–5 Dimensions Questionnaire (EQ-5D)
• Work Productivity and Activity Impairment Questionnaire: General Health (WPAI:GH)

Findings

Measuring Visual Acuity

Snellen Eye Chart: The Snellen eye chart is a commonly employed, well-recognized test of visual acuity in clinical practice.^14,34 The chart presents a series of letters of decreasing size, with an increasing number of letters on subsequent lines. One or two mistakes per line are allowed and the smallest line that can be read corresponds to visual acuity. The resultant measure of visual acuity is expressed as a Snellen fraction, in which the numerator indicates the distance at which the chart was read, and the denominator the distance at which a person may discern letters of a particular size. A larger denominator indicates worsening vision. For example, a person with 20/100 vision can read letters at 20 feet that a person with 20/20 vision could read at 100 feet. Snellen acuity may also be expressed in metric units. As 20 feet is roughly equivalent to 6 m, 20/20 vision may be expressed 6/6, or 20/100 as 6/30. Snellen fractions may be expressed as decimal acuity where 20/20 is expressed as 1.00 and 20/100 as 0.2. Further, the logarithm of the reciprocal Snellen fraction may be calculated to produce a linear scoring system suitable for statistical analysis; Snellen fractions of 20/20 and 20/100 would correspond to log scores of 0.0 and 0.7, respectively.

A number of limitations of the Snellen charts, especially for clinical research, have been identified.^14,34 Specifically, this includes the use of letters with different difficulty scores (e.g., A and L are more easily discernible than B E, and F) and an unequal number of letters on each line, which allows for different percentage errors depending on the line read and number of errors made.³⁴ In addition, the change in letter size between chart lines is not uniform; thus, moving from line 20/25 to 20/20 represents a 20% improvement, compared with a 16% improvement when moving from line 20/30 to 20/25. Finally, differences in background luminance between charts, due to aging or to different manufacturers, and the use of dusty or aging projector equipment can reduce contrast and may result in unreliable measures of visual acuity.³⁴

Early Treatment Diabetic Retinopathy Study Letters: In response to the above limitations, alternative charts have been developed that are more appropriate in research.^14,34 The ETDRS charts are based on a design by Bailey and Lovie, and are commonly used in clinical research.^14,35–38 ETDRS charts present a series of five letters of equal difficulty on each row, with standardized spacing between letters and rows, for a total of 14 lines (70 letters). An ETDRS letter score can be calculated as follows: when 20 or more letters are read correctly at 4.0 m, the visual acuity letter score is equal to the total number of letters read correctly at 4.0 m plus 30. If fewer than 20 letters are read correctly at 4.0 m, the visual acuity letter score is equal to the total number of letters read correctly at 4.0 m (number recorded on line 1.0) plus the total number of letters read correctly at 1.0 m in the first six lines. Therefore, the ETDRS letter score could result in a maximum of 100.^25,39 Charts are used in a standard light box with a background illumination of approximately 150 cd/m². Standard chart testing distance is 4 m; however, shorter distances may be used when vision is severely impaired.^14,40 Letters range from 58.18 mm to 2.92 mm in height, corresponding to Snellen visual acuity fractions of 20/200 to 20/10, respectively. Further, letter size increases geometrically and equivalently in every line by a factor of 1.2589 (or 0.1 log unit) moving up the chart. Scoring for EDTRS charts is designed to produce a logarithmic minimal angle of resolution score (logMAR) suitable for statistical analysis in which individual letters score 0.02 log units. Holladay and Prager published the following formula to convert visual acuity scores derived from a Bailey-Lovie-style chart read at 2 m into a Snellen denominator, where X is the number of correctly read letters (see below).⁴¹ Thus, reading all 70 letters on a Bailey-Lovie chart corresponds to a Snellen visual acuity of 20/10.

Minimal Clinically Important Difference: To our knowledge, there has been no derivation of a minimal clinically important difference (MCID) for the ETDRS in myopic choroidal neovascularization (mCNV). Clinical trials assessing interventions for mCNV commonly use a loss or gain of three lines (15 letters), which corresponds to a moderate degree of change or a doubling of visual acuity, as the primary outcome of interest.⁴² For other vision disorders such as macular edema, the FDA recommends a mean change of 15 letters or more on an ETDRS chart, or a statistically significant difference in the proportion of patients with ≥ 15-letter change in visual acuity, as clinically relevant outcomes in trials.¹⁵ The 15-letter reference point is still a topic of discussion for the FDA. A symposium was held by the National Institutes of Health and the FDA to discuss visual acuity measures as outcome measures for clinical trials. In particular, the symposium focused on discussing alternatives to the most commonly used cut point of three line gains or losses on eye charts for classifying outcomes, and discussing the relationship between statistically significant differences and clinically significant differences.⁴³

The test–retest variability (TRV) of the measure can help to guide what would be considered a clinically meaningful change. Literature-based estimates of TRV range from ± 0.07 logMAR to ± 0.19 logMAR.³² This suggests that any change in score between baseline and follow-up of approximately 4 to 10 letters results in insufficient certainty that the difference in letters is not just due to chance alone. When TRV is high, the ability to detect a real change in score is low. For example, for a TRV of ± 0.19 logMAR, the sensitivity of a 0.1 logMAR change (5 letters) was 4% (0% to 14%). If the TRV is lowered to ± 0.11 logMAR, the sensitivity of the test increases to 38% (25% to 53%). If the TRV remains at ± 0.11 logMAR, and the threshold for change increases to a 0.2 logMAR (10 letters) change, the sensitivity of the scale increases to 100% (93% to 100%).

The baseline visual acuity of a sample population will affect the TRV of ETDRS letter scores²⁵ and as a result will also affect what would reasonably be considered an MCID. A TRV of ± 0.11 logMAR has been found in healthy participants,³² while higher levels of variability (± 0.15 logMAR to ± 0.20 logMAR) have been cited for individuals with pathological changes in vision.⁴⁴ Eyes with acuity better than 20/100 and a change in visual acuity of ≥ 5 letters have > 90% probability of being a real change, while for eyes worse than 20/100, a change of ≥ 10 letters is required for the same reliability.¹⁶ A cut point for clinically meaningful change in patients with advanced eye disease should be higher than in healthy individuals, and has been suggested to range between 10 and 15 letters.¹⁷ The studies contributing to this discussion are summarized in Table 34 below.

Table 34

Literature Assessing the Interpretability of Changes in ETDRS Scores.

Relationship of Visual Acuity to Visual Function and Vision-Related Quality of Life

Measures of high-contrast visual distance acuity using ETDRS charts are commonly used to assess treatment outcomes in clinical studies. A loss of ≥ 3 lines (≥ 15 letters) on an ETDRS chart corresponds to a doubling of the visual angle and is considered moderate visual loss, while a loss of ≥ 6 lines (≥ 30 letters) corresponds to a quadrupling of the visual angle and is considered severe. However, visual acuity is only one component contributing to overall visual function — the ability to perform everyday visual tasks (e.g., reading, recognizing faces, driving, and using the telephone). Overall visual function also depends upon variables such as contrast sensitivity, near vision, colour vision, and sensitivity to glare.⁴⁵ The various components of visual function will affect the performance of different vision-related tasks by varying degrees. For example, use of distance acuity to measure the success of treatments for age-related macular degeneration (AMD) is not optimal, given that distance vision is usually two ETDRS lines better than reading vision,⁴² and difficulty with reading is a common complaint among persons with eye disease.³³ Rather, contrast sensitivity is a more important contributor to reading performance.^42,46

Visual function and the resultant ability to perform everyday visual tasks have important implications for quality of life. Quality of life is very much a person-specific measure that ultimately depends on the value individuals place upon the ability to perform specific tasks. Quality-of-life instruments that do not include domains or tasks that are of importance to individuals will lack sensitivity to changes. Further, the impact of vision loss on quality of life may vary greatly, depending on the vision status of the fellow eye. For these reasons, there are limitations in the use of quality-of-life instruments to compare treatment effectiveness.⁴⁵

National Eye Institute Visual Function Questionnaire

The National Eye Institute Visual Function Questionnaire (NEI-VFQ) was developed as a means to measure vision-targeted quality of life. The original 51-item questionnaire was developed based on focus groups consisting of persons with a number of common eye conditions (e.g., age-related cataracts, AMD, and diabetic retinopathy), and thus may be used to assess quality of life in a broad range of eye conditions.³³ The original 51-item questionnaire consists of 12 subscales related to general vision, ocular pain, near vision, distance vision, social functioning, mental health, role functioning, dependency, driving, peripheral vision, colour vision, and expectations for future vision. In addition, the questionnaire includes one general health subscale.¹⁹

A shorter version of the original instrument, the NEI-VFQ-25, was subsequently developed, which retains the multidimensional nature of the original and is more practical and efficient to administer.^18,19 With the exception of expectations for future vision, all constructs listed for the NEI-VFQ were retained in the shortened version, with a reduced number of items within each. Thus, the NEI-VFQ-25 includes 25 items relevant to 11 vision-related constructs, in addition to a single-item general health component. Responses for each item are converted to a 0 to 100 scale, with 0 representing the worst and 100 the best visual functioning. Items within each construct, or subscale, are averaged to create 12 subscale scores, and averaging of the subscale scores produces the overall composite score. Different scoring approaches for the NEI-VFQ-25 have been proposed.⁴⁷ Rasch modelling is used to obtain measurements from categorical data. When comparing standard scoring to Rasch analysis and an algorithm to approximate Rasch scores, all methods were highly correlated.⁴⁷ However, standard scoring is subject to floor and ceiling effects whereby the ability of the least visually able is overestimated and the ability of the most visually able is underestimated.⁴⁷

Determination of what constitutes a clinically meaningful change in the NEI-VFQ appears to be linked to its correlation with visual acuity. A three-line (15-letter) change in visual acuity has been used as the outcome of interest in clinical trials, and corresponding changes in the NEI-VFQ are suggested as clinically meaningful endpoints. Specifically, for the study eye, which is typically the worse seeing eye, a 15-letter change in visual acuity corresponds to a 4-point change in overall NEI-VFQ-25 score.²⁰ For the better seeing eye, the clinically relevant difference for NEI-VFQ-25 scores based on a three-line change is 7 to 8 for overall score. Other studies have shown similar estimated clinically relevant differences.⁴⁸ The instrument showed weaker correlation or was not responsive to changes in the visual acuity of the worse eye.^49,50 This may have implications when evaluating patients with unilateral disease.

Both versions of the NEI-VFQ were reported to be valid and reliable measures of health-related quality of life among patients with a wide range of eye conditions^18,19,50 and all but two subscale scores (general health and ocular pain) have been shown to be responsive to changes in visual acuity in the better seeing eye.^49,50 However, more recent studies have indicated that the NEI-VFQ measures visual functioning, not quality of life.⁵¹ Assessments of the psychometric validity of the NEI-VFQ-25 using Rasch scoring and principal component analysis have identified issues with multidimensionality (measurement of more than one construct) and poor performance of the subscales.^51,52 The NEI-VFQ-25 subscales were found to have too few items and were unable to discriminate among the population under measurement, and thus were not valid.^51,52 Re-engineering the NEI-VFQ into two constructs (visual functioning and socioemotional factors) and removing misfit items (e.g., pain around eyes, general health, and driving in difficult conditions) improved the psychometric validity of the scale in individuals with low vision.^51,52 Considering this recent evidence of multidimensionality, the validity of the single composite score of the NEI-VFQ may be questioned.

EuroQoL (Quality of Life)–5 Dimensions Questionnaire

The EQ-5D is a generic quality-of-life (QoL) instrument that may be applied to a wide range of health conditions and treatments.^21,22 The first of two parts of the EQ-5D is a descriptive system that classifies respondents (aged ≥ 12 years) into one of 243 distinct health states. The descriptive system consists of the following five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each dimension has three possible levels (1, 2, or 3) representing “no problems,” “some problems,” and “extreme problems,” respectively. Respondents are asked to choose the level that reflects their health state for each of the five dimensions. A scoring function can be used to assign a value (EQ-5D index score) to self-reported health states from a set of population-based preference weights.^21,22 The second part is a 20 cm visual analogue scale (EQ-VAS) that has end points labelled 0 and 100, with respective anchors of “worst imaginable health state” and “best imaginable health state.” Respondents are asked to rate their health by drawing a line from an anchor box to the point on the EQ-VAS that best represents their health on that day. Hence, the EQ-5D produces three types of data for each respondent:

• A profile indicating the extent of problems on each of the five dimensions represented by a five-digit descriptor, such as 11121, 33211, etc.
• A population preference-weighted health index score based on the descriptive system
• A self-reported assessment of health status based on the EQ-VAS.

The EQ-5D index score is generated by applying a multi-attribute utility function to the descriptive system. Different utility functions are available that reflect the preferences of specific populations (e.g., US or UK). The lowest possible overall score (corresponding to severe problems on all five attributes) varies depending on the utility function that is applied to the descriptive system (e.g., –0.59 for the UK algorithm and –0.109 for the US algorithm). Scores lower than 0 represent health states that are valued by society as being worse than dead, while scores of 0 and 1.00 are assigned to the health states “dead” and “perfect health,” respectively. Reported MCIDs for this scale have ranged from 0.033 to 0.074.²³

The use of generic preference-based outcome measures to capture change in condition-specific populations including visual impairments was evaluated in a systematic review by Longworth et al. in 2014.⁵³ The EQ-5D was the most commonly used generic quality-of-life measure for vision-related studies. The identified studies included patients with glaucoma, AMD, cataracts, diabetic retinopathy, conjunctivitis, and other eye conditions. The ability of the EQ-5D to distinguish between visual acuity groups varied according to type of visual disorder, as did the construct validity of the EQ-5D with measures of visual acuity.⁵³ The authors found that half of the studies included did not find statistically significant correlations between the EQ-5D and measures of visual acuity, and two of three studies that assessed the responsiveness of the scale found statistically significant differences.⁵³

The responsiveness of the EQ-5D may be of particular concern for patients with low levels of vision. A study assessing patients attending private and hospital-based outpatient clinics with low vision (i.e., 10% < 20/500; 26% 20/200 to 20/500; 34% 20/70 to 20/200; 3-% > 20/70) reported that baseline utilities were highly skewed toward a value of 1.0 (mean = 0.74), while baseline visual ability as measured by the Activity Inventory (AI) was normally distributed with a mean of 0.63.⁵⁴ The EQ-5D was unable to capture changes in visual ability as a result of rehabilitation and following rehabilitation, and the correlation of change scores between the two measures was not statistically significant (Pearson correlation 0.056). Cohen’s effect size was below 0.1 for EQ-5D utility scores and ranged from 0.2 and 0.7 for the domains of the AI.⁵⁴ While the EQ-5D is the most common measure for assessing quality of life in vision-related studies, there are concerns with validity and responsiveness in this population. No published MCIDs could be found for the EQ-5D in mCNV or in other vision-related disorders.

Work Productivity and Activity Impairment Questionnaire: General Health

The WPAI:GH was designed to assess the impact of therapeutic interventions on work productivity and the ability to perform regular activities.²⁴ The questionnaire consists of six questions related to current employment status, the number of hours worked and missed from work over the past seven days, and the extent to which productivity and the ability to do regular daily activities has been affected by health problems over the past seven days. The impact of health problems on productivity and the ability to do regular activities is measured on a scale of 0 to 10, with 0 representing “Health problems had no effect on my work (or activities)” and 10 representing “Health problems completely prevented me from working (or doing my daily activities).” Four main outcome scores can be calculated and expressed as percentages: level of absenteeism from work; level of impairment at work; overall impairment at work; and level of impairment with regular activities.²⁴

The WPAI:GH was initially validated in a group of 106 employed individuals, between 30 and 50 years old.²⁴ The scale has since been adapted for use in various disease areas such as diabetes mellitus⁵⁵ and rheumatoid arthritis,^56,57 and has been translated and validated for use in various languages.^58–60 No studies have been found that use the WPAI:GH as an outcome measure for assessing the effectiveness of therapeutic interventions for vision-related disorders. Therefore, no MCID exists for this measure.

Conclusion

The validity of the ETDRS chart, NEI-VFQ-25, EQ-5D), WPAI:GH questionnaire, and the relationship between visual acuity, visual function, and quality of life were reviewed.

The ETDRS chart is the most widely used outcome measure to assess changes in visual acuity from a therapeutic intervention. It is a modified version of the Snellen Chart and scores are based on the number of letters correctly read by a patient. A loss or gain of three lines (15 letters) is the most commonly used MCID in clinical trials. Given the range of test–retest variability of the scale according to baseline visual acuity, a range of 10 to 15 letters may be a more appropriate MCID.

The NEI-VFQ-25 was developed to measure vision-targeted quality of life. The NEI-VFQ was reported to be a valid and reliable measure of health-related quality of life among patients with a wide range of eye conditions; however, recent studies have suggested that it may be more appropriately identified as a measure of visual functioning. The NEI-VFQ has a reported MCID of 4 points.

The EQ-5D is well validated as a generic, health-related quality-of-life measure. It is commonly used to measure changes in quality of life in the context of vision-related studies; however, its validity and responsiveness in this population is questionable. The psychometric properties of the EQ-5D are known to vary across eye conditions, with no study assessing the validity of the scale in mCNV. An appropriate MCID for use in studies assessing therapeutic interventions for eye disorders is unknown.

The WPAI:GH is a useful measure to assess changes in work productivity and activity impairment in therapeutic intervention studies. It has been adapted for use in many disease areas including diabetes mellitus and rheumatoid arthritis; however, no adaptation has been made for use in studies for vision-related therapeutic interventions. The psychometric properties and MCID for the WPAI:GH in vision-related disorders is unknown.