General Approach

We undertook a broad approach to identifying studies evaluating accuracy of reporting family history. We did not limit studies to those presenting specific diagnostic accuracy metrics and included studies whose primary aim was to ascertain repeatability (variation observed when conditions are kept constant by using the same instrument and individual and repeating within a short time interval).

Studies Reviewed

A total of 20 publications evaluated the accuracy of reporting family history and were eligible for data extraction. One study was based on two publications^{10, 11} leaving a total of 19 unique studies. Study and patient characteristics (such as study design, setting recruited, cancer type, relatives evaluated and criterion standard evaluated) are detailed in Appendix C ^* evidence tables.

We further classified studies by the type of accuracy that was evaluated as follows: 1) those studies (16 studies in 17 publications) which evaluated accuracy of family history reporting by attempting to verify the cancer status of relatives (i.e., accuracy compared with a gold standard), and 2) those (three) which evaluated the repeatability or reliability of the informant's knowledge of family history rather than the true status of the relatives (i.e., no external gold standard).

For the purposes of this review we use the terms “affected” and “unaffected” to refer to those relatives who have had cancer, and those who have not, respectively. We present the results for accuracy according to these groupings, and with regard to specific participant characteristics, type of accuracy evaluated (gold standard or reliability), method of verification, and potential predictors or confounders of accuracy of reporting family history (Figure 3).

Figure

Figure 3. Flow of accuracy studies.

In general we can summarize the accuracy studies as predominantly having recruited participants who had cancer. Within the 19 studies (20 publications), there were three that recruited an entire sample of patients who were free of cancer; two studies involving individuals at high risk for colorectal⁷ or breast cancer,⁸ and one involving women undergoing mammography.⁹ In the four case control studies (five publications),^10–14 the controls were derived from the general population matched for age,^{10, 11} spouses of the informants or regional general practice lists,¹⁴ and from a linkage from license registration and health care administration database.¹³

All studies were classified as case series except four which were case control studies. Several important factors restrict comparisons across accuracy studies, such as the cancer diagnosis of the informants and the cancer information collected about the relatives. There were more studies evaluating informants with breast cancer than other types of cancers; there was a single study evaluating ovarian cancer syndromes within the informants. Some studies probed only specific cancers within relatives while others reported on all cancers within their family histories. While there were only three studies with fewer than 100 informants, the number of relatives reported varied greatly between studies.

Studies Evaluating the Accuracy of Reporting by Verifying no Presence or Absence of Cancer in Relatives. Sixteen studies^{7, 8, 10–17, 19–24} evaluated the accuracy of family history reports by attempting to confirm the true cancer status of the relatives about whom informants provided information. Eight studies ^{13, 14, 19–24} verified the cancer status in relatives reported to be affected and those reported to be unaffected. The other eight studies (nine publications)^{7, 8, 10–12, 15–18} only confirmed the cancer status of relatives reported to be affected. We considered the former studies to be of higher methodological rigor and therefore evaluated these two groups of studies separately.

Studies With Verification in Both Affected and Unaffected Relatives. Table 2 shows the eight studies that verified the cancer status both of relatives reported to be affected and unaffected. Three were case control studies^{13, 14, 19} that recruited participants with colon or colorectal cancer. The remaining five studies evaluated breast cancer patients and a single study evaluated patients with breast, ovarian or colorectal.²⁴ A single study²² evaluated the accuracy of relatives' perception of “awareness of cancer” rather than informants' accuracy in reporting family members with cancer. Three studies^{13, 14, 23} recorded the informant's recollection of any type of cancer in relatives, and the remaining studies examined reporting of relatives' colorectal cancer,^{19, 22} breast cancer,²⁰ breast or ovarian cancer,²¹ or one syndromic group of cancers²⁴ (breast, ovarian or colorectal). In general, family history informant characteristics such as mean age, ethnicity, or education were poorly reported (Table 2). Similarly, characteristics of the relatives were also poorly reported within these studies.

Table 2

Study and patient characteristics of studies evaluating accuracy of reporting and verified in both affected and unaffected relatives.

The methods of family history collection varied with face-to-face interviews in two studies,^{13, 14} mailed survey in four studies,^{19, 21–23} and two with telephone interviews.^{20, 24} The methods of verification of relatives' cancer status varied between studies; also, within some studies different methods were used for checking the status of relatives reported to be affected and those reported to be unaffected. The methods used were: (1) personal interview (reportedly affected) and cancer registry; (reportedly unaffected²³) (2) face-to-face interview, survey, and death registry;²⁴ (3) self report from mail-in survey of relatives;²² (4) relatives' medical chart records and survey; (type not specified)¹⁹ (5) cancer registry alone;^{13, 14, 20} and (6) combined strategy (medical record or cancer registry or death certificate).²¹

Table 3 shows the sensitivities and specificities in studies that evaluated the status of both reportedly affected and reportedly unaffected relatives, where sufficient data were presented to compute these. One study²² was excluded from Table 3 as it evaluated accuracy only in terms of “awareness” of parent or sibling's colorectal cancer. The sensitivity varied by the cancer of interest; for ascertainment of relatives with breast cancer, the range was 85 to 95 percent based on three studies; for colon cancer, 57 to 65 percent (studies using personal interview) and 86 to 90 percent (studies using telephone interview and self report) based on four studies; for ovarian cancer, 67 to 83 percent based on two studies; and for prostate cancer, 69 to 79 percent based on two studies. It is not clear to what extent the verification method of cancer registry versus medical records/death certificates contributed to the ranges observed within a cancer type and between the different cancer types. Similarly, it is difficult to establish how the various methods of collecting family history may have influenced the estimates of sensitivity.

Table 3

Accuracy for studies evaluating patients who report cancer in first degree relatives in studies that verified the status of both affected and unaffected relatives.

In general, specificity across all cancer types and with varying modes of collection was consistently high, (Table 3). For ascertainment of relatives with breast cancer, the specificities were 95 to 98 percent; for colon cancer, 91 to 92 percent; for ovarian cancer, 96 to 99 percent; and for prostate cancer, 93 to 99 percent.

There were three case control studies that therefore allowed for comparison of reporting accuracy between cases and controls. They all involved cases who were patients with colorectal cancer, and controls who did not have cancer. The first study¹⁹ suggested that cases were slightly more accurate than controls (82 percent vs. 76 percent) in reporting history of colorectal cancer in relatives. The second¹⁴ indicated a sensitivity of 57 percent (95 percent CI 43–69) in cases compared with 53 percent (95 percent CI 31–74) in controls in reporting relatives with colorectal cancer. Within this study, the corresponding specificities were 99 percent (95 percent CI 98–99) in both cases and controls. The third study¹³ compared cases and controls with respect to accuracy of reporting several cancer types in their relatives: (1) sensitivity of reporting relatives' breast cancer - cases 85 percent (95 percent CI 55–98), controls 82 percent (CI NR); (2) sensitivity of reporting relatives' colorectal cancer - cases 65 percent (95 percent CI, 38–86), controls 81 percent (CI NR); (3) sensitivity of reporting relatives' ovarian cancer - cases 67 percent (95 percent CI, 9–99), controls 50 percent (CI NR); and (4) sensitivity for reporting relatives' prostate cancer - cases 69 percent (95 percent CI, 41–89), controls 70 percent (CI NR). The corresponding specificities were: 1) relatives' breast cancer status - cases 98 percent, controls 91 percent; 2) relatives' colorectal cancer status - cases 91 percent, controls 94 percent; 3) relatives' ovarian cancer status - cases 96 percent, controls 98 percent; and 4) relatives' prostate cancer status - cases 93 percent, controls 94 percent. Taken together, these data suggest broadly similar specificities across the reporting of cancer types and between cases and controls - i.e., generally, the participants with and without cancer themselves were fairly good at correctly identifying relatives without a history of cancer, irrespective of the specific cancer family history being enquired about. In contrast, the sensitivities were generally lower, meaning that informants appeared to miss some cancers in affected relatives; the highest sensitivities were seen for reporting relatives' history of breast cancer. The results also suggested some differences in sensitivities of reporting between cases and controls - controls being more likely than cases to miss colorectal and ovarian cancers in relatives. In addition, the data from this study would suggest differences in sensitivities such that controls are more accurate for colorectal cancer but less accurate for ovarian cancers. In contrast, the specificities were similar for the cancers evaluated, suggesting no difference between cases and controls with respect to their accuracy in identifying who of their relatives does not have specific cancers. These observations are based on a single study and therefore should be interpreted cautiously.

Studies With Verification in the Affected Relatives Only. Table 4 shows the eight studies (nine publications)^{7, 8, 10–12, 15–18} that verified the cancer status only of relatives reported to be affected by cancer. A single study (two publications) was a case control design^{10, 11} and the remaining were case series. Two studies involved participants who did not have cancer but who were at high risk for breast⁸ or colorectal cancer.⁷ Two studies^{15, 17} involved patients who had prostate cancer, and one study involved colorectal cancer patients;¹⁶ one study combined Li-Fraumeni Syndrome (LFS) and Hereditary Breast-Ovarian Syndrome (HBOCS)¹² (both women at genetic high risk and some with cancer) and one study (two papers)^{10, 11} involved women with breast cancer. A single study involved a range of participants with and without cancer.¹⁸

Table 4

Study and patient characteristics of studies evaluating the accuracy of reporting verified in the affected relatives only.

Five studies^{7, 12, 16–18} assessed the informant's ability to report any cancer within relatives, and the remaining studies appeared to assess reporting of relative's breast cancer ^{8, 10, 11} or prostate cancer¹⁵ history. In general, informant characteristics such as mean age, ethnicity, or education were poorly reported. Similarly, characteristics of the relatives were also poorly reported (Table 4).

The methods of family history collection varied with face-to-face interviews used in three studies (four papers),^{10, 11, 15, 16} telephone interviews in one study,⁷ interview with mode not reported in one study,¹⁷ survey completed in the clinic in one study,⁸ and mailed survey in two studies.^{12, 18} The methods of verification of the relatives actual cancer status included: (1) personal or telephone interview with relatives and medical records,⁸ (2) relatives' medical chart records alone,^{10, 11, 15, 17, 18} and (3) a combined strategy (medical record or cancer registry or death certificate).^{7, 12, 16}

From five studies^{7, 12, 16–18} that reported on the informant's ability to report any cancer within relatives, only two studies provided information on the percent agreement as a function of the cancer reported. One study¹⁸ indicated that breast and colorectal cancers had 93 percent and 89 percent agreement and lower rates of agreement for other cancers (42 percent for extra-colorectal alimentary tract and 37 percent uterine cancer). Another study¹⁷ showed similar results with higher percent agreements for breast, colon, and prostate cancer (95, 92, and 86 percent respectively) in patients with prostate cancer. One study ¹² who evaluated subjects with LFS and HBOCS found differences in the accuracy of reporting, with 85 percent agreement and 92 percent agreement with the reported cancers within their relatives.

Two studies reported on the accuracy of breast cancer within relatives and the percent agreement varied from 89 percent in one study⁸ (with greater accuracy in living relatives with unilateral disease 94 percent) to a sensitivity of 90 percent (CI 95 percent 81–96) in a second study.^{10, 11} The specificity for this latter study^{10, 11} was estimated at 3 percent suggesting errors in reporting of unaffected relatives. One study¹⁵ reported 90 percent agreement for relatives with prostate cancer. Another study¹⁶ reported on the accuracy of colorectal cancer in relatives, with a sensitivity of 61 percent (CI 95 percent 36 – 83) and a specificity of 96 percent (CI 95 percent 88–99). Although, the magnitude of the agreements are generally high for reporting on some cancers, caution should be used when interpreting the results from studies that evaluate accuracy by confirming the status of the affected relatives only, as these contain errors and bias.

Other Factors That May Affect Reporting Accuracy. A variety of factors which could potentially influence accuracy of family history reporting were considered in some studies. Table 5 shows the factors that have been evaluated within some of these studies and, indirectly, the degree of evidence for each of these. We examined 15 characteristics, although some were only evaluated in a small number of studies. Those characteristics infrequently evaluated were: (1) type of first degree relative (1DR), (2) vital status of the relative, (3) number of relatives, (4) cancer history of interest, (5) cancer type of the informant, (6) race of the informant, (7) marital status, (8) laterality within breast cancer, (9) population versus clinic setting recruitment, (10) health insurance status, and (11) gender or age of diagnosis of the relative. It is difficult to generalize for these factors from this heterogeneous series of studies evaluating informants with different cancers and reporting on different cancers within their relatives. Moreover, some of the studies did not actually statistically evaluate differences between the factors of interest; thus, these findings should be regarded as indicating attributes that could be further evaluated in the future research.

Table 5

Factors that can influence accuracy of reporting cancer family history.

Eight studies (nine publications)^{8, 10, 11, 13–15, 18, 19, 24} evaluated the effect of age of the informant on accuracy; no clear trend was observed, and it was not possible to separate any effect of informant age from the possible effects of their own cancer type, gender, or differences in how age was categorized.

Six studies^{7, 13, 14, 18, 19, 24} evaluated the effect of the informant's gender on accuracy, and suggested no general effect. One study¹³ suggested that women might be more accurate in correctly identifying relatives who had ovarian cancer. Another⁷ suggested that there were gender differences in knowledge of paternal versus maternal family history. A third²⁴ suggested that men may over-report cancers compared to women.

Six studies^{12, 14, 15, 18, 23, 24} evaluated whether accuracy varied with the degree of relative whose status was being reported; there was a consistent trend towards increased accuracy of reporting for 1DRs compared to second degree relatives (2DR) or third degree relatives (3DRs) (Table 5). Several studies^{14, 23} noted challenges in confirming the true status of 2DRs and also that fewer 2DR and 3DRs were identified overall, suggesting the potential for reporting and confirmation biases.

Five studies (six publications)^{10–13, 15, 19} evaluated the effect of education level using a variety of categorizations; all but one study¹² showed an effect on accuracy of reporting.

Quality Assessment of Studies

We evaluated quality of the accuracy studies at several different levels. At one level, we considered that the method by which the cancer status of the relatives was evaluated was of great importance in determining accuracy of reporting. At another level, we applied traditional internal validity criteria for study designs that included a comparison group or were considered diagnostic in their design. Since so few of the studies were of traditional study design with control groups, the majority of standardized assessment scales could therefore only be applied to a subset of papers. If we considered all the studies as “diagnostic” in their design, the QUADAS (a quality assessment scale for diagnostic studies) could be applied to most studies. However, not all 14 criteria (or biases) applied to the “diagnostic test” of “family history collection” were relevant in the context of accuracy of reporting; we selected three criteria from the QUADAS to compare the different studies.

Methodological Issues in the Verification of the Cancer Status of the Relatives. For accuracy of family history reporting, we considered verification of the status of both the affected and unaffected relatives to be of the highest quality. Studies that verified the status of the affected relatives only were considered to be of lesser quality or more susceptible to bias with respect to accuracy of reporting.

A number of difficulties were identified by authors with regards to ascertaining the cancer status of the relatives. The range of estimates of difficulties in obtaining some type of confirmation varied from 31 percent¹⁹ to 9 percent.²¹ Some of the difficulties with verification of cancer status of the relative included: (1) errors in medical records or pathology reports,^{8, 21} (2) death of relative prior to registry formation or other form of record keeping,²¹ (3) relative emigrated to another geographic region, for which medical records were not available to the researchers,^{8, 21} (4) informants provided incorrect address or contact information for hospitals where relatives were treated,⁸ (5) retrieval of death certificate information was impossible due to peculiar national laws affecting access by researchers or it was certain the files had been destroyed,¹⁸ (6) some difficulty obtaining medical records of fathers compared to brothers, mothers, and sisters,¹⁷ (7) reports concerned relatives for a branch of the family not of interest to the genetic investigation,¹⁸ (8) the reported cases were late onset common type tumors in distant relatives not likely of interest in the referral,¹⁸ and (9) informants were not in touch with the relatives concerned, so consent could not be obtained.¹⁸ Some studies found it difficult to obtain medical records of deceased relatives when recruitment of relatives for consent depended upon the informants contact.⁹ There was some suggestion that verification rates were lower among negative relatives¹⁹ as these tended to have less physician visits. Studies undertaken in countries with longstanding national cancer and death registries linked with service provision databases, tended to report very high rates of retrieval (97–98 percent) of verification of diagnoses on relatives.¹⁶

Although there were a variety of possible factors that impeded verification of the cancer status of the relative, not all studies excluded from the analysis those informants or relatives for which there were some difficulties in complete confirmation. Note that many studies did not compare the characteristics of the informants who did not wish to contact relatives for their medical records relative to those that did; similarly, comparisons between those relatives that provided consent to medical records and those that did not were not consistently undertaken.

QUADAS Assessment of Methodological Quality for Diagnostic Studies. We applied the QUADAS to those studies that verified the status within their relatives. The QUADAS, a 14 item quality assessment scale for diagnostic studies, was used to evaluate all studies eligible for accuracy of reporting. From these items, three were considered to be of greatest relevance to identifying potential biases within these studies that considered the collection of family history as the “diagnostic test” of interest and the method of verification as the “reference test”. The first challenge was to assume that the “diagnostic test” was the same method of family history collection, in order to compare ratings across studies; clearly, the tools or methods used to collect family history varied significantly amongst studies. The second assumption, we made was that the reference standards specified within each study were equivalent across studies; that is that cancer registry verification and death certificate verification were equivalent.

Three items from the QUADAS were selected to evaluate spectrum bias, verification bias (both differential and partial), and blinding of those who verified the cancer status of the relatives. If present within the studies, each of these biases will result in overestimation of accuracy.

Spectrum Bias. The first question within the QUADAS asks: Was the spectrum of patients' representative of the patients who will receive the test in practice? Theoretically, being asked to take the “test” of cancer family history collection may be received by any person (with or without cancer) in clinical practice. Thus, it was challenging to define which informants are not “typical” of those likely to be tested in practice.

We would indicate the presence of spectrum bias, when the study population did not reflect the spectrum of informants likely to be seen within the clinical setting. For example, patients recruited due to their high risk for familial cancer syndromes would not reflect the spectrum of patients who would report cancer “family history”, albeit they are an important group to evaluate. Similarly, in those studies with informants with cancer of differing severity or who were differentially assigned to study groups, the likelihood of spectrum bias is evaluated as high. We considered a sufficient spectrum of disease should include participants who reflect a complete range of staging (severity) of their cancer if the informant had cancer when the family history was collected. Additionally we believe that an adequate spectrum should reflect informants that included both genders in those studies that did not affect sex-specific organs, such as ovaries or prostate.

When considering the eight studies that verified the status of both the affected and unaffected relatives, the potential for spectrum bias was evident. In general, these studies did not report information on the informants with respect to the severity of disease. One case control study¹³ specified that the cases were “first primary cases” while the others of the same study design did not specify; however, there is still potential for spectrum bias in these studies. One of the studies evaluating breast cancer informants included women of restricted age (< 40 yrs), one third of subjects with bilateral breast cancer, referred to university hospital oncology centre.²¹ Another²³ included informants that were English speaking, North American born, without brain metastases and had a least one 1DR with breast cancer. Both these studies, although they reflect patients likely to be seen in cancer clinics, do not represent the spectrum of breast cancer patients and therefore these studies have spectrum bias.

When considering those studies that evaluated the status of the affected relatives alone, the potential for spectrum bias was also evident. Two studies^{7, 8} recruited cancer free informants who were at very high risk for familial cancers due to a history of 1DRs already diagnosed with the cancer of interest. For the remaining studies, the severity of cancer within the informants was not detailed. This suggests the potential for spectrum bias.

Verification Bias. The fifth question within the QUADAS asks: Did the whole sample or a random selection of the sample, receive verification using a reference standard? Partial verification bias occurs when not all members of the study group receive confirmation of the diagnosis by the reference standard. Similarly, differential verification bias can occur if a subgroup of patients is given a different reference standard test. Partial verification bias can occur if some of the relatives identified by the informant did not have their cancer status verified. Even in studies where both affected and unaffected relatives were evaluated, we did observe that some studies were not able to verify the status of some of the relatives for many of the reasons stated above. One study,¹⁹ (which employed very rigorous ascertainment methods of reportedly affected relatives, even sending notes to hospitals overseas for determining the status of deceased relatives), indicated that they did not attempt to check the medical record of all relatives who were cancer free (the overwhelming majority). Other studies^{7, 13, 19, 20, 22} limited their evaluation or reporting to 1DR only; this in itself may reflect a type of differential verification bias in that not all relatives reported by the informants were verified. In those studies that evaluated only the affected relatives, clearly partial verification bias was present. The presence of partial or differential biases may lead to overestimation of accuracy.¹⁰⁶

Blinding of Those Verifying Cancer Status in Relatives to the Status of the Informant. The eleventh question of the QUADAS states: Were the reference standard results interpreted without knowledge of the results of the index test? In the context of family history collection, our interest was in having those who verified the status of the relatives blinded to the cancer status of the relative and possibly the informant. It is possible that the research assistant extracting the cancer status of the relative, having knowledge of their cancer status, might interpret information (for example, from medical charts) differently than if they were not aware of the cancer status of the relative. Problems with lack of blinding may be less likely to occur in studies that use linkages with cancer or hospital registries; presumably the criteria for verification are not dependent on interpretation by a research assistant. However, there are errors associated with linking databases.

Of the eight studies that evaluated the status of both affected and unaffected relatives, three^{13, 14, 20} relied solely on linkages with cancer or population health registries, and one⁷ on patient report or health records alone; the remaining four studies used a combination of interview, health records and death registries. For those studies that evaluated the affected relatives alone, a single study¹⁸ used computerized linkage alone with patient records to ascertain the status of the relative. Overall, blinding of the status of the relative or the informant was not undertaken in the majority of studies.

Methodological Quality Assessment for Case Control Studies. We applied traditional internal validity criteria to the four case control studies (five publications),^{10, 11, 13, 14, 19} using the Down's and Black standardized quality assessment scale.¹⁰⁵ One study¹⁹ originated as a case control study but undertook a sample from the original to perform a validation study on accuracy of reporting; informants were selected on the basis of having relatives with cancer rather than their cancer status. We did not evaluate the quality of this study using the Down's and Black scale. The range of composite quality scores varied between 14 and 17 (from a possible score of 23), indicating a moderate level of quality for the three case control studies. One of the main methodological flaws was the omission of descriptions of the distribution of principal confounders in two of the studies (three publications).^{10, 11, 13} In addition, only one study¹³ enrolled subjects who appeared to be representative of the general population from which they were recruited and only one study (two publications)^{10, 11} indicated that cases and controls were recruited over the same time period. It was impossible to tell, based on the information contained in the studies, whether cases and controls were recruited from the same source population. There was insufficient information in all four studies to assess blinding, but all studies had reports of losses to follow up. The authors of one study¹² adjusted for potential confounders in the analysis.

The potential for selection or information bias in these four case control studies is difficult to assess. The lack of reporting on recruitment and blinding does not necessarily mean that the authors ignored these issues. It is possible that all subjects were recruited from the same source population and all subjects and investigators were blinded. The authors may simply not have reported this information in the published manuscripts.

Studies Reviewed

A total of 39 different tools, implemented in 40 unique studies, and reported in 45 publications passed full text criteria. Our initial focus was on identifying studies that described FHxTs developed or used in a primary care setting; however, after careful review, we noted that many studies described tools used in other settings that appeared potentially relevant to primary care (criteria for “primary care applicability” is outlined in Chapter 2). We also sent email queries to all authors of eligible studies that did not provide sufficient detail of the FHxT or a copy of the tool. Fifteen authors (of 16 publications) ^{8, 10, 11, 16, 17, 21, 23, 25–33} did not respond in time for the publication of this review and therefore we were unable to determine whether the reported FHxT was applicable for use within primary care. For those studies for which we evaluated the FHxT, six tools from seven publications^{13, 18–20, 24, 34, 35} were assessed as inappropriate for primary care; all of these had been developed and used in research settings. The scoring system and scoring of actual FHxTs is displayed in Appendix B.^* Of the remaining 22 publications, four ^36–39 described the prototype and final versions of the same FHxT (RAGS/GRAIDS), which we counted as a single tool; and two^{40, 41} were companion publications. Thus, 18 distinct tools, from 22 publications, were identified as being applicable to primary care settings (Figure 4). Full study details are summarized in the evidence table (Appendix C,^* Table 2).

Figure

Figure 4. Flow of accuracy studies.

Description of Tools

Target User. Fourteen tools^42–55 were designed for completion by patients, and four tools (eight publications)^{36–41, 56, 57} were designed for use by health professionals.

Format. Eleven tools^{43, 45–49, 51–55} were paper-based, generally in some form of questionnaire or structured questions. Four tools (eight publications)^{36–41, 44, 50} were presented in a form for use on a desktop or laptop computer, including web-based and touch screen applications, and one on a personal digital assistant.⁵⁷ One tool⁴² was an automatic telephone interview, and one was a structured interview schedule.⁵⁶

Cancer Type. Fifteen tools, reported in nineteen articles,^{36–43, 45–50, 52, 53, 55–57} were designed to collect data on family history of breast or breast/ovarian cancer. Nine tools (ten publications) ^{40–42, 46–50, 52, 57} captured data on colorectal cancer and two^{40, 41} tools (three publications)^40–42 on prostate cancer. Five tools (six papers)^{36, 37, 42, 47, 48, 57} also captured data on one or more additional cancer types. For two,^{51, 54} the tool appeared to invite information on any cancer type.

Clinical Setting. Four tools (seven publications)^{36–39, 48, 49, 56} described tools which were implemented in family practice settings, and four tools^{46, 52, 54, 57} in internal medicine clinics. One tool⁴⁷ was implemented in a gastrointestinal clinic, and another⁴⁵ in a screening mammography setting. Three tools^{46, 54, 55} were designed for use in cancer centers or clinics and three^42–44 were implemented in genetic clinics. One tool (two publications)^{40, 41} was web-based and designed for use by any health professional, and the remaining tool⁵³ was used in a large population-based research study. The published reports indicated that eight of the tools were used in a proactive way,^{46, 48, 49, 51, 52, 54, 55, 57} eight (12 papers) in a reactive manner,^{36, 38–41, 43–45, 47, 53, 56} and two in a mixed approach.^{42, 50}

Links to Risk Assessment Tools. The output of five tools (nine publications)^{36–41, 44, 45, 57} was linked directly to some form of defined risk assessment tool (RAT) (i.e., the family history data were converted directly into a risk categorization), although several of the publications describing other tools also described companion RATs.

Content of FHxTs. Fourteen tools^{36–39, 42–45, 47–52, 54–56} reported in seventeen publications, were designed to capture data on all, or selected, 1DRs. Eleven tools (fourteen papers)^{36–39, 42, 44, 45, 47, 49, 50, 52, 54–56} were designed to capture data on all or some 2DRs, and one⁴⁹ on grandparents only. Five tools^{42, 44, 45, 47, 50} explicitly went beyond 2DRs, although not necessarily to capture all 3DRs. For the remaining tools, the extent of family history enquiry was not explicitly described. For all tools except five^{48, 51, 53, 55, 57} there were explicit instructions for users to capture data on relatives on both sides of the family. Two tools^{49, 54} were designed to explicitly capture ethnicity data. Further details of the data captured are presented in Summary Table 7.

Table 6

Study and patient characteristics of studies evaluating reliability.

Table 7

Characteristics of family history tools.

Other Family History Tools. Eleven web-based FHxTs were also identified during the grey literature search. Nine tools were actually available from the web, and these are listed with relevance scores in Appendix B.^* For all except one, (JamesLink)⁵⁰ which was included in the main review, no information was provided on their development or evaluation, which precluded their inclusion in the main review. The highest scoring of these tools for applicability to primary care were the Family History Tool developed by American Academy of Family Practice¹⁰⁷ and the U.S. Surgeon General's Family History Initiative.¹⁰⁸

Evaluating the Family History Tools

The tools were evaluated using a range of study designs. In order to avoid ambiguity in terminology, we drew a distinction between the concepts of “comparator” and “control” (or “controlled”). In keeping with the methods described in Chapter 2, we use the term “comparator” to refer to the use of a reference method to assess the extent, nature and/or accuracy of the family history data captured by the tool in question, the comparators being either “ideal”, best estimate interview, or current (“standard”) practice. We use the term “controlled” to indicate a study design where there are at least two arms, one of which is the tool in question and the other an alternative method of capturing family history data. Thus, in a controlled design, participants are assigned (randomly or otherwise) to either the “tool” group or the control group. We considered crossover studies, where the order of data capture (tool or comparator method) was reversed for some participants, to be controlled studies. Table 8 describes the distribution of studies, in which tools were used, between the four possible categories of study design. We noted that one tool 44 was evaluated in a controlled study, but that no comparator for family history data capture was used, and no outcomes were reported which were relevant to the tool performance as a method of family history data collection (although outcomes relevant to performance as a RAT are presented under Question 3).

Table 8

Classification of study types.

Using this approach, for the purposes of this review, we considered those studies which were uncontrolled studies with no comparator as descriptive, and those which either had a comparator or were controlled to be evaluative, so long as outcomes were reported which were directly relevant to the use of the tool as a method of capturing family history data.

Validity and Reliability

Six tools (nine publications) were described as having undergone a development or piloting phase^{36–39, 42, 45, 48, 49, 51} including one tool (two publications) (Risk Assessment in Genetics, RAGS)^{38, 39} which was the prototype for the Genetic Risk Assessment and Decision Support (GRAIDS) tool,^{36, 37} and a self-completion tool which was developed from a previously validated interview schedule.⁵¹ Five studies assessed acceptability and ease of completion of the tool.^{36, 37, 42–44} Qualitative techniques were also described in studies of four tools, including semi-structured interviews with practitioners^{38, 39} and patients,⁴⁹ and focus groups with practitioners.^{40, 41, 49} Three studies,^{42, 44, 45} reported how long it took to complete the tool, ranging from 8 to 30 minutes. One study⁴² reported test-retest reliability of 97 percent for 1DR, and 93 percent for 2DR respectively, and 98 percent for cancers identified.

Six tools were presented in seven descriptive papers,^{40, 41, 48, 53–56} without a comparator group or control arm. One study of a family history tool embedded in a RAT⁴⁴ presented no outcome data pertaining specifically to performance in capturing family history data.

The performance of the 11 remaining tools was assessed in some way against a defined comparator. For five tools,^{42, 43, 45, 49, 51} this was a genetics interview. For one tool,⁵¹ the self-completion questionnaire was assessed against the parent interview schedule administered by non-genetics investigators. Six tools (eight publications)^{36–39, 47, 50, 52, 57} were compared with current practice in some form. This included the family history as recorded in patient charts, and accuracy or completeness of pedigrees derived from simulated patient histories drawn without access to a tool.

Outcomes

Evaluated Against Genetics Interview. Acheson and colleagues⁴² described an automated telephone interview tool which was evaluated in a sample of genetics patients. Pedigrees obtained by the tool were blindly compared with those obtained from their clinic interview with a genetic counselor. There was an overlap between the data captured by the tool and the interview. The tool was statistically significantly better than genetics interview at identifying 2DRs and first cousins, and identified more cancers in 2DR and distant relatives. When the risk stratification based on the tool and interview pedigrees was compared, there was good agreement (kappa=0.70) for the breast cancer risk assessment, and moderate agreement for colorectal cancers and all cancers combined. Three families classified as high risk by the tool would be classified low risk on the basis of the interview, and one family classified as low risk by the tool would be classified high risk by the interview pedigree. The tool showed high test-retest reliability.

Qureshi and colleagues⁴⁹ described a paper-based, self-completion family history questionnaire, which was compared with a genetics interview conducted by trained researchers. On the basis of the family history captured, 24 percent of tool histories, and 36 percent of interview pedigrees, suggested possibly elevated disease risk which would warrant further investigation. The interview identified 15 percent more 1DRs, and 51 percent more 2DRs, than the tool. The validity of the risk assessments was not determined by a full genetics assessment, so it is not possible to conclude whether the tool was less sensitive or more specific than the interview comparator.

Benjamin and colleagues⁴³ assessed a standard paper-based, mailed, self-completion family history questionnaire with a clinical genetics interview, as part of a study whose primary aim was to evaluate a companion RAT. Using the interview as the gold standard, the tool had 95 percent sensitivity and 96 percent specificity for family breast cancer risk assessment. On the basis of the tool data alone (before the interview), 51 percent of patients would be assessed as having an elevated risk of familial breast cancer; following the genetics interview, this figure was 62 percent.

Fisher and colleagues⁴⁵ assessed a paper-based, patient-completed family history questionnaire in a study whose primary aim was to assess its embedded risk categorization scheme. The participants were women attending for routine breast screening, and the history obtained by the tool was confirmed by follow up telephone interview by a genetic counselor. The authors report that this was to check that the tool data reflected the women's current knowledge of their family history, not to verify it. Of 45 women classified at population risk by the tool, none were reassigned a higher risk on the basis of the genetics interview. Of 45 women classified at elevated risk, none were reclassified as population risk. Further validation of the risk status of the participants through full genetic assessment was not reported.

Kelly and colleagues⁵¹ describe a paper-based, patient-completed tool which was assessed in a sample of cancer patients. In a study whose primary aim was to explore psychosocial outcomes related to accuracy of family history reporting, they compared the questionnaire with an interview-based version of the same tool, using a randomized crossover trial design. The authors report around 77 percent concordance for reporting relatives' age, 81 percent concordance for reporting of relatives' diagnoses, and 82 percent concordance for reporting of age of diagnosis. There were no discrepant data on whether or not a relative had cancer. The order of completion of tools was not associated with differences in these outcomes.

Evaluated Against Current Practice. Emery and colleagues describe the development of a family history tool and RAT (GRAIDS), the prototype for which was RAGS.^36–39 GRAIDS was evaluated using a pragmatic cluster randomized controlled trial,^{36, 37} but no outcomes relating to performance as a FHxT were specifically reported. However, data were reported from a evaluation of the RAGS prototype,³⁹ in which 36 family physicians used three different methods to draw pedigrees and assess the risk of simulated patients. Pedigrees produced using the RAGS tool were statistically significant and more likely to be accurate than those prepared by a genetics software package (Cyrillic) or by traditional pen and paper methods (median correct pedigrees, 5.0/6 for RAGS, 3.5/6 for Cyrillic, 2.0/6 for pen and paper). Participating physicians also preferred RAGS (75 percent) over the other methods (8 percent preferring Cyrillic and 17 percent preferring pen and paper).

Frezzo and colleagues⁴⁶ compared a paper-based, patient-completed family history questionnaire with a genetics interview in a quasi-randomized parallel group study. Of the 39 internal medicine patients who completed the tool, two were identified at elevated risk of breast/ovarian cancer, three at risk of colorectal cancer, and one at risk of prostate cancer. Review of these patients' charts revealed only one patient at elevated risk, of colorectal cancer. In the group whose risk was assessed by interview, the corresponding figures are five at risk for breast/ovarian, and four at risk of colorectal cancer, on the basis of the interview, compared with two and two, respectively, on the basis of chart audit. No data were presented regarding the outcome of eventual genetic risk assessment, if any, of the participants.

Schroy and colleagues⁵⁷ developed an educational intervention for internal medicine residents and assessed the effect of a software tool designed for use on a personal digital assistant. Patients' family history relevant to colorectal cancer risk was assessed by a structured interview with a research assistant. Patients' charts were then audited to assess whether positive and negative colorectal cancer family histories were correctly documented. Of 33 residents to whom the software was sent, 29 acknowledged receipt, two acknowledged downloading it, and one indicated that they had used it clinically. Residents supplied with the tool were no more likely than control residents to document a positive cancer family history in patients' charts (41 percent versus 48 percent), but they were statistically significantly more likely to document a negative family history (89 percent versus 48 percent). The study had low statistical power to detect small to medium effects, and the residents supplied with the tool also received extra educational intervention compared with controls.

Sweet and colleagues⁵⁰ describe the JamesLink system, which is a touch screen, patient-completed tool for capturing family history data. In a study of 362 ambulatory cancer patients, data for 165 indicated moderate or high risk status when reviewed by a geneticist; of these, 16 percent were consistent with a family cancer syndrome. Of 101 patients in the high risk category on the basis of tool data, the chart records suggested family cancer history for 69; seven of the latter had received a full genetics assessment. It was noted that the charts of only 69 percent of patients using JamesLink had family history information available.

Grover and colleagues⁴⁷ prospectively assessed concordance between family history information captured by a paper-based, patient-completed family history questionnaire and then subsequently (and independently) recorded in their cancer clinic charts. They noted discordance between data recorded by the two methods. For 127 (41 percent) of the cases in which there was discordant data, 37 charts (29 percent) had reported a negative cancer history, or not documented a cancer history, which was captured by the tool. For 69 patients (54 percent), only some cancers captured by the tool were documented in the notes, and in 21 patients (17 percent), the tool and the notes were completely discordant. Charts did not document 32 percent of cancers reported by patients in the tool, and a third of notes missed cancers in 1DRs captured by the tool.

Murff and colleagues⁵² compared a paper-based, self-completion family history questionnaire with the charts of 310 internal medicine patients. They noted that the tool identified more 1DRs and 2DRs with colorectal, breast, or ovarian cancer than the charts and were more likely to capture the age of diagnosis for affected relatives, as well as more likely to identify relatives who were diagnosed before the age of 50. For all cancers together, the age of diagnosis was recorded in the chart for about 62 percent of affected 1DRs compared with 95 percent of those captured in the tool. The corresponding figures for 2DRs were 27 percent and 76 percent, respectively. These differences were highly statistically significant. Out of 48 patients who were identified as being at increased risk, the tool identified 29 who would have been missed by charts alone.

In summary, compared to genetic interviews as a gold standard, many FHxTs performed well. However, the studies reported here are limited because the genetic interviews were not supplemented with confirmation of relatives' reported medical histories. Compared with current practice, generally the family history documented in patient charts, FHxTs appeared to identify more relatives, more relatives with cancer, and more details about these relatives. In some cases, this would lead to reassignment of risk category and altered prevention plans. Again, validation of the “true” status of relatives was not performed.

Quality Assessment of Studies

Quality assessment using standardized checklists was undertaken on seven observational studies, five parallel RCTs, and one study⁵¹ that was a crossover trial in which cancer patients were randomized to the order of either a personal interview or a survey and a second study. The quality scores for the seven observational studies^{10, 11, 13, 34, 46, 48, 53} ranged from 14 to 21, thereby indicating a moderate to high level of quality. Initial reporting of hypotheses, interventions, outcomes, and sample characteristics was transparent and complete. However, the authors of only three of the studies^{34, 46, 53} listed important confounders (two adjusted for confounding in the analysis^{46, 53}) and one author⁵³ reported on blinding. Reporting of subject recruitment was also lacking. Confirmation that subjects were representative of the entire population from which they were drawn was provided in two studies;^{11, 46} recruitment of cases and controls from the same source population was mentioned in three studies.^{19, 48, 53}

The five parallel RCTs scored either a 4^{36, 44, 55} or 5^{39, 57} on the extended Jadad quality scale.¹⁰⁹ Major quality issues centered around a failure to describe randomization,^{44, 55} non-reporting of blinding,^{36, 39, 44, 55, 57} and non-reporting of withdrawals,^{44, 55} or methods used to assess adverse effects.^{36, 39, 57}

The absence of information on issues such as recruitment, randomization, and blinding suggests potentially biased results. Since it is not possible to assess whether the absence of information is linked to poor methods or poor reporting, the actual impact of any biases cannot be ascertained.

Other Methodological Aspects. Few studies described a sample size calculation.^{23, 36, 37, 39, 42, 49} Further, for comparative studies where concealment was necessary in qualitative assessment of the FHxT, only a few studies provided evidence that this had been performed.^{43, 49}

The participants of most studies would have had a better recall of their family history than the general public due to the fact that very few studies used an unselected general population.^{46, 48, 49, 54} Special populations included, for example, respondents with the cancers of interest,^{47, 51} or on a cancer registry,²⁵ and patients seen in specialist clinics.^{42–45, 50} Also, the sequence of FHxT evaluation against comparator may have affected patient recall. The FHxT was given first followed by the best estimate in six studies.^{23, 43–45, 47, 49} In one study, interpretation would have been affected by the paper family history questionnaire and structured “best estimate” interview having identical formats, with both approaches being delivered immediately after each other.⁵¹ Other study designs affecting interpretation included non-randomized allocations^{46, 49, 52} and variable response rate to FHxT. When reported, this varied from 40 percent⁴⁹ to 98 percent.⁴⁷ Non-completion of items accounted for about half the errors in an in-office self-completed FHxT.⁴⁵

General Approach

For the purposes of this review we followed the definition of RAT as described in Chapter 2. Some papers were identified which described tools consistent with this definition but which were not developed for use by PCPs, or were evaluated in settings other than primary care. We included some where we considered them to be “potentially applicable to primary care”, in that they did not appear to require specialist genetics knowledge to be applied as intended.

Studies Reviewed

Sixteen publications, representing ten distinct tools, were included in this section of the review. Full study details are summarized in evidence tables (Appendix C ^*), which include information on the evidence cited in support of risk stratification and/or recommended clinical actions. Table 9 presents a description of the tools, assessed against the defined tool characteristics. All tools fulfilled the criterion of timing of use (designed to be used before the health professional or patient takes the relevant decision).

Table 9

Assessment of risk assessment tool characteristics.

Description of Tools

Cancer Type. Six tools, reported in seven papers,^{43–45, 58–61} were designed to assess risk of breast or breast/ovarian cancer only, four tools (seven papers) were designed to assess risk of breast/ovarian and colorectal cancer,^{31, 36–39, 62, 63} and one tool (two papers) focused on breast/ovarian, colorectal and prostate cancer.^{40, 41} No tool was identified that focused solely on ovarian cancer risk, colorectal cancer risk, or prostate cancer risk.

Clinical Purpose of Tool. All ten tools (16 papers) were designed to, in simple or complex ways, stratify individuals into risk categories, and all had a component which indicated some form of clinical or personal action.

Target User. Three of the tools^{31, 44, 45} were designed for use by patients or the general population, the remainder having been designed for health professionals.

Knowledge Component. Each of the ten tools indicated at least one basis for the knowledge component. These components included: the Claus model;^{36–39, 43, 44} the Gail model;^{31, 40, 41} national recommendations (e.g., French National Agency for Health Evaluation,^{40, 41} the Australian National Breast Cancer Centre,⁴⁵ the U.S. Preventive Services Task Force,⁵⁸ and the Scottish Executive Health Department;^{62, 63} guidelines developed by professional groups (e.g., the UK Cancer Family Study Group^{43, 60, 61} and the American Medical Association;^{31, 58}) and guidelines developed by local groups.^{36, 37, 58, 59} For one tool (four papers),^36–39 it was indicated that it was designed to facilitate the implementation of appropriate knowledge components in general, not any specific guideline or risk calculation program.

Implementation Format. Five of the tools (nine papers)^{36–41, 44, 62, 63} were presented in a computer or web-based format and the other five (six papers)^{43, 45, 58–61} were presented in document-based format (Table 10). The five computer-based tools incorporated some form of family history data capture with risk calculation and guideline-based recommended actions.^{31, 36–41, 44, 62} Of the document-based tools, one was a paper-based form with checklist for each relative and an embedded scoring system,⁵⁹ two were paper questionnaires incorporating suggested actions;^{43, 45} one was a pocket laminated card;⁵⁸ and one was an information pack with a laminated card and other components.^{60, 61}

Table 10

Tools presented in format designed to facilitate implementation.

Applicability to Primary Care. Of the seven tools intended for use by professionals, five were developed explicitly for use by PCPs—either family physicians (four tools, 9 papers)^{36–39, 58, 60–63} or physicians working in ambulatory care settings (one tool, two papers).^{40, 41} Two appeared to have been developed in settings other than primary care, or without involving primary care practitioners, but intended for eventual use in that setting.^{43, 59} One patient tool³¹ was developed in a primary care setting, and the other two ^{44, 45} were considered potentially applicable to use in primary care settings.

Evidence of Effectiveness. Findings related to the development of one distinct tool (RAGS/GRAIDS)^36–39 is presented across a number of publications. In general, we report findings for this as one distinct tool, but, where appropriate, we present (and clearly indicate) separate data relating to the evaluation of the prototype version (RAGS)^{38, 39} and the current version (GRAIDS).^{36, 37} For four tools (nine papers)^{36–39, 44, 60–63} data were presented relating to effectiveness against a defined comparator, in achieving outcomes relevant to supporting decisions by users in practice. One tool³¹ was evaluated in an uncontrolled before-after study.

Data are reported to the evaluation of four tools (seven papers)^{31, 36, 37, 60–63} implemented in routine practice settings, including the GRAIDS tool, and three studies of two tools^{38, 39, 44} where evaluations were conducted under “laboratory-type” conditions, including the RAGS prototype tool.^{38, 39} Table 11 summarizes the key points of these studies, including the range of outcomes measured. The remaining studies were tool development or descriptive studies, or the outcomes presented related to the validity or evidence base underlying the stratification system used rather than practice related outcomes.

Table 11

Summary of evaluative studies.

Quality Assessment of Studies

Standardized quality assessment checklists were employed on the five studies that used randomized trial design. The Jadad scores ranged from 4 to 6.^{36, 39, 44, 60–63} Major problem areas were a failure to report whether the studies were blinded^{39, 44, 60, 62} and a failure to report numbers of withdrawals.^{44, 60, 61}

The potential for bias in these studies appears quite low. Concerns about non-differential misclassification are always relevant when there is no blinding, but it is impossible to say whether subjects and investigators were not blinded or whether the authors of the manuscripts simply omitted mention of blinding in their published articles.

Outcomes

Of the evaluative studies of tools directed towards professionals, one (two papers) (the RAGS prototype) was conducted under “laboratory-type” conditions^{38, 39} and three (five papers) were implemented in routine practice settings,^{36, 60–63} including the GRAIDS tool.^{36, 37} In the first of these, the computer-based RAGS prototype application^{38, 39} was compared with pen and paper risk calculation and a specialist risk calculation software package, Cyrillic. The evaluation showed a statistically significant effect of the tool on clinical management decision making for hypothetical cases presented in vignette form. In the study by Watson and colleagues,^{60, 61} a hereditary breast cancer information pack (presented with or without an active educational co-intervention) was compared with no intervention. An analysis of referral letters subsequently received by the relevant genetics centers and breast clinics indicated a statistically significant trend across the three groups in terms of compliance with referral criteria. In the study by Emery and colleagues,³⁶ a randomized controlled cluster trial was used to evaluate a complex intervention which comprised a web-based decision support system (the GRAIDS software, for which RAGS was the prototype) and a nominated “lead clinician” within the practice who received extra training in use of the software and was expected to manage all patients expressing concerns about family history of colorectal or breast cancer. All physicians and nurses in intervention practices also received a short educational session on cancer genetics and an introduction to the GRAIDS software. The control intervention was a mailed paper copy of the relevant regional guidelines, along with a short educational session on cancer genetics. The intervention arm contained an “adaptive” sub-group, in which extra training or software adjustment was used to increase actual use of the intervention. The primary outcome was appropriateness of referrals made to the regional genetics clinic, as assessed by comparison of each referral letter with the regional guidelines. For both cancer groups combined, 95 percent of referrals made by physicians in the intervention group met the guideline criteria, compared with 79 percent in the control group, a statistically significant result. For breast/ovarian cancer referrals, the proportions were 93 percent and 73 percent, respectively (statistically significant) and for colorectal cancer referrals, the proportions were 99 percent and 92 percent (not statistically significant). Overall, there were no statistically significant differences in proportions of patients who were subsequently assessed as being at increased cancer risk by genetics specialists. At the patient level, cancer worry scores were lower in those referred from intervention practices than from control practices, but no statistically significant differences were observed in knowledge or risk perception scores. The fourth study^{62, 63} compared a stand-alone computer based decision support tool with a control intervention of national guidelines disseminated by mail to family physicians. All practices within the health care administrative region were included in the trial, and all intervention practices received the intervention in some form. The primary outcome was physician confidence in four domains related to assessing risk, making clinical management decisions, and counseling patients, and no statistically significant differences were detected between intervention and control groups for any of the four domains. No statistically significant differences between groups were observed in secondary outcomes related to patients' risk perceptions, beliefs about breast cancer causation, or the risk of referred patients as assessed by genetics specialists.

Of the evaluation of tools directed towards patients, one was conducted under laboratory-type conditions,⁴⁴ and one was evaluated under conditions approaching routine practice.³¹ The former⁴⁴ was an evaluation of the patient oriented “GRACE” tool. It was framed as an equivalence or non-inferiority trial, but was not statistically powered for testing of a priori hypotheses. The comparator was a consultation with a nurse specialist who used the same evidence base to assess risk and offer advice. Outcomes related to patient acceptability, risk perception, anxiety and cancer worry, were all either statistically non-significant, or favored the control arm. In the second study;³¹ the Cancer Risk Intake System (CRIS), a touch screen system for patients, was implemented in three primary care clinics. On the basis of family and other history, patients received tailored printouts including up to three messages regarding cancer prevention, to be used as an aid for discussions with their physician. A before-after evaluation suggested that the proportion of patients reporting a physician discussion about tamoxifen use increased from 4.8 percent at baseline to 27.7 percent after using CRIS; the corresponding pre- and post-figures for cancer genetic counseling were 2.8 percent and 28.2 percent, and for colonoscopy were 16.1 percent and 45.2 percent. The lack of a control intervention makes it difficult to assess the extent to which completing the baseline survey acted as a co-intervention.