Research Needs

The orientation calls to each member of the Stakeholder Panel were conducted between May 14 and 18, 2012. The first teleconference with all the Stakeholders occurred in two sessions: the first with four Stakeholders participating on May 21, 2012, and the second with six Stakeholders participating on May 29, 2012. Two Stakeholders participated in both calls. A total of nine research needs (see Appendix E) were identified through a combination of the CER findings and input from the Stakeholder panel. During the first teleconference, the Stakeholders reviewed the preliminary list of nine research needs, made the suggestion that they be more detailed rather than generic, and also discussed possible future research questions and study designs.

After the first teleconference, the Stakeholders completed the online SurveyMonkey^®, ranking their top three research needs from the list of nine, using the revised EHC program selection criteria previously described. The response rate was 100 percent (n=8). Total scores ranged from 11 to 3, with a single item receiving no votes and a score of 0. The results of the first survey are provided in Appendix F.

A second teleconference was held on June 14, 2012, to discuss the results of the Stakeholders’ prioritization of the research needs and to solicit input on research questions. Again, to accommodate the busy schedules of participants, calls were made at two separate times, with four Stakeholders participating in a morning session and three in an afternoon session (a total of 6 of 8 Stakeholders participated; the remaining two provided followup input in one-on-one calls with the team leader).

Based on an evaluation of scores, there was no single, clear cut-off point on which to distinguish high- from low-priority items. In the teleconference discussion, the Stakeholder Panel used an empirical approach to establish this cut-off, based on the observation that a logical progression for a diagnostic assay like PCA3 was to move from an understanding of the clinical validity of the signal to informed applications of its use in decisionmaking followed by either a chain of evidence or direct evidence suggesting a positive impact on long-term health outcomes.

The top-ranked research needs were as follows:

• Research Need 1: Lack of information on the comparative performance of PCA3 and currently used biomarkers to detect prostate cancer; lack of “matched studies” on comparisons;
• Research Need 8: Lack of studies on how PCA3 actually helps in treatment decisionmaking;
• Research Need 5: Lack of information on the impact of PCA3 on short-term outcomes, such as biopsy decisionmaking; and
• Research need 6: Uncertainty of impact of PCA3 in biopsy decisionmaking on long-term health outcomes.

Based on their empirical approach, the Stakeholders suggested combining Research Needs 5 and 8 into one item addressing decisionmaking and including Research Need 6 as the final item determining the impact of PCA3 testing on long-term health outcomes.

The resulting three top-ranked research needs were as follows:

• Research Need 1: Lack of information on the comparative performance of PCA3 and currently used biomarkers to detect prostate cancer; lack of “matched studies” on comparisons;
• Research Need 2: Lack of studies on how PCA3 actually helps in biopsy or treatment decisionmaking; and
• Research Need 3: Uncertainty of the impact of PCA3 in biopsy decisionmaking on long-term health outcomes.

Research Need 1 addresses the diagnostic accuracy of PCA3 in comparison to other tests for prostate cancer. In the PCA3 CER, the most common comparison observed was between PCA3 and tPSA. A small number of articles described PCA3 compared to percent-free PSA or to nomograms; however, the information was either low in quality or insufficient to draw conclusions about the relative performance of these tests.

Research Need 2 addresses how understanding the aggressiveness of identified or missed tumors might be used in decisionmaking about whether to biopsy or defer and follow the patient over time, or used to determine whether a patient should be offered active surveillance versus aggressive therapy.

Research Need 3 addresses the critical question of whether PCA3 testing has an impact on long-term health outcomes. To the extent that PCA3 testing reduces the need for biopsy or identifies aggressive tumors that warrant treatment, the assay has the potential to increase benefits and reduce harms. The panel recognized that this research need might be difficult to address, but concluded that test accuracy alone would not necessarily equate to test effectiveness.

Research Questions

Research questions for each of the research needs were generated from the PCA3 CER and input from the Stakeholder Panel. Relevant research questions were the subject of discussion at both teleconferences as Stakeholders considered mechanisms for addressing the important research needs identified. Based on a Stakeholder suggestion, one research question was added to the original three from the CER for Research need 1, but none were added to the original two questions for Research Need 2 or the one question for Research Need 3.

Once the final list of research questions was compiled, Stakeholders again completed an online SurveyMonkey^® to prioritize the research questions (see Appendix G). Because of the well-established hierarchal approach provided by the Stakeholders, the prioritization of research questions was modified to rate questions within each research need, rather than to generically evaluate these questions collectively across the research needs.

For Research Need 1, there were a total of four research questions, and each Stakeholder ranked the top three of most importance. For Research Need 2, the Stakeholders ranked the two questions in order of perceived importance. Research Need 3 had only one research question and was, therefore, not included for prioritization. The prioritization results for the research questions for Research Needs 1 and 2 are presented in Appendix H.

For Research Need 1, the three highest priority research questions were as follows:

• Research Question 1.2: What is the comparative effectiveness of PCA3 compared to the two commonly used add-on tests of fPSA and tPSA velocity/doubling time in predicting prostate biopsy results?
• Research Question 1.1: What are PCA3’s diagnostic performance characteristics in patients with elevated tPSA levels?
• Research Question 1.3: What is the comparative effectiveness of PCA3 compared to externally validated nomograms in predicting prostate biopsy results?

For Research Need 2, both research questions were ranked equally, and therefore remained in the following order:

• Research Question 2.1: What information does PCA3 provide about the aggressiveness of prostate cancer? Do positive results correlate to tumors with aggressive features on biopsy or upgrading of tumors on prostatectomy? Do negative results correlate to tumors that may not require identification or aggressive treatment?
• Research Question 2.2: Does the addition of PCA3, either alone, or in combination with other markers, change prostate cancer biopsy or treatment decisionmaking for the patient or physician?

The final three research needs, in priority order, along with the associated prioritized research questions and PICOTS are presented in Table 2.

Table 2

Priority PCA3 research needs with research questions and PICOTS.

Study Design Evaluation

Research Need 1: Lack of information on the comparative performance of PCA3 and currently used biomarkers to detect prostate cancer; lack of “matched studies” on comparators.

• Research Question 1.1: What is the comparative effectiveness of PCA3 compared to the two commonly used add-on tests of fPSA and tPSA velocity/doubling time in predicting prostate biopsy results?
• Research Question 1.2: What are PCA3’s diagnostic performance characteristics in patients with elevated tPSA levels?
• Research Question 1.3: What is the comparative effectiveness of PCA3 compared to externally validated nomograms in predicting prostate biopsy results?

Table 3 provides the study design evaluations for Research Questions 1.1–1.3, which were combined because they require the same basic approach. In each, the objective is to determine the comparative performance of PCA3 against the current standard of care by establishing diagnostic accuracy for each test of interest.

Table 3

Study design evaluations for PCA3 Research Need 1, Research Questions 1.1–1.3.

An ideal study to establish diagnostic accuracy would be to identify a large cohort of men being screened for prostate cancer (most commonly based on elevated PSA and/or abnormal DRE). All positive patients would be tested for analytes of interest: tPSA (repeated to assess regression to the mean), free PSA, PSA velocity, nomograms etc. All men would undergo biopsy to allow for a head to head comparison of PCA3 to currently used tests. Although it is unlikely everyone would be biopsied, as many as possible should be biopsied and reasons for dropout carefully described. It is important to note that this information was uniformly absent from studies of PCA3.

This ideal study, like most studies in the current literature, should be of matched design (comparative biomarkers are measured in the same individuals in the same setting allowing each individual to be a self-control). The data should be subject to matched analyses (e.g., McNemers test), stratified by biopsy-positive and biopsy-negative subgroups. Although all of the studies in this CER were matched, none of them were subject to a matched analysis.

It is important to note that although matched analysis is missing from the current literature on PCA3, most studies were performed using a matched design. It might be possible for individual investigators or groups of investigators to re-analyze existing data to better understand test performance. To the extent current reports have information on biopsy yield and reasons for drop-out, the ideal study may be approximated using existing data

While prospective-retrospective studies can sometimes be used to establish performance of a test using banked samples in patients with well-established demographic and clinical information, this is not possible for PCA3 testing at this time. While there are a number of biobanks of blood from patients studied for prostate cancer, no similar banks of stored urine are known to exist. Case control studies allow for testing in patients with established disease and in controls without disease. While this approach has the advantage that it often facilitates identification of samples for study, this design does not provide testing in the intended-use population, has a high risk of bias, and is generally reserved for exploratory studies only.

Research Need 2: Lack of studies on how PCA3 actually helps in biopsy or treatment decisionmaking.

• Research Question 2.1: What information does PCA3 provide about the aggressiveness of prostate cancer? In other words, do positive results correlate to tumors with aggressive features on biopsy or upgrading of tumors on prostatectomy? Do negative results correlate to tumors that may not require identification or aggressive treatment?

Table 4 provides the study design evaluation for Research Question 2.1 which, like the four research questions for Research Need 1, focuses on establishing the diagnostic accuracy of the PCA3 test. However, this question differs from the first three research questions by having as its endpoint the detection of indolent versus aggressive disease. Ideally, aggressiveness would be carefully defined, either using standard techniques established for biopsy (i.e., the Epstein criteria) or using information obtained by comparing biopsy results with staging results in menundergoing radical prostatectomy. Again a predefined cutoff would be selected, and test performance for predicting indolent or aggressive disease described in terms of diagnostic performance of clinical sensitivity, specificity, and predictive value of positive and negative results.

Table 4

Study design evaluations for PCA3 Research Need 2, Research Question 2.1: What information does PCA3 provide about the aggressiveness of prostate cancer?

• Research Question 2.2: Does the addition of PCA3, either alone or in combination with other markers, change prostate cancer biopsy or treatment decisionmaking for the patient or physician?

Table 5 provides the study design evaluation for Research Question 2.2, which addresses the issue of whether the magnitude of changes in prediction for biopsy outcomes provided by PCA3 are significant enough to cause physicians or patients to make changes in either biopsy or treatment decisions. The goal of the research question would be to determine if the trade-off between reduction in biopsies and missed cancers could be communicated to physicians or patients in a manner that would allow for more informed decisions and improved outcomes.

Table 5

Study design evaluations for PCA3 Research Need 1, Research Question 2.2: Does the addition of PCA3, either alone or in combination with other markers, change prostate cancer biopsy or treatment decisionmaking for the patient or physician?

While an optimal manner of looking at this issue would be by performing a randomized clinical trial looking at management decisions with and without use of PCA3, alternative techniques might be to perform a prospective cohort study or to use surveys or conjoint analysis to approximate decisionmaking in the face of varying test performance. Surveys or conjoint analyses are predicated on having good data about the comparative performance of the prognostic tools currently available, and would only be as reliable as performance estimates derived from data provided to answer research questions 1.1–1.3 and 2.1.

Research Need 3: Uncertainty of impact of PCA3 in biopsy decisionmaking on long-term health outcomes.

Research Question 3.1: Does addition of PCA3 testing change long-term health outcomes in prostate screening?

Table 6 provides the study design evaluations for Research Question 3.1, which focuses on the bottom line: Does PCA3 testing change long-term health outcomes? The gold standard for making this determination would be to perform a randomized controlled clinical trial, comparing management of patients with and without PCA3 and following a large number of patients over long periods of time to determine actual impact of testing on morbidity, mortality, and quality of life. An alternative would be a prospective cohort study. Stakeholders recognized the challenges such studies would present; however, as one Stakeholder noted, accuracy of testing does not assure effectiveness of testing.

Table 6

Study design evaluations for PCA3 Research Need 3, Research Question 3.1: Does addition of PCA3 testing change long-term health outcomes in prostate screening?

A possible alternative or interim process for establishing the merit of PCA3 testing as an intervention might be a careful evaluation of the prognostic impact of the test, followed by an effort to establish a chain of evidence to support testing. There was vigorous discussion of formal modeling as a mechanism to replace the need for more formal short- or long-term trials, but concerns were raised about the assumptions that would need to be made. Missed tumors should exhibit the same or more indolent behavior than those identified by testing; the ability of testing to reduce biopsies or to detect clinically significant tumors should not be a result of confounding by demographic characteristics in patient subgroups (age, ethnic background, family history, etc.). Shorter-term studies similar to the REDUCE trial have already been used to study PCA3 performance,^18,19 but could be replicated or expanded to better understand how the test impacts outcomes. The suggestion was also made to piggyback PCA3 testing on current trials studying active surveillance, similar to the recent report by Tosoian,⁵ but with more detail on outcomes (prostatectomy results) and longer term followup.