U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of How Do Preferences for Treatment Change After Patients With Lung Cancer Start Chemotherapy?

How Do Preferences for Treatment Change After Patients With Lung Cancer Start Chemotherapy?

, MD, PhD, , MD, , , MD, , MD, , MD, , MD, , RN, MN, PhD, , and .

Author Information and Affiliations

Structured Abstract

Background:

Lung cancer in the United States accounts for 14% of cancer diagnoses and 28% of cancer deaths annually. Because there is no cure for advanced-stage lung cancer, the main treatment goal is to prolong survival. Chemotherapy regimens produce different side effects. Identifying individual patients' preferences related to side effects could result in patient-centered choices, leading to better treatment outcomes. We found no previous studies or tools for assessing and utilizing patient chemotherapy preferences in clinical settings.

Objectives:

Study aims were to determine (1) patients' definition of treatment success and whether individual preferences, characteristics, and treatment experiences affect their definition of treatment success; (2) how preferences among adverse effects relate to choice of chemotherapy regimen; and (3) whether oncologists are likely to change their chemotherapy treatment strategy when provided with information related to patient preferences.

Methods:

To address aims 1 and 2 of the study, we conducted a quantitative study utilizing an observational longitudinal open cohort of patients with advanced-stage non–small cell lung cancer (NSCLC). Data sources included the patient's medical record and patient interviews before, during, and after chemotherapy. The interviewer asked about the patient's definition of treatment success and preferences for addressing chemotherapy side effects. We used univariate and multivariate regression methods to analyze the findings. To address exploratory aim 3, we conducted a pre–post randomized intervention study with oncologists from the participating cancer centers. All participating oncologists, regardless of study arm assignment, were provided with a case scenario patient who reported least-desired side effects that were common to all lung cancer chemotherapy drugs. Following this, oncologists were randomized to an intervention arm (in which the least-desired side effects occurred only with 1 lung cancer chemotherapy drug) or to a control arm (in which the least-desired side effects were common to all lung cancer chemotherapy drugs).

Results

  • Aim 1: Most (156 of 235 [83%]) of the cohort at the baseline interview defined treatment success as good quality of life and reaching an important personal goal with or without survival, and only 12% considered “survival alone” as treatment success (N = 235). Overall, 47% of patients changed their definition of treatment success as they moved through chemotherapy. A bivariate analysis showed patients whose annual income was less than $45 000 were twice as likely to change their treatment success definition compared with patients whose reported income was $45 000 or more (odds ratio, 2.33; 95% CI, 1.11-4.9; P = .0245).
  • Aim 2: At first interview, shortness of breathing (29%), bleeding (21%), fatigue (12%), and dizziness (11%) were the most commonly reported chemotherapy side effects that patients most hoped to avoid.
  • Aim 3: All oncologists (13 of 13) who were randomized to a case scenario patient whose least-desired side effects occurred with only 1 drug chose a different drug than in the first round, in which the case scenario patient's least-desired side effects were common to all lung cancer chemotherapy drugs. In the control group, the second case scenario was similar to the first round, and 5 of the 8 oncologists switched treatments. This result suggests that oncologists do pay attention to patients' preferences for side effects of chemotherapy.

Conclusions:

Patients' definitions of treatment success are dynamic, and changed during treatment, making it imperative to ensure effective patient–provider communication throughout the clinical care continuum.

Limitations:

Our study results are limited to patients with advanced NSCLC and drawn from a predominantly White patient population from the US Midwest and Florida. Generalizability is limited to similar populations and to a research setting using similar patient scenarios.

Background

Lung cancer is the leading cause of cancer-related deaths in the United States.1 Nearly a quarter of a million new cases of lung cancer were reported in 2016 (n = 224 390), accounting for about 14% of cancer diagnoses and 28% of all cancer deaths expected from lung cancer.2 In comparison with other cancers in the United States, lung cancer is a major source of health care costs and significant health care services utilization, as the average age at diagnosis is 70 years.3,4 Of patients diagnosed with non–small cell lung cancer (NSCLC), the most common type of lung cancer, 40% have advanced-stage disease. Most patients with lung cancer die within 1 year of being diagnosed.2

The treatment options for NSCLC are based mainly on the stage (extent) of the cancer, but other factors, such as a person's overall health and lung function, as well as certain traits of the cancer itself, are also important. The overall treatment goals for NSCLC are to prolong survival and control disease-related symptoms.5 Improvements in survival are similar regardless of the combination of chemotherapy used. However, many different toxicity profiles exist.6 As noted by the National Cancer Institute, toxicity profiles play an important role in determining treatment choices and treatment success.7

Literature Gap

After conducting a thorough literature review, we found no large-scale systematic reviews and no studies concerning our specific area of interest: assessing patient preferences in direct relationship to individualized chemotherapy drug treatment choices at the time of clinical treatment planning or for monitoring patient preference–based tolerance of chemotherapy for advanced-stage lung cancer. We found 1 cross-sectional study of patients' preferences of medical side effects, which utilized patient surveys sent by mail at variable time points during or sometime after their treatment, which potentially induced recall bias.7 The study also had low participation (31%) and high missing data (up to 16%) for some variables and did not link preferences of side effects to real-life drug choices. Additionally, the study did not evaluate changes in preferences before and after real-life experiences of side effects, did not collect clinical information, and did not evaluate potential changes in oncological clinical practice based on patients' preferences.

As treatments for NSCLC typically have different side effects, it is crucial for physicians to understand patients' preferences of treatment and side effects. However, to date, no clinical guide on how to integrate patients' preferences of side effects in treatment decisions has been published, although it is well known that most patients with cancer prefer either an active or shared role in decision-making.8,9

Moreover, most providers lack the tools, time, and resources needed to efficiently and effectively consider such patient-centered treatment plans with their patients.10 Furthermore, the Institute of Medicine identified poor patient–provider communication as a key barrier to achieving patient-centered care planning in real-life situations.10

Active patient participation in treatment decisions has been suggested to benefit patients both in the short and long term.11 Such participation improves patient satisfaction and quality of life (QoL), which leads to better outcomes.10 Furthermore, ethical and social justice calls for patients' active participation in treatment planning. One study among patients with breast cancer indicated that patients' participation at the multidiscipline cancer care team meeting was acceptable to both patients and health professionals.12 However, the study findings were limited as they examined only the acceptability and feasibility of patients' involvement in the treatment plan document. Therefore, our research addressed a critical gap in current knowledge and practice.

Impact

This study is important for patients and future health care performance in several ways. First, health care providers may benefit from new information about patient treatment preferences when offered choices about expected side effects of treatment that do not change the predicted outcomes of treatment. Second, the Affordable Care Act, the American College of Surgeons Commission on Cancer Patient-Centered Standards, and other programs encourage providers to adopt individualized care for better outcomes. This research provides valuable information and suggestions for physicians who interact with and engage patients in carrying out the treatment plan. Information gained from this study may also be used as guidance to develop facilitated treatment choices in patients with future advanced-stage lung cancer. Ultimately, if patients' drug preferences are more aligned with real-life treatment, patient satisfaction and compliance may increase.13 The results of this study are expected to empower patients as active participants in their cancer care, potentially improve QoL, and decrease caregiver burden during treatment. Additionally, implementation of the clinical research forms (CRFs) that we developed to collect patients' preferences for incorporation into the treatment plan will improve the delivery of cancer care for patients.

Patients can successfully play active, engaged roles in research ranging from participant to partner. Traditionally, treatment success for advanced-stage NSCLC has been measured by researchers and clinicians in terms of survival. The essence of our study was to determine if individual patient preferences, characteristics, and treatment experiences affect the meaning of treatment success among patients with advanced-stage NSCLC. We also examined the willingness of physicians to incorporate the patients' definitions of treatment success and preferences in the choice of treatment options, given that survival is similar for all combination chemotherapies for advanced-stage lung cancer.

Our goal was to emphasize the importance of patients' treatment choices for advanced-stage lung cancer by utilizing patients' preferences of drug options after real-life experiences of treatment effects. To achieve the study goals, we focused on the patients' definition of treatment success and on how clinicians can better understand and address cancer treatment from the patients' perspective when given patient characteristics, conditions, and preferences.

Patient and Stakeholder Participation/Engagement

Utilizing recommendations from relevant research,14 we engaged key stakeholders in a variety of roles related to the conduct and content of the study. We formulated and achieved the balance of perspectives needed using 4 main areas: study design, focus groups, patients, and oncologists. In every study phase, we selected the methods, modes, and intensity of engagement of stakeholders so that each phase built on the previous phase. The principal investigator (PI) and lead co-investigator, an oncologist specializing in lung cancer, conceived the study. Also, the lead co-investigator assisted the core group advisory committee chair and provided connections to clinical peers, patients, and other stakeholders. The PI also identified and recruited a spousal caregiver, the widow of a patient with lung cancer, who served as the lead patient advocate (PA) and a study co-investigator. She participated in both study design and implementation and recruited patients, family members, and advocacy stakeholders for the study. The PI, lead co-investigator, and/or PA completed identification and recruitment of stakeholders primarily in the design phase, and clinicians at participating clinical sites subsequently completed these steps in the focus group, patient, and oncologist phase.

Stages of the Study

Study Development Stage

Thirty-six stakeholders, including 7 patients, 6 family members, 3 PAs, and 20 oncologists and nurses, participated during the design period. We collaborated with these stakeholders through focus group sessions to help us (1) decide what to study, (2) design the study, and (3) choose study outcomes.

For example, while we initially considered collecting data electronically, possibly with an iPad, a focus group of patients during the development stage (focus group methods and results are published and attached in Appendix G) strongly opposed both electronic and hard-copy self-administered data collection. Patients favored having an in-person interview in the clinic as the primary option. Patients agreed that other ways of completing the instruments could be offered; however, they thought that most participants would prefer a personal interview. They also preferred that much of the information asked in the proposed questionnaire be extracted from the patient's medical record and that the questionnaire be shortened.

Also, in a focus group, family caregivers suggested promoting rapid adoption of research evidence into practice by recruiting additional cancer centers. Utilization of this suggestion more than doubled the number of participating clinics without any negative impact on budget and with a positive impact on the number of patients enrolled.

Study Implementation Stage

In addition, patients (not including those involved in the study development stage) and oncologists were involved as research study participants in the focus group sessions. They provided input on CRF development, patient recruitment, retention of study participants for the next steps, and dissemination of study findings. Our research focused on patient-reported outcomes of 235 patients with advanced-stage lung cancer. Enrolled patients participated in up to 3 personal interviews/questionnaires (at baseline, which is before chemotherapy; after the first cycle of chemotherapy; and after the completion of the first planned chemotherapy cycles) about their definition of chemotherapy treatment success and their preferences about which possible side effects of chemotherapy they would most wish to avoid. Next, we engaged oncologists from the participating cancer centers in an experimental study with a case scenario to explore how the communication of patient preferences could be incorporated into their treatment planning.

The perceived positive impact of engagement of patients and other key stakeholders throughout this research included the relevance of the research question; the study design, processes, and outcomes; the study rigor and quality; the transparency of the research process; and the adoption of research evidence into practice.

Specific Aims Summary

The following were the specific aims and research questions:

Aim 1: Determine if Patient Characteristics and Treatment Experiences Affect Patient Definition (Meaning) of Treatment Success

  • Question 1.1: How do patients define treatment success before chemotherapy?
    • – This was a descriptive research question. Results included a description of variations in patients' definitions of treatment success and the frequency of reporting these descriptions.
    • – Additionally, in the development stage of the study, we undertook a qualitative inquiry of the definition of treatment success to support the development of the CRFs. We conducted 4 focus groups with provider, patient, and caregiver participants from 4 cancer centers in Nebraska and South Dakota. A total of 36 providers, patients, and caregivers participated in the focus groups. Each session lasted about 90 minutes and had between 6 and 11 adult participants. We conducted 2 focus groups with patients and their caregivers/advocates and 2 focus groups with health care providers (physicians and nurses). We conducted 2 of the focus groups in an urban cancer center and 2 in a rural facility. Results of this qualitative inquiry, including this specific question, are presented in a published manuscript (Appendix G; also, see Islam KM, Opoku ST, Apenteng BA, et al. Engaging patients and caregivers in patient-centered outcomes research on advanced stage lung cancer: insights from patients, caregivers, and providers. J Cancer Educ. 2014;29(4):796-801. doi:10.1007/s13187-014-0657-3).
  • Question 1.2: Are patients' age, gender, race, education, employment, marital status, income, insurance, and residence (ie, rural vs urban) associated with their definition of treatment success?
    • – The research question was an exploratory analytical inquiry with no prior stated hypothesis. Results included comparisons across socioeconomic and demographic risk factors commonly included in patient records against key themes of reported definitions of treatment success.
  • Question 1.3: Do patient definitions of treatment success change after chemotherapy?
    • – For this research question, we hypothesized that chemotherapy treatment would lead to a change in patient definitions of treatment success. We tested the following null hypothesis: Chemotherapy experiences affect patient definitions of treatment success. Our alternative hypothesis was that chemotherapy experiences do not affect patient definitions of treatment success.
  • Question 1.4: Are changes in definitions of treatment success associated with patient characteristics?
    • – This question was an exploratory analytical inquiry with no prior stated hypothesis. Results included reposting of risk factors of socioeconomic and demographic characteristics commonly included in patient records against the change in patient definitions of treatment success.

Aim 2: Assess Patients' Treatment Choices Based on Their Ranking of Unwanted Drug Side Effects

  • Question 2.1: Are patient characteristics (ie, age, gender, education, marital status, and residence) associated with the length of time patients are willing to tolerate chemotherapy side effects to attain a personal goal?
    • – This question was an exploratory analytical inquiry with no prior stated hypothesis. Results included reposting of characteristics commonly included in patient records against the length of time patients are willing to tolerate chemotherapy side effects.
  • Question 2.2: Does the length of time patients are willing to tolerate chemotherapy side effects to attain a personal goal change after they receive chemotherapy?
    • – For this research question, we tested the null hypothesis that patients' chemotherapy experiences affect the length of time they are willing to tolerate chemotherapy side effects. Our alternative hypothesis was that patients' chemotherapy experiences do not affect the length of time they are willing to tolerate chemotherapy side effects.
  • Question 2.3: What are the rankings of the chemotherapy side effects that patients would most wish to avoid?
    • – This question was a descriptive research question. Results included a ranking description of chemotherapy side effects that patients would wish to avoid (ranking of side effects ranged from the least- to the most-avoided side effects).
  • Question 2.4: Does the side effect patients rank as the worst change after they receive chemotherapy?
    • – For this research question, we hypothesized that chemotherapy treatment would lead to a change in patients' ranking of the worst side effects of chemotherapy drugs. We tested the hypothesis that chemotherapy experiences affect patients' ranking of drug side effects.
  • Question 2.5: What is the concordance between the drug received and the drug that the patient would most wish to avoid based on side effects?
    • – This question was a descriptive research question. Results reported a description of concordance of the drug received and the drug that the patient would most wish to avoid based on preferences of drug side effects.
  • Question 2.6: Does the concordance between the drug received and the drug that the patient would most wish to avoid based on side effects change after receiving chemotherapy treatment?
    • – The question was an exploratory analytical inquiry with no prior stated hypothesis. Results included whether patients' experiences with chemotherapy changed the concordance between the drug received and the drug they would most wish to avoid based on preferences of drug side effects.
  • Question 2.7: Are patient characteristics associated with the concordance between the drug received and the drug that patients would most wish to avoid?
    • – This an exploratory analytical research question with the following hypothesis: Patient characteristics (ie, age, gender, education, marital status, and residence) are associated with the chemotherapy drugs received. Our alternative hypothesis was that patient characteristics are not associated with the chemotherapy drugs received.

Aim 3: Determine if Oncologists Are Likely to Change Their Chemotherapy Treatment Strategy From the Initial Treatment Plan When Provided With Information Related to Patient Preferences

  • Question 3.1: Are oncologists in the intervention group who receive information about patient-reported side effects (which are unique to a single cancer drug in this group) more likely to change their treatment plan than oncologists in the control group who receive information about patient-reported side effects (which are common to all cancer drugs in this group)?
    • – For this research question, we tested the hypothesis that oncologists would be more likely to change the treatment plan after receiving patient-centered drug side effects information.
  • Question 3.2: What factors primarily influence oncologists' choice of first-line treatment of patients who have been newly diagnosed with or have newly progressed to advanced-stage NSCLC?
    • – The research question was a descriptive research question. Results included factors that influence oncologists' choices of first-line treatment for newly diagnosed advanced-stage NSCLC.

Specific Aim 1: Determine if Patient Characteristics and Treatment Experiences Affect the Definition (Meaning) of Treatment Success

Methods

Study Design

To achieve aim 1, we collected quantitative data utilizing an observational longitudinal open cohort study design. The clinic study nurse conducted patient interviews on 3 separate occasions using structured questionnaires that were designed to collect study-related quantitative and qualitative data at patients' scheduled visits before chemotherapy, during chemotherapy, and after completion of chemotherapy. The study nurse collected patient characteristics at baseline via review of medical records.

Participants

All patients who appeared for their chemotherapy treatment plan visit at the 9 study clinics between January 1, 2014, and March 31, 2016, were screened for possible enrollment in the study. The target sample size for this specific aim was 210.

All those patients who met the eligibility criteria (ie, diagnosed with advanced-stage [3b and above] NSCLC, aged 19 years or older, able to understand spoken English, willing and able to provide informed consent, eligible to undergo chemotherapy for advanced-stage NSCLC) were enrolled in the study.

Forming the Study Cohort

To establish a cohort for the specific aim 1, site coordinators identified patients by reviewing patient schedules (ie, looking for those patients with lung cancer), consulting with oncologists, and screening for patients with advanced-stage NSCLC, with particular interest in those who had been newly diagnosed and who met eligibility criteria. A list of eligible patients was kept at each site, along with documentation for those deemed ineligible for the study. Staff of the coordinating center (CC) conducted training of site staff on all aspects of the study, including patient screening and enrollment, as well as the study protocol standard operating procedures. The CC also monitored all key aspects of study processes and methodology at each site via regular site visits, email, phone calls, and video conferencing.

Study Settings

We enrolled subjects from 9 cancer centers located in the US Midwest and South: Nebraska (n = 5), South Dakota (n = 1), Kansas (n = 2), and Florida (n = 1). We chose the settings based on best fit for accomplishing research goals and geographical constraints of co-investigators and other research staff.

Study Participants

Of 292 patients eligible for recruitment, we enrolled all 235 adult (aged 19 and older) patients with advanced NSCLC at the cancer center sites who provided written consent and decided to undergo chemotherapy between January 2014 and March 2016. Eligibility criteria included patients diagnosed with advanced-stage (3b and above) NSCLC, aged 19 years or older, able to understand spoken English, willing and able to provide informed consent, and eligible to undergo chemotherapy for advanced-stage NSCLC.

Exposure

We administered no study intervention. However, treatment experience for each patient was based on medical record documentation of receipt of chemotherapy in compliance with each clinical site's standard of care.

Follow-up

Site staff scheduled participants for follow-up as per study protocol to collect interview questionnaire data before, during, and after first-line chemotherapy (3 visits, on day 0, 3-4 weeks, and 15-16 weeks). Of the enrolled patients, 71% (168 of 235) of the enrolled patients had at least 2 interviews, and the median follow-up of the entire group (including those with only 1 interview) was 36 days (range, 0-13 months). In addition to documenting the details of their chemotherapy receipt and treatment experience via medical record review and interview, a study nurse elicited information about patient definitions of treatment success using CRFs designed for this study.

Outcomes and Measures

The primary outcome of specific aim 1 was the patient's definition of treatment success in response to specific questions related to the treatment outcomes he or she wanted to achieve. To develop key categories defining success (ie, living longer or survival alone, living longer with other options, QoL without living longer, undecided or not reported), we held focus groups at the development stage of the study with patients and stakeholders. We also used these focus groups to develop an appropriate open-ended question interview tool for eliciting patient definitions of treatment success. In this questionnaire, patients were given a question that allowed them, in an open-ended manner, to define treatment success in their terms. In addition, the study nurse asked patients specific questions, such as the following: “Would you be willing to tolerate ALL of the side effects of treatment listed in question 1 above if it meant you might live longer?” The questionnaire about patients' willingness to tolerate side effects if it meant living longer listed chemotherapy drug–related side effects (the questionnaires are attached in Appendix C, Appendix D, Appendix E, and Appendix F). Interviewers administered these questionnaires to the patients at enrollment, when they came for their chemotherapy treatment plan, after starting chemotherapy (during a scheduled clinic visit after completing the first chemotherapy cycle), and then after completing the entire chemotherapy cycle.

Key category themes that emerged from the focus groups and the patient interviews included the following: living longer (LL) or survival alone; improvement of QoL only, without LL; improvement of QoL and LL; and other responses related to personal goals. An example of the latter category would be, “I want to live until the birthday of my grandson.” We used these categories, which centered on survival (LL), improved QoL, and attainment of personal goals, to define the definition of treatment success outcome variable. (The questionnaires used for the study and the process of the focus groups are included in Appendix C, Appendix D, Appendix E, and Appendix F) and a prior published paper (see Islam KM, Opoku ST, Apenteng BA, et al. Engaging patients and caregivers in patient-centered outcomes research on advanced stage lung cancer: insights from patients, caregivers, and providers. J Cancer Educ. 2014;29(4):796-801. doi:10.1007/s13187-014-0657-3).

Patient characteristics collected from the medical records using CRFs included socioeconomic and demographic characteristics such as age, gender, race, education, employment, marital status, income, primary method of payment, geographical location, and availability of insurance.

The outcome measurement of interest to this study aim was the patient's definition of treatment success measured at interviews prior to (first interview) and after (last interview) chemotherapy, which collectively described the exposure as treatment experience.

We also investigated the hypothesis that these characteristics were related to the patient's definition of success, as well as to the likelihood of change in its definition following chemotherapy. The before and after comparison of definitions of success could potentially reveal baseline characteristics that predicted change and could be used to guide treatment decisions that anticipate changes in what constitutes treatment success after patients undergo treatment.

Data Collection and Sources

Trained research staff at the study sites used face-to-face interviews, guided by study-developed questionnaires (Appendix C, Appendix D, Appendix E, and Appendix F), to gather data related to treatment success definition, treatment experience, and ranking of preference of side effects from enrolled patients when they came to the clinic for their first clinical visit after being diagnosed with advanced NSCLC. In addition, clinical record review provided supplemental information related to patients' chemotherapy treatment, including drugs received, side effects, comorbidities, and sociodemographic information (for complete data collection, refer to Appendix C, Appendix D, Appendix E, and Appendix F). Site coordinators kept research logs and calendars to track when follow-up data collection interviews were due for each participants. They also checked with clinicians, their center's master schedule, and the patient's charts, and read obituaries to maximize follow-up rate, to minimize and ascertain reasons for loss to follow-up, and to record reasons for dropouts and missing data. We put procedures in place to ensure that study participants' expenditure of time and energy was kept to a minimum and did not exceed that specified in the informed consent. The patient survey broad time points for administering a maximum of 3 possible interviews were before, during, and after first-line chemotherapy, and we scripted interview questions appropriately to correspond to that time frame. Specifically, interview No. 1 occurred before or early in first-line chemotherapy (before a second infusion or dose of chemotherapy). Interview No. 2 could occur anytime between the second infusion or dose and nearing the end of first-line therapy. Interview No. 3 could occur near the end of first-line therapy when the patient was entering therapy maintenance, and/or during a post-therapy oncologist visit after concluding chemotherapy.

Additional data sources included patient medical records, with specific retrospective baseline and prospective tracking data collected by authorized research staff at participating sites. We entered the latter data into the REDCap data collection system and submitted them electronically to the CC at the awardee university.

Analytical and Statistical Approaches

For aim 1, we tabulated the distribution of the treatment success definition according to the study-identified key categorical themes at the first interview and last interview (for those patients with more than 1 interview). We also assessed changes in these treatment success definitions between the first and last interview using the McNemar test. We assessed the association between each of the patient characteristics and the 2 main categorical (dichotomized) themes, which were living longer and living longer plus other options, at first interview using bivariate (for each characteristic) chi-square and Fisher exact tests and using multivariate analyses based on logistic regression, OR, and 95% CI. We developed the categories as they emerged from the focus group and cohort data. First, we dichotomized the outcome data since the overall research hypothesis was whether patients define treatment success as survival alone or survival plus something else. Then, we conducted subgroup analyses to capture all thematic areas by using multiple categories. For multivariate logistic regression, we included variables based on expert opinion and literature review findings. We also assessed association between patient characteristics and changes in their definition of treatment success between the first and last interview using bivariate (for each characteristic) chi-square and Fisher exact tests and using multivariate analyses based on logistic regression, OR, and 95% CI. We excluded patients with missing variables from the multivariate analyses were excluded; we did not impute the missing values, as missing values for variables were relatively minimal.

Overall Methods for Handling Missing Data (for All Aims)

We collected most of the study data via interviewing patients. We anticipated that the most missing data would be due to 1 or more of the following: because of patient-specific occurrences, behaviors, and choices (eg, dying, moving away, or responding “I don't want to talk to you now”); and/or influenced by our IRB requirements that all subject input is voluntary, which means that any question may be left blank if the patient chooses, and the patient need not give a reason for choosing not to answer any question (see Appendix J). Our clinic-based, personal approach enabled us to minimize missing data elements compared with data collection by indirect means, such as mail or fax. Thus, site coordinators could easily review patient responses while patients were still in the clinic. We rigorously evaluated missing data and accuracy of data once data were received at the CC. To increase the probability of detecting any errors or missing data, different research staff members fully checked data entries twice. Additionally, we compared database data to hard-copy CRFs on an ongoing basis for quality assurance and control.

Per the study protocol, we had planned to employ multiple imputation only if the missing data proportion was greater than 10%. Since the highest missing data proportion of variables used in our analysis was 7.8%, we analyzed the data by excluding missing values and did not use validated methods to deal with missing data. According to statistical standards, this low level of missing data is unlikely to affect data estimates negatively.

Amount of Missing Data: Patient Level

Figure 1 describes patient flow as well as information that relates to missing enrolled subjects. Out of 292 eligible patients, 235 were enrolled; of those who were not enrolled, the most common reason was because they declined to participate in the study (n = 53). Among the 235 enrolled patients, 168 (71%) completed at least 2 interviews; of those who did not complete the second interview, the most common reason was because investigators determined that they already had prior experience of chemotherapy (n = 27). The next most common reason for patients not having a second interview was patient death before the second interview (n = 18).

Figure 1. Phase 2 (Patients): Participant Flow.

Figure 1

Phase 2 (Patients): Participant Flow.

Analyses at baseline included all 235 patients, and post-chemotherapy analyses included all 168 patients who participated in the second interview. Analyses that evaluated change between baseline and second interview considered those patients who had at least 2 interviews only.

Amount of Missing Data: Variable Level

In assessing the association between patient definition of treatment success and patient characteristics, 235 of 244 (4% missing) patients provided evaluable information for assessment and 218 of 235 (7% missing) provided evaluable information for multivariable assessment of association between the characteristics and definition of treatment success. In these analyses, we excluded those patients whose values were missing from the specific computations of measures of association that they affected but included them in all other analyses.

In assessing associations between patient characteristics and changes in patient definition of treatment success after chemotherapy (ie, between baseline and second interview), we considered for the bivariate analyses all 168 patients who had at least 2 interviews, excluding only a few patients who had missing information related to their education status (n = 4 [2%]) and income status (n = 50 [30%]) from those specific bivariate analyses. All the patients who did not provide information about their income indicated that they preferred not to. In the multivariable model assessing this association, we included 164 of 168 (2% missing) patients in the model that considered variables observed in the bivariate analyses and/or based on expert opinion; we did not include those variables with substantial missing values.

Conduct of the Study (for all aims)

Due to the patient-centered, progressive nature of the design and implementation of this study, wherein we used results of each stage to inform and refine the next, we were careful to seek advice before proceeding with anything that might be perceived as a deviation from protocol. Therefore, we frequently discussed with—and followed through on—guidance from PCORI program officers, our University of Nebraska Medical Center (UNMC)–based core group advisory council specifically formed for this study, and others as appropriate. For example, with the review and approval of our UNMC scientific review committee, which checked to ensure that we would not be deviating from our proposed protocol, we refined the eligibility criteria to increase the number of patients eligible to enroll. One of the refinements we made was to include subjects who were not naive to chemotherapy for advanced stages of NSCLC. This and other such refinements gave us the ability to reach and exceed our sample size goal and resulted in reliable study results and better representation of patients with advanced-stage lung cancer.

In the protocol, we planned to conduct the study in 4 cancer centers. Over the course of the study, however, we recruited more cancer centers, ending with 9 participating clinical sites. The main reasons for this action were to enhance the quality of the results by accruing as many subjects as time and funds would allow and to enhance our dissemination efforts. This decision also accomplished our goal to implement what we learned through engaging with and disseminating information to many key stakeholders. We believe that participating in this study heightens awareness of the importance of patient-centered outcomes research and encourages use of the patient-informed tools that we developed for this project, which could result in more rapid integration of approaches and tools into clinical practice. We undertook these steps to promote positive patient outcomes both immediately and in the long term.

Results

We identified 292 patients with advanced-stage NSCLC who were eligible for the study, of whom 53 declined to participate and 4 were not able to participate because they either decided not to undergo chemotherapy or were too ill. We enrolled 235 patients in the study (Figure 1).

Of the 235 enrolled patients, 58% (136 of 235 [58%]) were naive to advanced-stage chemotherapy. The remaining participants were undergoing chemotherapy (70 of 235 [30%]); had completed chemotherapy or progressed to maintenance chemotherapy (23 of 235 [10%]); opted to have chemotherapy elsewhere (5 of 235 [2%]); or decided to decline chemotherapy after being enrolled in the study (1 of 235 [0.4%]). Overall, 71.5% of patients participated in at least 2 interviews.

We recruited 235 patients to answer question 1.1 (we needed 196 subjects) to achieve statistical power of at least 80%. To address question 1.2, we recruited 168 subjects, exceeding the needed 108 subjects to achieve power of at least 80%.

The average (SD) age of subjects was 68 (10) years. Being mainly from the US Midwest, they reflected the general population in that they were predominantly White (95.3%) and more rural or highly rural (34%) than other regions of the United States. Slightly more men (55.3%) than women (44.7%) participated in this study. Most were unemployed or retired (74.5%) and married (61.7%). About half (51.5%) were Medicare beneficiaries, and a third used private insurance (Table 1).

Table 1. Phase 2 (Patients): Characteristics at Baseline (N = 235).

Table 1

Phase 2 (Patients): Characteristics at Baseline (N = 235).

We analyzed patient definitions of chemotherapy treatment success and examined the effect of patients' characteristics and experience with chemotherapy on their definition of success. At their first interview, most patients defined treatment success as more than survival alone (60.4%). They wished to live longer with a good QoL, have time with family and friends, and/or reach personal goals. Of patients, 23% defined chemotherapy treatment success as simply a good QoL. Less than 12% considered survival alone as their definition of chemotherapy treatment success. The proportional distribution of what constituted treatment success was similar before and after chemotherapy (Table 2). However, the definition of success changed for about a half (47%) of patients who were interviewed at least twice—including before and after in-study initiated chemotherapy (Table 3).

Table 2. Phase 2 (Patients): Outcome Measures.

Table 2

Phase 2 (Patients): Outcome Measures.

Table 3. Phase 2 (Patients): Patient's Definition of Treatment Success and Changes Between Their First and Last Interviews.

Table 3

Phase 2 (Patients): Patient's Definition of Treatment Success and Changes Between Their First and Last Interviews.

Table 4 shows the direction of the change in the definition of treatment success and between the first and last interviews. While not reaching statistical significance, of the 12% of patients who at first interview defined treatment success as survival alone, 80% changed their definition to more than just survival and now included improved QoL at the last interview following chemotherapy. Also, more than 90% of those who initially indicated that treatment success meant both QoL and LL chose that definition at the last interview following chemotherapy (Table 4). A further analysis (data not shown) suggested that, among patients who first gave survival alone as their treatment success definition, 52.4% changed it to LL plus other goals, and 23.8% changed it to improved QoL only. These findings highlight the importance of QoL for patients before and following treatment and do not diminish their desire to live longer.

Table 4. Phase 2 (Patients): Change in Treatment Success Definitions Between First and Last Interviews: “Living Longer” vs “More Than Survival Alone” (N = 152).

Table 4

Phase 2 (Patients): Change in Treatment Success Definitions Between First and Last Interviews: “Living Longer” vs “More Than Survival Alone” (N = 152).

Bivariate data analysis between patient characteristics and definition of treatment success from the first interview showed no statistically significant associations. However, younger patients (<60 years old) and those who were unemployed preferred living longer alone (both P = .13) (Table 5). Also, while males aged 61 to 70 years and married men tended to define treatment success as including more than survival alone, neither of these characteristics was statistically significant in multivariate analyses (Table 6).

Table 5. Phase 2 (Patients): Treatment Success Definition at First Interview by Patients' Characteristics, With 2 Outcome Categories (N = 224).

Table 5

Phase 2 (Patients): Treatment Success Definition at First Interview by Patients' Characteristics, With 2 Outcome Categories (N = 224).

Table 6. Phase 2 (Patients): Multivariable Analysis of the Association Between Patients' Characteristics and Treatment Success Definition at First Interview: “Living Longer” vs “More Than Survival Alone” (N = 218).

Table 6

Phase 2 (Patients): Multivariable Analysis of the Association Between Patients' Characteristics and Treatment Success Definition at First Interview: “Living Longer” vs “More Than Survival Alone” (N = 218).

Also, in bivariate analysis, patients who earned an income <$45 000 were more likely to change their definition of treatment success after chemotherapy (P = .02; Table 7). Those with less than a college degree (P = .14) and those with Medicare and Medicaid and without private insurance (P = .13) were more likely to change their definition of treatment success (Table 7). Multivariate analyses, however, revealed no statistically significant associations between patient characteristics and a change in the definition of treatment success (Table 8).

Table 7. Phase 2 (Patients): Changes in Treatment Success Definition Between First and Last Interview by Patients' Characteristics for Patients Who Had >1 Interview (N = 168).

Table 7

Phase 2 (Patients): Changes in Treatment Success Definition Between First and Last Interview by Patients' Characteristics for Patients Who Had >1 Interview (N = 168).

Table 8. Phase 2 (Patients): Multivariable Analysis of Changes in Treatment Success Definition by Patients' Characteristics (N = 164).

Table 8

Phase 2 (Patients): Multivariable Analysis of Changes in Treatment Success Definition by Patients' Characteristics (N = 164).

Specific Aim 2: Assess Patients' Treatment Choices Based on Their Ranking of Unwanted Drug Side Effects

Methods

Study Design

To achieve aim 2, we collected quantitative data utilizing an observational longitudinal study design. We interviewed patients at baseline and after receiving chemotherapy during the January 1, 2014, through March 31, 2016, study period.

All patients who appeared for their chemotherapy treatment plan visit at the 9 study clinics between January 1, 2014, and March 31, 2016, were screened for possible enrollment in the study.

All of those who met the eligibility criteria were enrolled in the study. The eligibility criteria included patients diagnosed with advanced-stage (3b and above) NSCLC, aged 19 years or older, able to understand spoken English, willing and able to provide informed consent, and eligible to undergo chemotherapy for advanced-stage NSCLC.

The purpose of this aim was to create an algorithm to identify chemotherapy drug choices that were sensitive to patient-identified preferences about tolerance level for possible chemotherapy drug side effects. We had created a list of common chemotherapy side effects of commonly used drugs for advanced-stage NSCLC. A detailed list of the side effects appears in Appendix A. A study nurse collected patients' tolerance level of side effects using a simple data collection form. In this form, we showed patients a distressed faces scale, numbered 1 to 7. Then we asked patients to show which distressed face (No. 1-7) corresponded to their level of tolerance for each of the side effects. We used the information from the distressed faces scale to identify patient tolerance of side effects.

We then showed patients a list of drug-specific patient-centered adverse events (PCAEs)/side effects (9 side effects). We presented these as paper cards with the side effects written on them. Then the nurse coordinator asked the patients to order the cards from the 1 listing the side effect that they would hate the most (on top) to the 1 that they would hate the least (on the bottom). We called this process a ranking exercise. We used the information from the ranking exercise to link patients' side effect tolerance to patients' drug choice. More details are provided under the “Outcomes and measures” heading under this aim.

Forming the Cohort

For specific aim 2, we utilized the same patient population or cohort used in aim 1.

Study Settings

We enrolled subjects from 9 cancer centers and their affiliated clinics located mainly in the US Midwest, as described in specific aim 1.

Study Participants

We enrolled all 235 adult (aged 19 years and older) patients with advanced-stage NSCLC at the above-mentioned cancer centers (from January 2014 through March 2016) who gave written informed consent, understood spoken English, and were eligible to undergo chemotherapy for advanced-stage NSCLC.

Exposure

Patients received chemotherapy treatment as evidenced by the treatment record. For aim 2, we conducted an observational study with no intervention.

Follow-up

Follow-up. Participants had a follow-up scheduled to collect interview data before and after the chemotherapy cycle. To maximize the follow-up schedule, we used a data collection interview/questionnaire before, during, and after first-line chemotherapy for advanced-stage NSCLC, depending on when the patient enrolled in the study.

Outcomes and measures

The outcomes of this specific aim were (1) patient-reported length of time they were willing to tolerate side effects, categorized as no time, time less than 1 year, and time more than 1 year; (2) patient ranking of side effects (most wanted to avoid); (3) concordance between the drug received and the drug that the patient would most wish to avoid based on side effects; and (4) changes during the course of chemotherapy in the concordance between the drug received and the drug that the patient would most wish to avoid based on side effects.

To measure the length of time that patients were willing to tolerate the side effects, they had to respond to a question with 3 categories of timelines related to length of tolerance (no time, time less than 1 year, and time more than 1 year) as part of a questionnaire administered by an interviewer. The questionnaire was administered at the time of enrollment, when patients came for their chemotherapy treatment plan, and after chemotherapy (during a scheduled clinic visit after completing the first chemotherapy cycle and then after completing the entire chemotherapy cycle).

To obtain the ranking of side effects, we generated an inventory of side effects (Appendix A). To create this side effects inventory, we searched the UpToDate website (https://www.uptodate.com/home) for the reported side effects associated with chemotherapy to develop a table with the side effects that are related to each of the lung cancer chemotherapy drugs. Then the research team identified the unique side effects of the drugs for this study. We developed an interview tool with CRFs to collect patient preferences about side effects based on this inventory. The nurse coordinator presented paper cards to the patients, each with 1 side effect written on it. Then the nurse coordinator asked the patients to order the cards from the one listing the side effect that they would hate the most (on top) to the one that they would hate the least (on the bottom). The objective of this exercise was to sort the possible unwanted side effects from “bad” (first) to “least bad” (ninth).

The exercise was completed with the patients in the following order:

  1. Place the 3 heading cards (red ink) before you from Hate–Most (on left), Hate–Medium (in the middle), and Hate–Least (on right).
  2. Next, sort the remaining cards into 3 stacks of 3 each under each heading: Hate–Most, Hate–Medium, and Hate–Least.
  3. Last, order each of the 3 cards within a group with the most dreaded side effect on top (first), the next most dreaded side effect second, and the least dreaded side effect third in each of the 3 columns. You should end up with the 9 cards in order from Hate–Most (the side effect you would dread most) on the top of the final stack or column to Hate–Least (the side effect that is the “least bad” for you) on the bottom of the final stack or column.

We also measured the changes in QoL score before and after chemotherapy (see aim 1). To assess a patient's drug choice from a list of PCAEs, we linked the side effect that patients would wish to avoid the most with the chemotherapy drugs associated with the specific side effect. Before the study, we had prepared a list of commonly used chemotherapy drugs for patients with advanced-stage NSCLC and their reported side effects. Since 1 PCAE could be associated with more than 1 drug, and vice versa (one drug could be associated with more than 1 side effect), we linked a specific side effect to a specific chemotherapy drug based on the higher percentage of reported side effects for each drug published at UpToDate. For each side effect that the patient would wish to avoid the most, we assigned the drugs that are highly associated with that side effect.

Data collection and sources

We used the patient cohort from specific aim 1 for specific aim 2. We recruited 235 patients with advanced-stage NSCLC for the quantitative data to answer aim 2. The data collection and sources are described in specific aim 1.

Additional data sources included patient medical records, with specific retrospective baseline and prospective tracking data collected by authorized research staff at participating sites. We entered these latter data into the REDCap data collection system and submitted them electronically to the CC at the awardee university.

Analytical and statistical approaches

We tabulated data to describe the proportional distribution of length of time that a drug side effect could be tolerated, which was categorized as no time, time less than 1 year, and time more than 1 year. We described proportions of drug side effects based on patients' ranking before and after chemotherapy treatment. Then, we linked the most unwanted drug (based on the proportion of patients ranking it most unwanted) with patients' ranking of side effects.

We used a chi-square test or Fisher exact test, as appropriate, to examine the association between each patient characteristic and outcomes. We used McNemar or Bowker tests to assess the discordance of individual patient responses between first and last interview, to meet the assumption of paired data. We used a chi-square test or Fisher exact test to examine the association between patients' characteristics and their relationship to drugs to avoid and drugs to receive. We set the significance level for all analyses at P < .05. We performed all statistical analyses using the statistical software package SAS, version 9.4 (SAS Institute, Inc).

Results

To address aim 2, we examined the association between patient characteristics and the length of time that patients said they could tolerate chemotherapy drug side effects. The results were consistent between the first and last interview. No patient characteristics showed a statistically significant association with the patient's ability to tolerate side effects. Only marital status showed a borderline significant association with the length of time that a patient could tolerate chemotherapy side effects (P = .059 in the first interview and P = .078 in the last interview). Overall, patients who were married tended to be more willing to tolerate treatment side effects for months or years while unmarried patients tended not to tolerate treatment side effects for any period (Table 9 and Table 10).

Table 9. Phase 2 (Patients): Tolerance Time at First Interview by Patients' Characteristics (N = 232).

Table 9

Phase 2 (Patients): Tolerance Time at First Interview by Patients' Characteristics (N = 232).

Table 10. Phase 2 (Patients): Tolerance Time at Last Interview by Patients' Characteristics (N = 167).

Table 10

Phase 2 (Patients): Tolerance Time at Last Interview by Patients' Characteristics (N = 167).

In the first interview, the proportion of patients who answered “months” (41%) was similar to those who answered “years” (43%), and 16% of patients were not willing to tolerate the side effects for any period. However, in the last interview, the results trended toward a slightly higher percentage of patients who indicated their ability to tolerate side effects was measured in months (50%) versus years (36%) (Table 11). About 48% of the patients changed their willingness to tolerate side effects between starting chemotherapy (first interview) and completing first-line chemotherapy (last interview). These results are shown in Table 11.

Table 11. Phase 2 (Patients): Length of Time Patients Willing to Tolerate Side Effects.

Table 11

Phase 2 (Patients): Length of Time Patients Willing to Tolerate Side Effects.

Between the first and last interviews, more patients shifted their answer from years of tolerating side effects to months, rather than going from months to years (Table 12). Among those who initially answered “months,” 24% changed to “years,” while 36% of patients who initially said “years” changed their answer to “months.” Although these results were not statistically significant (P = .475), they might highlight the importance of incorporating patients' views about side effects throughout treatment.

Table 12. Phase 2 (Patients): Changes in Tolerance Time Between First and Last Interview (N = 167).

Table 12

Phase 2 (Patients): Changes in Tolerance Time Between First and Last Interview (N = 167).

Patients were asked to rank side effects associated with 4 commonly used chemotherapy drugs in the treatment of advanced metastatic NSCLC (Table 13). The 3 side effects that patients would most wish to avoid (ie, shortness of breath, bleeding, and fatigue) stayed the same between the first and last interviews. However, many patients redefined their level of tolerance to months of experiencing a side effect versus years.

Table 13. Phase 2 (Patients): Proportion of Patients Who Ranked the Listed Side Effect as the One They Would Most Like to Avoid (Worst Ranked) at their First and Last Interviews (N = 168).

Table 13

Phase 2 (Patients): Proportion of Patients Who Ranked the Listed Side Effect as the One They Would Most Like to Avoid (Worst Ranked) at their First and Last Interviews (N = 168).

We linked the worst-ranked side effect with the chemotherapy drugs used for advanced-stage NSCLC (see Appendix A for matching of drug side effects with drugs). The distribution of chemotherapy drugs that patients wished to avoid based on these poorly tolerated side effects is shown in Table 14. Two of the drugs included at least a third of the side effect profile that patients would most wish to avoid. This distribution of the drugs did not change between the first and last interviews (Table 15).

Table 14. Phase 2 (Patients): Predicted Drug to Avoid Based on the Side Effects Patients Ranked as the Ones They Would Most Like to Avoid Matched With the Drug's Profile Based on Reported Side Effects Data, All Patients (N = 232).

Table 14

Phase 2 (Patients): Predicted Drug to Avoid Based on the Side Effects Patients Ranked as the Ones They Would Most Like to Avoid Matched With the Drug's Profile Based on Reported Side Effects Data, All Patients (N = 232).

Table 15. Phase 2 (Patients): Predicted Drug to Avoid Based on Patients' Preferences of Side Effects They Ranked Worst at First and Last Interview Matched With the Drug's Profile Based on Reported Side Effects Data.

Table 15

Phase 2 (Patients): Predicted Drug to Avoid Based on Patients' Preferences of Side Effects They Ranked Worst at First and Last Interview Matched With the Drug's Profile Based on Reported Side Effects Data.

Research staff who conducted the interviews were not instructed either to share or withhold patients' side effect preference information from the oncologists. Additionally, the interviews might or might not have been collected near the time of patient–physician interactions. Therefore, we did not assume that the oncologists were blinded to, or conversely had knowledge of, patient preference information when they prescribed the chemotherapy drugs that were given in real life. In Table 16, Table 17, and Table 18, those in the no discord category were patients who did not receive the drug that they would most wish to avoid. Patients in the discordant category were those who did receive the drug with the highest probability of the side effects that they ranked as ones they would most wish to avoid.

Table 16. Phase 2 (Patients): Comparison Between Drugs Received and Drugs to Avoid Based on Patients' Worst-Ranked Side Effects, All Patients (N = 226).

Table 16

Phase 2 (Patients): Comparison Between Drugs Received and Drugs to Avoid Based on Patients' Worst-Ranked Side Effects, All Patients (N = 226).

Table 17. Phase 2 (Patients): Comparison Between Drugs Received and Drugs to Avoid at First Interview by Patients' Characteristics, All Patients (N = 226).

Table 17

Phase 2 (Patients): Comparison Between Drugs Received and Drugs to Avoid at First Interview by Patients' Characteristics, All Patients (N = 226).

Table 18. Phase 2 (Patients): Comparison Between Drugs Received and Drugs to Avoid Based on Patients' Worst-Ranked Side Effects at First and Last Interviews, for Patients With at Least 2 Interviews.

Table 18

Phase 2 (Patients): Comparison Between Drugs Received and Drugs to Avoid Based on Patients' Worst-Ranked Side Effects at First and Last Interviews, for Patients With at Least 2 Interviews.

Results show that most of the patient chemotherapy drugs prescribed were concordant with their preferences; therefore, they are grouped in the no discord category (>85%) in both first and last interviews. This finding can be interpreted to mean that a large majority of patients did not receive the drug with the side effects they said they would most wish to avoid (Table 16 and Table 18). Of patients in the discordant group, results show disproportionately high numbers of patients with the following characteristics compared with the no discord category (Table 17): having education less than high school, not being married, having Medicaid as primary method of payment, and having a rural residence. Patients' willingness to tolerate chemotherapy side effects are reported in Table 19 and Table 20. In Table 21 we also report results of the QoL index, which showed no statistically significant changes before and after first-line chemotherapy.

Table 19. Phase 2 (Patients): Patients' Willingness to Tolerate Side Effects If It Meant They Might Live Longer.

Table 19

Phase 2 (Patients): Patients' Willingness to Tolerate Side Effects If It Meant They Might Live Longer.

Table 20. Phase 2 (Patients): Changes in Willingness to Tolerate Side Effects Between First and Last Interview (N = 168).

Table 20

Phase 2 (Patients): Changes in Willingness to Tolerate Side Effects Between First and Last Interview (N = 168).

Table 21. Phase 2 (Patients): Changes in QoL Score Between Before and After First-Line Chemotherapy Interviews Among Patients Who Started With First Interview and Had >1 Interview (N = 120).

Table 21

Phase 2 (Patients): Changes in QoL Score Between Before and After First-Line Chemotherapy Interviews Among Patients Who Started With First Interview and Had >1 Interview (N = 120).

Specific Aim 3: Determine if Oncologists are Likely to Change their Chemotherapy Treatment Strategy from the Initial Treatment Plan When Provided with Information Related to Patient Preferences

Methods

Study Design

To achieve aim 3, we conducted an exploratory pre–post randomized intervention study with oncologists from participating cancer centers as the study participants.

Intervention

First, all participating oncologists were provided with a clinical scenario of an NSCLC case and asked to provide a treatment plan. Then, these oncologists were randomized into intervention and control groups as described below. After that, using the same clinical scenario with additional intervention information about patient preferences, they were asked again to provide a treatment plan.

Participants

The participants were oncologists specializing in lung cancer treatment who were willing to participate in the study.

Eligibility

If oncologists provided verbal consent, the site coordinator administered a written informed consent.

Recruitment

The center nurse coordinator approached eligible oncologists to recruit as study participants for specific aim 3. She explained the study purpose and then answered any questions before obtaining informed consent. Using standard data collection forms (CRFs), we collected participating oncologists' demographic information and their practice type (ie, urban, rural, community, or academic setting). The PI and project coordinator trained the nurse coordinators and then reinforced this initial training via emails, phone calls, video conferencing, and site visits to ensure that site staff fully understood study enrollment criteria and the study protocol's standard operating procedures.

Study Settings

Five cancer centers participated in the study. Twenty-two oncologists from the 5 centers were randomly assigned to intervention and control groups.

Randomization, Allocation, and Concealment

We developed a list of 22 oncologists listed in the chronological order of their consent to participate. Then, using this chronological list, we randomly generated numbers (1-22) to assign them to either the intervention or control group. Based on these numbers, each consenting oncologist, as he or she enrolled, was assigned to the study arm corresponding to the group assignment of the random number generated for his or her assignment. According to this randomization method, 13 oncologists were assigned to the intervention arm and 9 oncologists were assigned to the control arm. The oncologists were blinded to their study arm assignment.

Interventions

Initially, all participating oncologists, regardless of study arm, were provided with a case scenario patient, and each oncologist was asked to provide a treatment plan based on this information. Following this, the oncologists who were randomized to the intervention arm were then presented with the same case scenario patient, along with patient-reported side effects that the patient would most wish to avoid that occurred with only 1 chemotherapy drug. The intervention arm oncologists were asked to provide a treatment plan based on this patient case scenario. The control group oncologists were offered the patient case scenario, in which the patient reported his or her preferences for side effects to avoid, which included side effects that are common to all chemotherapy drugs. The control group oncologists were asked to provide a treatment plan based on this case scenario.

The analysis then focused on any changes in the treatment plans before and after receiving the second case scenarios (which defined the intervention). While it is possible to assess the intervention arm for treatment plan changes that included specific and unique drugs associated with side effects that patients wanted to avoid, our analysis focused on identifying any change in the treatment plans in each of the study arms after receiving the second case scenario that defined the intervention.

In the intervention group, the oncologists received information about the side effects that the case scenario patient would most wish to avoid. These side effects occur with only 1 specific drug.

In the control group, the oncologists received information about the side effects that the case scenario patient would most wish to avoid. These side effects are common to all chemotherapy drugs.

Follow-up

No follow-up was needed for this intervention because the pre-data and post-data collection and the introduction of the intervention occurred at the same time.

Study Outcomes

For aim 3, the primary outcome was to determine if any change in oncologists' treatment plans occurred, measured before and after an intervention in which the oncologists received information about a case scenario patient's side effects preferences, including a scenario in which a simulated patient reported side effects unique to a specific drug as the ones they would most wish to avoid. The goal was to assess whether the oncologists took into account patient preferences regarding unique side effects compared with general side effects of chemotherapy drugs in developing a chemotherapy treatment plan for the patients. We performed this exercise only once, toward the end of the study.

Data Collection and Sources

Each oncologist was asked to complete a questionnaire that described several case scenarios and options for treatment to measure the outcome, which was the change in the oncologists' treatment plans measured before and after the intervention. The intervention group received discriminatory patient preference information while the control group received nondiscriminatory patient preference information of common PCAEs that provided little or no useful discrimination advantage to choosing a specific chemotherapy drug(s). Using standard data collection forms (CRFs), the nurse coordinator collected participating oncologists' demographic information and their practice type (ie, urban, rural, community, or academic setting). The case scenarios for intervention and control group are attached in Appendix H.

Sample Size

Aim 3 was an exploratory pilot aim. We took a convenience sample of oncologists based on all available consenting oncologists from the 4 centers who originally agreed to participate in this phase of the study. All of the approached oncologists participated in the study: 3 from UNMC, 3 from St. Francis, 3 from Callahan, and 3 from Avera (N = 20). Additionally, to increase sample size, we approached 1 more cancer center, and included 2 additional oncologists from that center, resulting in 22 oncologists.

Analytical and Statistical Approaches

We reported descriptive analysis results using proportions. We compared the characteristics of the oncologists in intervention and control groups using a Fisher exact test. To address aim 3, we analyzed before and after changes in oncologists' treatment plans between the intervention and control groups and evaluated them using Fisher exact test. We set the significance level for all analyses at P < .05.

Results

We completed an interview with 22 oncologists who were randomly assigned to an intervention group (n = 13) or control group (n = 9). Table 22 describes the characteristics of the oncologists. In both groups, most oncologists were male and White. In the intervention group, more than half of the oncologists (53.8%) primarily worked in a teaching hospital, which is higher than in the control group (33.3%). On average, physicians in the intervention group also had been practicing clinical oncology care for longer than physicians in the control group.

Table 22. Phase 3 (Oncologists): Characteristics of Physicians Participating in Phase 3 Study.

Table 22

Phase 3 (Oncologists): Characteristics of Physicians Participating in Phase 3 Study.

Results show that all oncologists in the intervention group (100%) adjusted their recommended treatment after receiving the study intervention, which was an actionable patient preference (preferences about drug side effects), while 62.5% of those in the control group changed their recommended treatment (P = .0421) (Table 23). Additionally, oncologists identified the 3 most influential factors on their chemotherapy drug choices as efficacy (86.4%), toxicity (81.8%), and patient factors (68.2%) (Table 24). We also explored oncologists' opinions about the importance of having additional information about which side effects of chemotherapy a patient would most wish to avoid. About three-fourths of the oncologists ranked the information as beneficial to very beneficial. The remaining fourth expressed a neutral opinion (Table 25).

Table 23. Phase 3 (Oncologists): Changes in Chemotherapy Agent Between Intervention Group and Control Group (N = 21).

Table 23

Phase 3 (Oncologists): Changes in Chemotherapy Agent Between Intervention Group and Control Group (N = 21).

Table 24. Phase 3 (Oncologists): Factors That Influence Chemo-Drug Choices (N = 22).

Table 24

Phase 3 (Oncologists): Factors That Influence Chemo-Drug Choices (N = 22).

Table 25. Phase 3 (Oncologists): Benefit of Patient Preference Information (N = 22).

Table 25

Phase 3 (Oncologists): Benefit of Patient Preference Information (N = 22).

Discussion

We used a multicenter prospective longitudinal patient-centered outcomes research study to explore chemotherapy drug treatment choices for patients diagnosed with advanced-stage NSCLC, and to determine if clinicians would incorporate patients' drug choices into the treatment plan. To our knowledge, no studies have attempted to define the treatment success for patients with advanced-stage NSCLC according to patients' terms or to assess patient preferences in direct relationship to individualized treatment choices at the time of clinical treatment planning for advanced-stage lung cancer. Our study was the first to address both these issues.

Our study was unique in 2 ways. This is the first study of this subject in which patients were involved from inception to implementation of the study. Second, the patient advocate who was co-investigator of the study had real-life experience working with patients who have lung cancer. This patient-centered approach ensured success at every step of the study.

Decisional Context

Aim 1: Determine if Patient Characteristics and Treatment Experiences Affect Patient Definition (Meaning) of Treatment Success

Our study reveals that, for two-thirds of patients, what constitutes treatment success is more than simply survival. Although ours is the first study on lung cancer to compare patient definitions of treatment success, we did find a study15 on physical therapy in which authors showed that patient definitions of treatment success criteria did not change pretreatment and posttreatment. We found that patient definitions of treatment success are dynamic, changing as patients go through treatment. This has important implications, especially for a global medical-centered approach in which survival is paramount and (as in the case of the Affordable Care Act) may play an important role in the payment system used by the physician and/or the health facility. More extensive review of evaluating outcomes will be useful, especially for patients with terminally aggressive conditions. Our exploratory assessment of socioeconomic and demographic factors that may be at play in any heterogeneity in the definition reveals no statistically significant variations. However, the trends showed that those who were younger and unemployed tended to define success primarily as survival, while men and married individuals (of either sex) tended to define success as more than survival. These trends need to be investigated further with a more heterogeneous study population than our relatively homogeneous population.

Although our findings fail to reach statistical significance about the impact of time on the experience of chemotherapy and changes in the definition of what constitutes treatment success, our results emphasize the importance of QoL before and (even more) after treatment without diminishing the desire to live longer. The change of QoL was especially prominent among those at the lower end of the income/education spectrum and among those on Medicare, which merits further investigation. Data showing the impact of treatment on QoL among patients with NSCLC are limited, and there are no data on the change in QoL before and after chemotherapy. One such study reported improvement of QoL among patients with NSCLC using longitudinal data from a palliative setting.16 Our findings imply that a patient-centered approach may need to involve patients from the onset of developing their treatment plan in defining what constitutes treatment success and then periodically reevaluate this definition with ongoing treatment. Further study that enables a more robust set of analyses, including adjustment of real-life covariates, such as Functional Assessment of Cancer Therapy–Lung (FAC-L) scores for functional and physical well-being, the chemotherapy regimens patients receive, number of treatment cycles, and patient experience/distress with side effects, is necessary. This would reveal if the patients' actual experiences with treatment(s) or personal factors (eg, whether patients have small children at home) influence the results. Exploration of patient-specific factors may enable researchers to learn how these factors might influence subsequent treatment decisions. Clinicians may benefit from knowing this information in advance to help patients arrive at treatment decisions.

Aim 2: Assess Patients' Treatment Choices Based on Their Ranking of Unwanted Drug Side Effects

Our results reveal a trend in married patients being more willing to tolerate treatment side effects for much longer than unmarried patients. Whether this is due to a patient's willingness to do everything not to leave a spouse alone or due to the support a spouse provides is not clear. This result indicates that familial factors and/or involvement of a spouse in the development of a treatment plan may be critical in ensuring treatment plan adherence.

About half of the patients changed their indication as to the length of tolerability of side effects between starting chemotherapy and completing first-line chemotherapy, with a large proportion of them redefining their level of tolerance in months versus years. This emphasizes the importance of the clinician constantly interacting and reevaluating a treatment plan with patients throughout the course of treatment.

While the side effects that patients would most wish to avoid (eg, shortness of breath, bleeding, and fatigue) remained stable at initiation versus after treatment, of note is the elevated prominence of fatigue on this list for many patients after their chemotherapy experience. Strategies that specifically monitor and try to control the effects of fatigue during chemotherapy may be warranted.

Two of the 4 drugs used for chemotherapy in our study include at least a third of the side effects patients most wished to avoid. In our study population, most of the patients did not receive those drugs whose profile included a side effect they were trying to avoid. However, a significantly higher proportion of patients with a risk profile indicative of poorer economic and social support (ie, having only a high school education, being single, receiving Medicaid, and living in rural areas) received those drugs whose profile included a side effect they would have preferred to avoid. Whether this observation is related to an aspect associated with their risk profile is not clear but needs further investigation. This observation also may indicate a need for clinicians to ask about and then listen closely to the preferences of this particular group in discussing their treatment plans and the options that may be available.

Aim 3: Determine if Oncologists Are Likely to Change Their Chemotherapy Treatment Strategy From the Initial Treatment Plan When Provided With Information Related to Patient Preferences

Our findings indicate that, while oncologists base their chemotherapy drug choices primarily on drug efficacy and toxicity, additional patient-centered side effects information would be beneficial for them to consider. In addition, our data suggested that when oncologists receive this patient-centered side effects information, they might use it to change their treatment plan depending on the clinical condition of the patient.

Study Results in Context

In the context of published literature, our study findings help advance current knowledge and confirm findings from other studies. Our review of the literature showed that ours is the first study that demonstrates the dynamics of patient-defined treatment success in real time during chemotherapy. Our study results confirmed findings from previous studies that showed multiple criteria need to be considered when making treatment decisions. For example, 1 study using a discrete choice experiment showed that, from patients' perspective, “progression-free survival” alone is not sufficient to serve as basis for treatment decision-making.17

Abundant literature shows that toxicity is an important consideration for patients in making treatment decisions,14 as also outlined by the National Cancer Institute.6 Dubey et al, for example, reported that, when faced with 2 different chemotherapy regimens with similar efficacy, most patients with NSCLC would consider the chemotherapy regimens' side effects.8 Our study confirmed the importance of toxicities in treatment planning, from both patients' and oncologists' perspectives. We noted that patients' perception about chemotherapy side effects changes over the course of the treatment, based on real-life treatment experience.

A literature review of patient' preferences for chemotherapy in NSCLC reported that baseline- and treatment-related characteristics are not predictive of patients' individual preference for chemotherapy, as is also the case in other cancers.13 Our study confirmed these findings to some extent. We found that patient characteristics did not significantly affect their treatment success definitions, although there was an indication that age, gender, and marital status may influence patient definitions of treatment success. This finding highlights the need for patient–provider communication during the decision-making process that respects patient values and circumstances. Further, our study findings improve current knowledge by demonstrating that oncologists are likely to incorporate patient preferences into their treatment plans.

Implementation of Study Results

Our study results indicate that patients can successfully play active, engaged roles in clinical research, ranging from participant to partner and key informant. Patient and stakeholder engagement is crucial in facilitating study implementation, especially for older patients with advanced-stage NSCLC. Participants suggested ways to recruit and retain patients with cancer in prospective clinical research. Acquiring adequate sample size for cancer care is an important issue. We collaborated with 9 cancer centers and their affiliated sites to successfully enroll patients, caregivers, and clinicians for this study, utilizing a patient- and clinic-centered approach. Study data showed that patients' views about cancer treatment outcomes differed from the traditional view of the definition of cancer treatment success. This knowledge is important for the clinical community that desires to improve patients' satisfaction, adherence, and improved QoL regarding chemotherapy.

Oncologists may engage patients efficiently using the tools that we developed for identifying patient preferences and incorporating patient choices into cancer treatment plans. When physicians are informed of the top 1, 2, or 3 side effects that patients would most wish to avoid because they feel they are least tolerable, physicians are usually willing to act on this information in a way that honors the patient-centered relationship in the clinical setting.

Physicians are likely to change their chemotherapy choice after they are informed about patient preferences. However, 62.5% of physicians in the control group also changed their choice (this might be due to the Hawthorne effect since they knew that they were under observation). In the control group, 37.5% of physicians did not change their choices. For patient compliance and improved QoL, we would like to see more physicians become willing to adapt their chemotherapy choices based on patient preferences.

The study data can be used to improve patient care by enhancing physician–patient communication, screening patients for comorbidity, identifying patient preferences for chemotherapy side effects (using our study-developed patient-centered tools), monitoring patient-reported and clinically documented drug toxicities, and conducting additional patient-centered clinical research. Our study results are available not only to clinicians but also to patients and their caregivers, which may further support patient-centered cancer care.

Research involving patients with advanced-stage lung cancer is limited for examination of patients' involvement in treatment decision-making.18 However, there is evidence that patients are confident in their role in clinical decision-making; their confidence can be further improved by involving them in the early stages of treatment planning.19 Evidence suggests that only half of patients with cancer undergoing either chemotherapy or radiation therapy or both perceive that they have been offered treatment choices.20 Our study highlights a patient-centered approach to engage patients in improved study design, execution, translation, and dissemination of study findings.

Patient preferences for treatment reflect patients' values, understanding of their illness, and understanding of the risks and benefits associated with treatment choices. Patient participation in treatment decision-making is a more complex issue than simply giving patients information and choices. We developed patient-centered tools for clinicians (ranking exercise and distress scale) to identify patients preferences for incorporation into the treatment plan. Our research data and tools will help patients and their caregivers make informed treatment choices for lung cancer care. This research corroborates Barry and Edgman-Levitan's statement that “shared decision-making is the pinnacle of patient-centered care.”21

Generalizability

We believe the findings from this study can be adapted to other diseases, including other types of cancer as well as some chronic diseases. Patient definitions of treatment success in our study may not be entirely identical in another population, but the essence of our findings can be generalized to any population. Knowledge and understanding about patient preferences, development of CRFs, recruitment, retention, and dissemination of study finding in patient-preferred ways can be translated to other cancer or chronic diseases research. Our study results reflect views of the patients with NSCLC from the US Midwest and Florida, predominantly a White population. The generalizability should be limited to similar populations.

Study Limitations

Our study had several limitations. While there is potential for selection bias due to voluntary participation of the patients, we believe the impact on our study is minimal since we managed to recruit >80% of the patients we approached. Thus, we believe that our results should be generalizable to typical patients with advanced-stage lung cancer in the Midwest. The randomized clinical trial with oncologists that we used to address exploratory aim 3 was limited due to small sample size.

Future Research

The overarching finding of this study is that both patients and physicians are willing, and in fact want, more patient-centered care in the clinical setting. The caveat for physicians is that they fear disruption of the flow of patients through the clinic. The caveat for patients is that they want their preferences and views to be heard and used in their treatment planning and implementation. We recommend further clinical trials with a larger oncology clinician study group to gain stronger evidenced-based knowledge as to whether oncologists are likely to change their clinical practice after receiving a detailed communication of the chemotherapy side effects that their patients wish to avoid.

Conclusions

We conclude that patients' goals for treatment change over the course of treatment. Overall, half of the patients changed their treatment goal after receiving chemotherapy. Our interviews before, during, and after a course of chemotherapy showed that some patients' priorities for quality and length of life changed. More than 60% of patients preferred QoL, along with survival, over survival alone. Our findings also showed that familial factors and/or involvement of a spouse in clinical development of a treatment plan may be critical in ensuring treatment plan adherence.

In addition, patients can identify chemotherapy side effects that they would most wish to avoid if given the opportunity. In our study, the 3 side effects that patients would most wish to avoid (ie, shortness of breath, bleeding, and fatigue) stayed the same between the first and last interviews.

Finally, the study results suggest that oncologists are likely to change the chemotherapy treatment plan after being informed about patient preferences for chemotherapy drugs. Study results showed that all oncologists in the intervention group (100%) adjusted their recommended treatment after receiving the study intervention, while 62.5% of those in the control group changed their recommended treatment (P = .0421).

References

1.
Howlader N, Noone AM, Krapcho M, et al. SEER Cancer Statistics Review, 1975-2013. National Cancer Institute; 2016. Accessed July 25, 2019. https://seer​.cancer.gov​/archive/csr/1975_2013/
2.
American Cancer Society. Cancer Facts & Figures 2016. American Cancer Society; 2016.
3.
Goodwin PJ, Shepherd FA. Economic issues in lung cancer: a review. J Clin Oncol. 1998;16(12):3900-3912. [PubMed: 9850036]
4.
Kutikova L, Bowman L, Chang S, Long SR, Obasaju C, Crown WH. The economic burden of lung cancer and the associated costs of treatment failure in the United States. Lung Cancer. 2005;50(2):143-154. [PubMed: 16112249]
5.
American Cancer Society. Treatment Choices for Non-Small Cell Lung Cancer, by Stage, 2017. American Cancer Society; 2017. Accessed November 7, 2017. https://www​.cancer.org​/cancer/non-small-cell-lung-cancer​/treating/by-stage.html
6.
Maher AR, Miake-Lye IM, Beroes JM, Shekelle PG. Treatment of Metastatic Non-Small Cell Lung Cancer: a Systematic Review of Comparative Effectiveness and Cost-Effectiveness. Department of Veterans Affairs; 2012. [PubMed: 23905205]
7.
American Cancer Society. Cancer Treatment and Survivorship Facts & Figures 2014-2015. 2014.
8.
Dubey S, Brown RL, Esmond SL, Bowers BJ, Healy JM, Schiller JH. Patient preferences in choosing chemotherapy regimens for advanced non-small cell lung cancer. J Support Oncol. 2005;3(2):149-154. [PubMed: 15796447]
9.
Bruera E, Willey JS, Palmer JL, Rosales M. Treatment decisions for breast carcinoma: patient preferences and physician perceptions. Cancer. 2002;94(7):2076-2080. [PubMed: 11932912]
10.
Balogh EP, Ganz PA, Murphy SB, Nass SJ, Ferrell BR, Stovall E. Patient-centered cancer treatment planning: improving the quality of oncology care. Summary of an Institute of Medicine workshop. Oncologist. 2011;16(12):1800-1805. [PMC free article: PMC3248780] [PubMed: 22128118]
11.
Coulter A, Elwyn G. What do patients want from high-quality general practice and how do we involve them in improvement? Br J Gen Pract. 2002;52(suppl):S22-S26. [PMC free article: PMC1316137] [PubMed: 12389766]
12.
Choy ET, Chiu A, Butow P, Young J, Spillane A. A pilot study to evaluate the impact of involving breast cancer patients in the multidisciplinary discussion of their disease and treatment plan. Breast. 2007;16(2):178-189. [PubMed: 17158048]
13.
Sen S, Fawson P, Cherrington G, et al. Patient satisfaction measurement in the disease management industry. Dis Manag. 2005;8(5):288-300. [PubMed: 16212514]
14.
Blinman P, Alam M, Duric V, McLachlan SA, Stockler MR. Patients' preferences for chemotherapy in non-small-cell lung cancer: a systematic review. Lung Cancer. 2010;69(2):141-147. [PubMed: 20570009]
15.
Zeppieri G, George SZ. Patient-defined desired outcome, success criteria, and expectation in outpatient physical therapy: a longitudinal assessment. Health Qual Life Outcomes. 2017;15:29. doi:10.1186/s12955-017-0604-1 [PMC free article: PMC5282693] [PubMed: 28143546] [CrossRef]
16.
Thomas M, Spigel DR, Jotte RM, et al. nab-paclitaxel/carboplatin induction in squamous NSCLC: longitudinal quality of life while on chemotherapy. Lung Cancer (Auckl). 2017;8:207-216. doi:10.2147/LCTT.S138570 [PMC free article: PMC5679693] [PubMed: 29138610] [CrossRef]
17.
Muhlbacher AC, Bethge S. Patients' preferences: a discrete-choice experiment for treatment of non-small-cell lung cancer. Eur J Health Econ. 2015;16(6):657-670. [PMC free article: PMC4464047] [PubMed: 25135768]
18.
Leighl NB, Shepherd FA, Zawisza D, et al. Enhancing treatment decision-making: pilot study of a treatment decision aid in stage IV non-small cell lung cancer. Br J Cancer. 2008;98(11):1769-1773. [PMC free article: PMC2410111] [PubMed: 18506180]
19.
Burton D, Blundell N, Jones M, Fraser A, Elwyn G. Shared decision-making in cardiology: do patients want it and do doctors provide it? Patient Educ Couns. 2010;80(2):173-179. [PubMed: 19945818]
20.
Stacey D, Paquet L, Samant R. Exploring cancer treatment decision-making by patients: a descriptive study. Curr Oncol. 2010;17(4):85-93. [PMC free article: PMC2913835] [PubMed: 20697520]
21.
Barry MJ, Edgman-Levitan S. Shared decision making—pinnacle of patient-centered care. N Engl J Med. 2012;366(9):780-781. [PubMed: 22375967]

Journal Publications

The following peer-reviewed journal publications resulting from the research supported by PCORI award #CE-12-11-4351 have been published:

•.
Islam KM, Copur M, Tolentino A, et al. Patient-centered outcomes research in advanced stage lung cancer: study design and population. Int J Cancer Stud Res. 2014;3(101):31-35. doi:10.19070/2167-9118-140005 [CrossRef]
•.
Islam KM, Opoku ST, Apenteng BA, et al. Engaging patients and caregivers in patient-centered outcomes research on advanced stage lung cancer: insights from patients, caregivers, and providers. J Cancer Educ. 2014;29(4):796-801. doi:10.1007/s13187-014-0657-3 [PubMed: 24744120] [CrossRef]
•.
Islam KM, Wen J, Jiang X, Ryan JE, Fetrick A, Ganti AK. Developing a web-based, patient-centered data collection and management approach for a multi-center lung cancer study. J Integr Oncol. 2014;3(2):121. doi:10.4172/2329-6771.1000121 [CrossRef]
•.
Islam KM, Opoku ST, Apenteng BA, et al. Coping with an advanced stage lung cancer diagnosis: patient, caregiver, and provider perspectives on the role of the health care system. J Cancer Educ. 2016;31(3):554-558. doi:10.1007/s13187-015-0840-1 [PubMed: 25900672] [CrossRef]

Acknowledgment

Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#CE-12-11-4351) Further information available at: https://www.pcori.org/research-results/2013/how-do-preferences-treatment-change-after-patients-lung-cancer-start

Original Project Title: Patient-Defined Treatment Success and Preferences in Stage IV Lung Cancer Patients
PCORI ID: CE-12-11-4351
ClinicalTrials.gov ID: NCT02190864

Suggested citation:

Islam KM, Ganti AK, Ryan J, et al. (2020). How Do Preferences for Treatment Change After Patients With Lung Cancer Start Chemotherapy? Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/02.2020.CE.12114351

Disclaimer

The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.

Copyright © 2020. University of Nebraska Medical Center. All Rights Reserved.

This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/

Bookshelf ID: NBK589556PMID: 36888735DOI: 10.25302/02.2020.CE.12114351

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (6.0M)

Other titles in this collection

Related information

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...