An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Heidi M. Crane, MD, MPH, Rob Fredericksen, PhD, and Paul K. Crane, MD, MPH.
Author Information and AffiliationsGrowing recognition of the value of routine, systematic collection of patient-reported data in care has led to the development and use of self-report assessments, referred to as patient-reported outcomes (PROs). PROs assess conditions and behaviors that might otherwise be overlooked or that are difficult and time-consuming for clinicians to assess in brief clinical encounters.
We had 3 aims for this study:
Our results are from busy HIV specialty clinics in academic settings. Their geographical diversity is a strength, but our findings might not be generalizable to other clinical settings.
Priorities in HIV clinical care vary substantially between patients and clinicians, and by patient gender, race, and level of engagement in care. We identified social support and intimate partner violence as key clinically relevant domains of care for measure development.
A patient-reported outcome (PRO) is any report that comes directly from a patient (without interpretation by physicians or others) about how they function or feel about a health condition and its therapy.1 PROs provide the patient's perspective to help measure and understand effects of disease and treatment on general well-being, function, symptoms, satisfaction with treatment, and other outcomes.2 PROs are gaining increasing importance and acceptance in patient-centered outcomes research and clinical practice.3-5 The PCORI Methodology Standards emphasize that for patient-centered outcomes research, data should be collected with PROs unless there is a compelling reason not to do so; patient-centered outcomes are best reported by patients (PCORI Methodology Report, p. 43).83
To maximize their relevance to patient-centered outcomes research, PROs must be integrated into clinical care. PRO collection in routine care fails if PROs do not deliver valuable information to patients and clinicians or if they interfere with patient flow. Integrating PROs into routine clinical care has many potential benefits, including improved attention to patient concerns,6,7 improved patient–clinician communication,8-11 improved satisfaction with care,12 and detection and management of conditions.13-15 PROs can be used to screen for risk behaviors such as substance use, to signal patients' health-related quality of life (HRQOL) deficits, to prevent further HRQOL deterioration, and to tailor treatments to optimize HRQOL. Well-developed, validated assessment tools and systematic but efficient disease and symptom monitoring can enable clinicians to improve the quality of care.3
There are more than a million people living with HIV (PLWH) in the United States,16,17 and >50 000 new cases occur each year.18 HIV disproportionately affects African Americans and other minorities.18 Declines in mortality since antiretroviral therapy was introduced19-21 have led to an emphasis on long-term morbidity of HIV and its treatments.22,23
PROs regarding mental health, substance use, symptoms, adherence, and HRQOL are increasingly important in clinical research and care. Many PLWH are have a chronic illness or disability, with multiple coexisting conditions and many symptoms, making it difficult for them to communicate their concerns in a brief clinic visit. PRO collection helps patients prioritize and discuss their concerns with health care professionals.
PROMIS® developed a series of item banks for domains relevant to health and health care (see http://www.healthmeasures.net/explore-measurement-systems/promis and https://www.assessmentcenter.net for additional details). However, many PROMIS instruments are intended for research settings; fewer of these measures have been evaluated in the clinical care setting.
We have led the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort in overcoming barriers to PRO collection in care in high-volume HIV clinics across the United States.24,25 We collect data with handheld touchscreens such as iPads; patients have found this system easy to use regardless of their computer literacy level. Patients are scheduled with enough time to interact with PRO data collection immediately before their visit with a health care clinician. Patients and clinicians (including physicians and other clinicians) review PRO results together during the visit. Computerized collection of PROs facilitates truthful reporting of socially undesirable behaviors.26
We focus PRO collection on domains that patients value highly and that clinicians find interpretable and actionable. Clinicians are enthusiastic advocates for PRO collection when the results are clinically valuable. Simply delivering PRO results to them does not improve clinical decisions3; only when those results provide succinct, crucial information linked to evidence-based clinical recommendations can they facilitate better-informed clinical decision-making.3
Our successful collection of PROs considers clinician and patient preferences, patient burden, and clinic flow. The process facilitates the long-term accumulation of PRO data for patient-centered outcomes research; thus, research is a byproduct of the PROs' role in clinical care. Many CNICS patients are members of groups that are often underrepresented in research, including African Americans, Spanish speakers, and women; thus, the CNICS sites are an ideal setting to assess and optimize PRO implementation and content. This study was built on the successful implementation of PROs in CNICS to maximize clinical relevance while minimizing patient burden and allowing ongoing collection in busy clinical care settings.24,25,27,28
Much of the PROMIS item bank and instrument development has been in the context of clinical research studies rather than clinical care. More needs to be learned about how PROMIS instruments and banks work in the specific context of clinical care. Critically, the diversity of our patients has not been reflected in the diversity of people used to develop PROMIS items or to derive item calibrations for most PROMIS domains. We can, and should, do better to make sure that PROs reflect the voices of all our patients.
We also need to be certain that PROs reflect the voices for whom patient–clinician discordance has been shown to be high: This discordance is particularly high for African Americans and those recently diagnosed with HIV. Three evidence gaps are especially important: (1) the domains PLWH and other stakeholders value highly; (2) psychometric relationships between legacy instruments (specifically the instruments used in clinical care that pre-dated PROMIS) and the PROMIS instruments; and (3) the usefulness of PROMIS domains in the context of care, including understandability, psychometric characteristics, and construct validity among PLWH for the PROMIS domains that those patients value.
This study had 3 specific aims:
In aim 1, we focus on PROMIS domains that patients and other stakeholders consider most important for clinical care. This emphasis is meant to foster better patient–clinician communication that is targeted to areas most in need. We include the viewpoints of stakeholders such as patients and clinicians to facilitate domain selection.
Aim 2 fosters compatibility between legacy instruments and PROMIS instruments for high-priority domains. This is important because it facilitates seamless patient-centered outcomes research before and after changes in PROs at an individual clinic. For example, clinics might migrate to PROMIS measures from legacy instruments that had previously been integrated into care. Cocalibration makes historic PRO data usable by providing crosswalks between legacy instruments and PROMIS, and facilitates comparisons across sites that use different instruments over time for the same construct.
Given PROMIS' methodological sophistication and the efforts made to calibrate to nationally representative samples, we suspect that PROMIS might become standard for some domains.29 The aim 2 crosswalks will be especially useful to the research community, improving the comprehension and usability of data collected using legacy instruments.
In aim 3, we address the content validity of high-priority PROMIS measures to allow their integration into routine clinical care for PLWH. Important questions remain to be answered about the content validity of PROMIS item banks and the appropriateness and availability of PROMIS measures for use in clinical care. PROMIS measures, particularly the original measures, were developed primarily in research rather than clinical settings, and many critically important subgroups of patients were not as prominent as they might have been in the development and calibration of the measures. Aim 3 allows us to address some of these limitations more fully to facilitate the use of PROMIS measures among PLWH in clinical care.
Using a nationally distributed cohort of PLWH in clinical care, this project extends PROMIS validation work in PLWH. This is an important, understudied, and demographically diverse population that includes large numbers of underserved patients, including African American and other minorities, women, and people with disabilities. This project identifies and validates domains that patients and other stakeholders consider most important for their clinical care.
We engaged patients and other stakeholders—including clinicians, clinic leadership, and clinic social workers—in this project. We involved stakeholders at every step of our research on PROs. We used established strategies for patient engagement, including patient advisory boards and community advisory boards, engaging patients and focusing on outcomes that are important to them, and repeatedly soliciting their input on priorities to make sure that our work remained patient-centered.
We formed a patient advisory board of 3 diverse and engaged patients at the University of Washington (UW); they began collaborating on the project as we developed the proposal and remained involved throughout. We met with the board primarily to make sure patient priorities were included but also to address research design questions, evaluate results, monitor progress, and address dissemination plans. One of the more useful things we did when we formed the board was to recruit a patient who had been a vocal opponent of the clinical PRO assessment in the UW clinic. His participation gave him the opportunity to express his concerns and allowed us to learn from him and make improvements.
We also involved established community advisory boards from 3 clinics (Fenway Community Health Center in Boston, MA; the University of Alabama at Birmingham [UAB]; and UW in Seattle); we met with them for additional input and to disseminate findings. The number of participants varied over time and by site, but each board always had between 5 and 10 participants.
The patient advisory board was a collaborative partner in all aspects of this project from its conception; however, the largest areas of impact were in the first year, particularly related to setting patient priorities and selecting domains. The board's impact was also substantial in the planning phases for the project and while we wrote the proposal. An example of this impact is that board engagement and input led us to change the format and structure of the aim 1 rank order exercises to make them easier for patients. Another example is that the board (along with clinicians) advocated for the inclusion of intimate partner violence as a domain of interest. They also suggested changes in some of the wording in the Spanish version of the intimate partner violence items; this input led us to add another round of patient interviews with Spanish speakers to make sure that our meaning was clear after completing the English-to-Spanish and Spanish-back-to-English translation steps.
For all 3 aims, both the patient advisory board and the 3 community advisory boards were involved in various aspects, from design to reviewing results and collaborating on dissemination plans. By allowing aim 1 findings to drive the domains included in aims 2 and 3, the project addressed a key aspect of PCORI's definition of patient-centered outcomes research: the entire project focused on outcomes that patients care about and that were identified and determined with patient input. Engaged patients and other stakeholders exerted meaningful influence on the design and the conduct of the research, starting with discussions that began while we were developing the proposal.
In addition to meeting with the boards, we met with specific patient groups to gain input as needed. For example, we met with patients from Ethiopia and Eritrea regarding implementation of the Amharic language version of the assessment and how to deal with all the variants and dialects. Amharic is the third most common language (after English and Spanish) in 2 of our CNICS sites (University of California at San Diego [UCSD] and UW), both of which have large numbers of patients from Ethiopia and Eritrea. The meetings provided us with key insights into how to improve our approach to the CNICS PRO assessment for Amharic-speaking patients.
We used state-of-the-art methods for each analysis to develop a rigorous approach to our study. These approaches included analyses of qualitative data from patient focus groups and individual interviews using grounded theory.30 We also used quantitative data analyses, including psychometric approaches such as structural equation modeling; item response theory (IRT); and approaches to identify and account for differential item functioning (DIF) that we had developed,31 refined,32,33 and used in a variety of settings.34-41 We have made these psychometric methods freely available.42
Our project builds on the extensive resources of the CNICS cohort. CNICS was established in 2002 to better define relationships between patient and treatment factors and long-term outcomes among PLWH.24 The network contains clinical data from electronic health record (EHR) systems for those receiving HIV care at 8 sites across the United States: Case Western Reserve University, Cleveland, Ohio; Fenway Community Health Center of Harvard University, Boston, Massachusetts; Johns Hopkins University, Baltimore, Maryland; UAB; University of California at San Francisco (UCSF); UCSD; University of North Carolina at Chapel Hill (UNC); and UW, Seattle.
The CNICS cohort is racially diverse, has excellent sex and age representation, and is geographically diverse, with sites in multiple regions of the United States. We collected quantitative data from PLWH in routine clinical care in CNICS. Patients completed the PROs in English, Spanish, or Amharic (this option was added in 2016) as part of routine clinical care visits. Patients did not complete the PROs if they did not speak one of these 3 languages or were acutely intoxicated or otherwise cognitively impaired.
For qualitative work, we focused on enrolling diverse PLWH from at least 4 sites: Seattle, San Diego, Birmingham, and Boston. We identified informants based on the level of the domain in question to include sufficient numbers of patients with mild, moderate, and severe levels.
We oversampled key groups as needed to include adequate representation (most often Spanish speakers).
For quantitative work (aims 2 and 3), we included PLWH seen in care consecutively at 6 to 7 participating sites and continued to collect data until we met enrollment targets. Refusal rates for PRO assessments within CNICS have been consistently very low (<1%-<5% depending on the site). Lack of time has been the most commonly cited reason for postponement or refusal.
We sought clinician input across all 8 CNICS sites.
CNICS data types include demographic characteristics; medications; utilization data, including inpatient, outpatient, and emergency visits; PROs; HIV genotypic resistance; biologic specimens; and census block data. Diagnoses include AIDS-defining diagnoses, cardiovascular disease, diabetes, dyslipidemia, hypertension, mental illness, and substance use. The treating clinician prospectively recorded these diagnoses in an EHR when care was provided, using a constrained list of standardized diagnosis codes. Laboratory data include HIV-1 RNA levels, CD4 cell counts, and metabolic markers. Clinicians entered antianxiety, antidepressant, antipsychotic, antihypertensive, diabetes, lipid-lowering, and mood-stabilizing medications directly into the EHR, or medication data were downloaded directly from institutional pharmacy systems. Demographic data include sex, race, ethnicity, age, and HIV transmission risk factor. PROs distinguish CNICS from other HIV cohorts.24,27,43-47 Utilization data include scheduled and kept visits for HIV care and mental health care, and inpatient and emergency department visits.
This was not a longitudinal comparative effectiveness or intervention study, so follow-up and intervention descriptions are not relevant.
Although we did not have specific interventions, we did make selections in 2 areas that drove our research and results: high-priority domains and legacy instruments. First, we identified domains of great importance to PLWH and clinicians (aim 1). Our selection process ensured that the domains included item banks developed with the highest methodological quality and that they were of great relevance to patients and other stakeholders. Second, we identified legacy instruments for selected domains for cocalibration crosswalks. We chose legacy instruments based on literature reviews for key domains to identify commonly cited instruments and on discussions with patients and with experts among our colleagues, locally and nationally.
We identified a broad range of widely used legacy instruments. Our infrastructure and cocalibration study design enabled us to administer legacy instruments with instruments available through PROMIS. This yielded adequate data for cocalibration without excessive patient burden.
Outcomes from aim 1 were the PRO domains of greatest relevance for PLWH and their health care clinicians. The overall goal of aim 1 was to identify domains that patients and clinicians found to be of greatest value. These results then drove the aim 2 and aim 3 outcomes and kept the entire project focused on domains that patients and clinicians care about. Aim 2 outcomes focused on cocalibration; aim 3 outcomes focused on content validity and implementation in clinical care of domains important to patients and other stakeholders.
We have briefly described our overall data collection process and the key elements of CNICS data in the section on CNICS clinical data. Of most relevance to this project were PRO data from CNICS, which we collected during clinic appointments using a web-based survey software application.24,48-50 PROs are collected in English, Spanish, and (starting in 2016) Amharic; they include depression, anxiety, substance use, HRQOL, medication adherence, symptoms, and physical activity.51-66 However, questions remain as to whether these were the domains with the greatest effect on clinical care and, if not, how they can be improved. Patients completed the PROs at the beginning of routine care visits, and clinicians received the results or feedback at the start of the same-day visit. The approach to providing the clinician with PRO feedback varied by clinic; some used paper-based feedback and others used electronic feedback, depending on the clinician, the EHR, and clinic preferences.
Patients use touchscreen tablets connected to a wireless network. Network communications are encrypted using secure sockets layer/transport layer security; no patient data are stored on tablets.
The PRO interface is easy to navigate. Questions are at an elementary school reading level and are displayed with large, clearly labeled radio buttons for responses. Typing is not required or permitted. To prevent double or ambiguous answers, only one response is allowed per item. Respondents can easily correct mistakes. Skip patterns decrease patient burden. The system records the time of completion for each item, facilitating quantitative assessment of patient burden. The system has been well accepted by patients.24
We can easily rotate additional instruments (eg, for aim 2 cocalibration) into and out of the CNICS PRO data collection. This ability has allowed us to meet quantitative data collection goals across sites.
Aim 1 included 4 phases (see Table 1). In phase 1, we convened 3 small patient focus groups to develop and beta-test an individual interview guide for phase 2 interviews about the importance of domains. We held groups at UAB, UCSD, and UW. We have previously found that using these 3 locations gives our projects racially and ethnically diverse patient stakeholders and a mix of urban and rural inhabitants. The UCSD group was conducted in Spanish with Spanish-speaking patients.
Phases in Aim 1.
In phase 2a, patients ranked the domains by priority. We conducted in-depth interviews with a subset of patients who had completed this exercise, using the guide developed in phase 1. We conducted these activities in 5 clinics: Fenway, UAB, UCSD, UCSF, and UW. Phase 2a enabled us to represent patients in underserved populations by oversampling the following: patients living in rural areas; women; African Americans; monolingual Spanish speakers; English-speaking Hispanic patients; patients with multiple coexisting conditions; patients who are poorly engaged in HIV care; and patients aged 55 years or older.
In our previous research, we found that recently diagnosed PLWH (<5 years) had among the greatest discordance with clinicians on domain importance. For that reason, we also oversampled this group, which is known to be a high-risk group for a number of poor outcomes in HIV care, such as more stigma and poorer retention in care.
In phase 2b, HIV clinicians across all 8 CNICS sites completed the rank order exercise.
Phase 3 consisted of focus groups at 5 sites (Table 1). In these groups, patients individually performed the rank order exercise from phase 2, then viewed the aggregate results from their group and participated in a group discussion of the rationale for rank order choices. After the discussion, they had the opportunity to re-rank their domains.
In phase 4, we presented our findings from interviews, surveys, and focus groups from phases 2 and 3 to the 3 community advisory boards (Table 1). We solicited recommendations from the community advisory boards and from the patient advisory board to help determine which domain areas to advance to aims 2 and 3.
We tallied responses to surveys and ranked domains in order of importance. Two trained coders transcribed and coded interviews for domain-specific content and direct statements describing rationales for rank order selection. We used an open coding process to identify important attributes within each domain, alongside thematic content within statements of rationale for rank order selections. We assessed coded content and interrater agreement. We used the method devised by Hollander and Sethuraman to test whether the difference in rankings between patients and health care clinicians was statistically significant.67
We identified highly informative legacy instruments based on instruments currently in use in CNICS or frequently cited in MEDLINE. We performed literature reviews to identify widely used legacy instruments that measure domains identified in aim 1 by patients, clinicians, and other stakeholders. We administered the CNICS legacy instrument (if applicable), such as the 9-item Patient Health Questionnaire (PHQ-9) for depression and the consumption version of the Alcohol Use Disorders Identification Test (AUDIT-C), the PROMIS Alcohol Use Short Form (PROMIS SF), and others. We collected data on ≥500 patients for each domain to ensure that key subgroups were well represented. We included all patients in care for at least the first 500 patients; after the first 500, however, we often left an instrument in the assessment for specific groups of interest, such as women or Hispanic patients, to ensure that underrepresented patient groups were appropriately captured.
We cocalibrated legacy instruments to the PROMIS metric and compared measurement properties of the instruments, using script files (code that enables a statistical package to perform specific analyses) that we have published.68 (Cocalibration is calibrating 2 measures so that the resulting scores are on the same metric.) The steps involved in these cocalibration analyses include assessing dimensionality (ie, are the 2 instruments measuring the same construct?), determining whether PROMIS item parameters can be used for PLWH, determining whether the PROMIS SF and the legacy instrument can be modeled using the same single-factor model, and conducting IRT analyses with PROMIS item parameters fixed to their PROMIS values (if all assumptions are met). This process enables us to obtain scores from any subset of items on the same metric; in this case, the metric defined by the PROMIS scale.
We also performed analyses for DIF related to several covariates. We conducted known group comparisons, which is an important step of validity testing. For example, individuals with less social support would be predicted to have more depressive symptoms and poorer HRQOL than those with more social support. This was a general approach to cocalibration. The following paragraphs describe this approach as applied to the alcohol use domain and cocalibration of the PROMIS alcohol measure and the AUDIT-C (alcohol consumption) measure.
Aim 2 cocalibration example: alcohol use. Although alcohol use assessment is important, the optimal measure to screen for at-risk alcohol use in routine clinical care settings has not been clearly identified. One well-known option that is frequently used to screen for at-risk or hazardous alcohol use is the AUDIT-C,53,54 a 3-item instrument that measures the frequency and amount of alcohol a respondent has consumed, including binge drinking, within the past year.
Previous work has cocalibrated the related 10-item AUDIT to the PROMIS Alcohol Use and Alcohol Negative Consequences domains. The PROMIS Alcohol Use domain has shown better measurement precision than the AUDIT for alcohol use as defined by the PROMIS items.69
The previous analyses of the PROMIS Alcohol Use domain did not, however, consider at-risk alcohol use as identified by the AUDIT-C. The AUDIT-C focuses specifically on the consumption or use items of the AUDIT; it is much more commonly used for screening in clinical care because of its brevity and its ability to identify patients with at-risk alcohol use who may benefit from intervention. Even if the PROMIS instrument measures the domain defined by its items with more precision than the AUDIT-C does, if it does not identify substantial proportions of people with at-risk alcohol use it might not be suitable as a stand-alone instrument to screen for that clinically relevant behavior.
As part of the aim 2 cocalibration analyses for alcohol use, we conducted a study to better understand the measurement of at-risk alcohol use by the PROMIS Alcohol Use instrument compared with the established AUDIT-C in clinical care. First, we administered both instruments as part of one assessment at routine clinical care visits at Fenway, UAB, UCSF, and UW. Second, we used modern psychometric approaches to evaluate the dimensionality (ie, are the 2 instruments measuring the same construct?) and measurement properties of the AUDIT-C and the PROMIS SF. We used 3 different definitions of at-risk alcohol use based on AUDIT-C responses (a lower threshold, a higher threshold, and a definition of binge drinking based on the third AUDIT-C item) and determined the proportion of patients with at-risk alcohol use identified by the PROMIS SF. Third, we analyzed AUDIT-C data available from follow-up visits at least 30 days after the baseline visit with both the PROMIS SF and the AUDIT-C collected at the same time. With these data, we evaluated the ability of the PROMIS SF to predict future at-risk alcohol use based on AUDIT-C responses among those who did and did not have at-risk alcohol use at the baseline visit.
We followed established PROMIS workflows for aim 3, including the performance of several qualitative tasks. These included reviewing PROMIS items for comprehensibility, reviewing PROMIS short forms and conducting simulated computer-adaptive test (CAT) administrations to identify areas patients consider important for health care clinicians to know in the specific context of clinical care, identifying existing PROMIS items that address those content areas, developing new items (if needed), and identifying related legacy instruments.
PLWH from 5 sites—Fenway, UAB, UCSD, UCSF, and UW—were included in interviews, focus groups, and other qualitative data collection. We conducted interviews to the point of data saturation—meaning that no new codes were needed when coding interviews sequentially. We conducted interviews until 3 consecutive interviews produced no new themes.70
On the basis of the qualitative analyses, we identified a group of items to administer across CNICS, including CNICS' legacy instrument (if applicable), the PROMIS Short Form, additional PROMIS items likely to be administered using CAT techniques, additional PROMIS items with important content, and candidate new items. We conducted cognitive interviews to assess how well items were understood and whether some items were not understood and thus needed to be eliminated from the instrument. In these interviews, PLWH completed the draft instrument, “thinking aloud” as they did so about how they interpreted an item and how they chose a response. This step identified items with problematic or confusing language or formatting. We iteratively revised the items in consultation with PLWH.
In the debriefing process for the cognitive interviews, we used standard procedures developed at the National Center for Health Statistics Cognitive Survey Laboratory.71,72 Questions focused on (1) whether participants found the instrument easy or difficult, (2) whether they found problems with the wording or the response categories, and (3) their overall reaction to the instrument. We addressed the 5 relevant classes of problems specified by the Principles of the Questionnaire Evaluation Aid for Survey Methodologists73: (1) unfamiliar technical term, (2) vague or imprecise relative term, (3) vague or ambiguous noun phrase, (4) complex syntax, and (5) working memory overload.
We administered these items to PLWH at 6 sites across CNICS (Fenway, UAB, UCSD, UCSF, UNC, and UW). We conducted quantitative analyses for assessing dimensionality, determining whether PROMIS item parameters can be used for PLWH, and analyzing legacy instruments as described in aim 2. We calibrated scores using IRT.
We used those calibrations to generate relevant scores. In particular, we considered standard and augmented short forms and CATs (augmented here refers first to additional PROMIS items and second to additional PROMIS items plus candidate new items). We compared the measurement properties of each of these instruments. We analyzed the items for DIF.
For example, one of the domains for aim 3 was social support. Consistent with the grounded theory approach,74 we presented social support as a generic concept; this meant that we let patients' descriptions of their experiences and needs guide our development of the measure. We developed a pool of legacy social support items based on a literature review, then categorized items similar in content and winnowed items within each of these categories to the strongest alternatives using a qualitative item review process.75 We designed an interview guide representative of content areas, conducted concept elicitation interviews with patients (n = 32) in English and Spanish, and coded transcripts to match item pool content. We developed new items for salient unrepresented content. In focus groups in both languages (n = 23), patients ranked highly matched items. We performed cognitive interviews (n = 30) in both languages with patients on the high-priority items. The resulting measure included several dimensions of social support, including both emotional support and practical support.
Overall, we conducted the study as originally planned in the grant. Modifications were minimal; primarily, additional data collection as needed. For example, we extended quantitative data collection among women after overall targets were met to ensure sufficient numbers for analyses, collected data at an additional (eighth) site to make sure we had sufficient diversity, and conducted additional interviews among Spanish speakers to clarify the wording of intimate partner violence items. Otherwise, we conducted the study as outlined in the grant proposal, and we met or exceeded all our enrollment targets.
This project had 3 main areas that corresponded to our 3 aims: (aim 1) implementing and selecting domains and the priorities for PROs; (aim 2) conducting quantitative analyses, particularly those related to instrument cocalibration, with emphasis on alcohol use as the domain with the findings most relevant for clinical care; and (aim 3) developing instruments.
Our results from aim 1 focus on PRO implementation and domain priorities.76-78 Strong evidence suggests that PROs aid in managing chronic conditions, reducing omissions in care, and improving patient–clinician communication. However, clinician adoption of PROs and their use in clinical HIV care are not well known.
We interviewed 27 clinicians from 4 geographically diverse HIV and community care clinics in the Centers for AIDS Research CNICS. These clinics (Fenway, UAB, UCSD, and UW) have integrated PROs into routine HIV care. We queried these respondents regarding perceived value, challenges, and use of PRO data.
During these interviews, clinicians identified perceived benefits, including the ability of PROs to identify less-observable behaviors and conditions, especially suicidal ideation, depression, and substance use. Clinicians also said that PROs might be useful in setting agendas before visits and reducing social desirability bias in patient–clinician communication. Challenges included integrating these activities into the existing workflow of the clinics and facilitating the interpretation of PRO feedback.
Health care clinicians valued same-day, electronic patient-reported measures for use in clinical HIV care; however, they noted some caveats about PROs. They wanted PROs that are (1) tailored to be the most clinically relevant to their patient population; (2) well integrated into clinic flow; and (3) easy to interpret, highlighting chief patient concerns and changes over time.
We sought to understand how PLWH, HIV health care clinicians, and HIV researchers set priorities for self-reported domains of clinical care. Participants rank-ordered 25 domains (see Appendix Table A.1). PLWH and clinicians showed high discordance in rank order priorities (P = .001); for example, PLWH ranked domains related to social context (eg, HIV stigma, social support) high, whereas clinicians were more likely to rank drug, alcohol, and tobacco use high (Table 2).
Highest-Priority and Lowest-Priority Domains, Ranked by Patients and HIV Health Care Clinicians (Mean Ranking Score).
Among patients, rank-ordering differed significantly between men and women (P = .02); between Hispanic and White patients (P = .004); and between those who were poorly engaged and those who were well engaged in care (P = .02). HIV stigma was ranked high by patients who were poorly engaged in their HIV care and by Hispanic patients. Women and Hispanic PLWH ranked social support higher than other groups.
Interviews with 82 patients revealed concern that negative social context adversely affects other health domains, including depression and self-medicating behavior. Many patients believed that root social causes must be addressed to improve depression and reduce substance use and risk behaviors. Chronic pain was also identified as a root cause of mood-based symptoms, substance use, and risk behaviors.
In partnership with our patient advisory board, we selected intimate partner violence and social support as areas for further development for aim 3.
We noted a disparity between the high rates of intimate partner violence observed among PLWH79-81 and patients' low importance ranking for intimate partner violence. We explored this further and found that PLWH typically thought that intimate partner violence referred exclusively to physical injury severe enough to warrant going directly to an emergency department and had a hard time visualizing the necessity of an intimate partner violence assessment in the context of routine care. Many patients simply believed that it was a rare problem, which is clearly not the case.
Given the potential injury/lethality of intimate partner violence, its high prevalence, and the known negative virologic outcomes for PLWH,82 we brought the disparity to the attention of our patient advisory board. Its members advised us to pursue intimate partner violence as a domain. Other stakeholders—including clinicians, clinic leadership, and clinic social workers— also considered this domain to be a high priority.
Social support and its proxies (eg, social isolation, social roles, HIV-related stigma) ranked quite high among PLWH. In reflecting on social support, PLWH referenced these other domains. They spoke of the effect of social support on other highly ranked domains, such as depression, and on health behaviors in general. Patients strongly emphasized its importance in the qualitative interviews, often repeatedly returning to social themes during a single interview. For these reasons and because of its high ranking, the patient advisory board agreed that social support is a very important domain.
Effect of PRO feedback. For all same-day visits, we compared HIV clinician medical record information on (1) documentation or awareness and (2) interventions or actions. We considered these 2 distinct outcomes from the 8-month period before clinicians received PRO feedback and the period during which they received point-of-care PRO feedback. During the feedback period, clinicians received same-day results (a paper-based feedback form from the patient's PRO assessment) as they entered the examination room, along with other relevant information, such as the patient's vital signs and medication list.
We were specifically interested in clinician documentation in the EHR (ie, awareness) and interventions (ie, actions) for patients who self-reported clinically relevant levels of depression or risk behaviors. We conducted this study at 1 site: UW. During the study period, PLWH completed 2289 PRO assessments. As shown in Figure 1, comparing the 8 months before clinicians received PRO feedback with the period during which they received feedback, we could see that clinicians were more likely to document certain behaviors in the latter period: depression (74%; 95% CI, 62%-85% before vs 87%; 95% CI, 82%-91% during feedback; P = .02) in patients with moderate to severe depression (317 assessments); at-risk alcohol use (41%; 95% CI, 20%-61% vs 64%; 95% CI, 56%-72%; P = .04, 155 assessments); and substance use (60%; 95% CI, 47%-73% vs 80%; 95% CI, 73%-86%; P = .004, 212 assessments). During the period after feedback began, clinicians were less likely to incorrectly document good adherence to treatment among patients who in fact had inadequate adherence (42% vs 24%, P = .02, 205 assessments).
Clinician Documentation or Awareness in the 8 Months Before Initiation of PRO Feedback vs During Feedback for Patients With At-Risk Symptoms and Behaviors.
PRO feedback about depression and adherence were followed by increased clinician intervention, but increased intervention was not statistically significant for other domains (eg, sexual risk behavior and drug use) (Figure 2).
Clinician Action in the 8 Months Before Initiation of PRO Feedback vs During Feedback for Patients With At-Risk Symptoms and Behaviors.
Our results from aim 2 focus on quantitative analyses, especially as related to instrument cocalibration.83 We conducted cocalibration analyses for depression, alcohol use, intimate partner violence, global health, and social support measures. In the following sections we describe 2 of the analyses. The alcohol analysis had important clinical implications, because previous studies have demonstrated a high prevalence of at-risk alcohol use by PLWH, and providers had noted that they often miss at-risk alcohol use in their patients.25,84 Because we developed a social support instrument as part of aim 3, we also describe social support as an example of a traditional cocalibration analysis.
Aim 2 example: alcohol use. Across 4 CNICS sites (Fenway, UAB, UCSD, and UW), we administered AUDIT-C and PROMIS SF items to 2497 PLWH who reported at least 1 drink in the past 12 months. The mean age was 43.7 years (SD, 10.9; range, 20-80 years); we included 2199 men (88%) and 298 women (12%). Regarding race and ethnicity, 1354 people described themselves as non-Hispanic White (54%), 724 as non-Hispanic Black (29%), 306 as Hispanic (12%), and 113 as some other description or unknown (5%).
The PROMIS SF and AUDIT-C items were sufficiently unidimensional that we could use IRT methods: The AUDIT-C items could be considered indicators of the domain defined by the PROMIS SF items. We plotted SEs of measurement against observed scores on the PROMIS metric for the 2 instruments. In our data, the range of the AUDIT-C items extended below the range of the PROMIS SF items, indicating that the AUDIT-C had less of a floor effect. For PROMIS SF scores above the floor, PROMIS had better measurement precision than the AUDIT-C.
We were specifically interested in clinically relevant at-risk alcohol use as defined by the AUDIT-C and comparisons with PROMIS scores. In this sample, 1136 people reported at-risk alcohol use as defined by a lower AUDIT-C threshold (45%). In all, 731 individuals met a higher AUDIT-C threshold (29%) and 21% met criteria for binge drinking on the basis of their responses to AUDIT-C item 3. In total, 1145 people (46%) met either the lower threshold or the binge criterion, and 763 (31%) met either the higher threshold or the binge criterion.
People who met at-risk alcohol use thresholds were found at every level of PROMIS SF scores (Figure 3). Among this study population (all of whom reported at least some current alcohol use on the AUDIT-C) 60% endorsed the very lowest category (the “never” response) for all the PROMIS SF items and thus had a score at the floor (labeled Min in Figure 3). Of these, 26% had at-risk problem drinking by at least 1 definition on the AUDIT-C.
Frequency of PLWH Who Met At-Risk Alcohol Use Thresholds by AUDIT-C–Based Criteria at Different Levels of Alcohol Use Defined by PROMIS Alcohol Use SF Scores.
One might consider PROMIS Alcohol Use SF items as assessing both a level of concern about drinking among people with alcohol use and the level of consequences they have experienced. Among people with at least one definition of at-risk alcohol use according to the AUDIT-C, a small minority of people with at-risk alcohol use had the very highest levels of concern about their drinking (Figure 4).
Distribution of PROMIS Alcohol Use SF Scores for People With At-Risk Alcohol Use According to at Least 1 Definition Based on the AUDIT-C Criteria.
We included 1608 individuals, with a subsequent visit at least 30 days after the baseline visit. We categorized people on the basis of at-risk baseline AUDIT-C scores (defined using either the higher total score threshold or the binge threshold) and at-risk baseline PROMIS scores (defined as a score > 1 SD above the national norms). In the group of people who were not at risk according to the AUDIT-C alone, 1057 were not at risk according to PROMIS, but 100 were at risk according to PROMIS.
We then examined whether there was predictive value to PROMIS information on the basis of subsequent at-risk AUDIT-C scores. The sensitivity of PROMIS for identifying the 108 people who would develop at-risk alcohol use at follow-up was only 0.22. The specificity was somewhat better at 0.93, but the positive predictive value was only 0.24. These data do not suggest great value in including the PROMIS instrument for people who did not have at-risk drinking according to the AUDIT-C, as more than three-fourths of those who went on to have at-risk AUDIT-C scores would not be identified by PROMIS (low sensitivity), and three-fourths of the people identified by PROMIS would not go on to have at-risk AUDIT-C scores (low positive predictive value).
We then considered the group of 451 PLWH who were at risk according to the AUDIT-C alone, of whom 236 were not at risk according to PROMIS and 215 were at risk according to PROMIS. We examined whether there was predictive value to PROMIS for these PLWH, using subsequent at-risk AUDIT-C scores. The sensitivity of PROMIS for identifying PLWH who would continue to have at-risk alcohol use at follow-up was 0.55; specificity was 0.66. The predictive power of a positive test was 0.74, and the predictive power of a negative test was 0.46.
The results of this study demonstrated that the PROMIS SF items were insufficiently sensitive to identify people with at-risk alcohol use as defined by the AUDIT-C. Indeed, 60% of all PLWH who reported any drinking over the past year responded “never” for all the PROMIS items; of these respondents, 17% to 26% had at-risk drinking as defined by the 2 ways of classifying AUDIT-C responses. We also evaluated the ability of the PROMIS score to identify people who would initiate or continue at-risk drinking at follow-up. The sensitivity was very low (0.24) to identify people who would initiate at-risk drinking and low (0.55) among those who would continue at-risk drinking.
The AUDIT-C assesses quantities of alcohol use on the basis of item content, regardless of how the individual feels about that quantity. By contrast, the PROMIS Alcohol Use SF items assess respondents' attitude about their alcohol use. Although the PROMIS alcohol use item bank includes items that capture consumption itself, those items are not included on the SF. Our results suggest that a substantial proportion of people whose quantity or pattern of alcohol consumption as reported on the AUDIT-C places them in an at-risk category do not self-identify their drinking as a problem.
The PROMIS SF had one advantage over the AUDIT-C: Its measurement precision was better in the region of scores above the floor of the instrument. The PROMIS instrument could play a role in determining which patients with at-risk drinking have a moderate or severe alcohol use disorder and thus need more intensive treatment or medication rather than a brief intervention. The PROMIS instrument might also be more sensitive to change over time for patients with more severe at-risk alcohol use, whether they are being managed in primary care or in specialty addiction treatment settings.
However, because of the PROMIS SF's insensitivity to identifying at-risk use, it cannot be substituted for the AUDIT-C for use in clinical care settings to identify at-risk alcohol use. Furthermore, in our analyses of follow-up AUDIT-C data, PROMIS SF scores did not reliably identify sufficiently large proportions of people who would continue to have at-risk alcohol use (sensitivity, 0.55) and especially people who would develop at-risk alcohol use by the time of the subsequent visit (sensitivity, 0.24). We do not have data to comment on the ability of the PROMIS SF scores to track progress in alcohol treatment programs or to identify people who are especially likely to enroll in treatment. Although the PROMIS alcohol measure might be especially valuable in these settings, we are not aware of evidence to support the clinical utility of using PROMIS in these ways.
Our results from aim 3 focus on measurement development. We chose intimate partner violence and social support as the 2 domains of interest, after taking into account patient and clinician input from aim 1 and substantial input and discussion from our patient advisory board.
With the assistance of a reference librarian, we performed an in-depth literature review of validated patient-reported intimate partner violence measures. The review yielded 13 measures, comprising 74 items.
We divided the item pool content into dimensions such as psychological, physical, and sexual intimate partner violence. Two coders open-coded the content in these dimensions and created “bins” for similar content within each. After binning, the coders performed a qualitative item review75 that winnowed the pool from 74 items to 14, representing unique concepts across dimensions of physical, sexual, and psychological violence.
Following the advice of our patient advisory board, we bypassed concept elicitation interviews for this domain. Board members were concerned that such interviews would cause unnecessary duress for victims of intimate partner violence and would be unlikely to enhance the existing rich item pool content. We deemed cognitive interviewing of the 14 items with PLWH to be sufficient for developing the measure.
We performed cognitive interviews with patients on candidate items in English and Spanish. All items pertaining to physical and sexual violence were well understood. Some concepts pertaining to psychological violence lacked specificity and were ultimately dropped; for example, feeling that a partner controls one's money was not universally perceived as either negative or an indicator of abuse. Although the concept of walking on eggshells in a relationship is widely understood in English, the expression has no Spanish equivalent, so the term was changed in both languages to being made to feel afraid. In addition, the Spanish-speaking patients considered terms such as hit, slap, and punch to be synonymous, so we abbreviated the Spanish version of the physical impact item.
Overall, we reduced the number of items to 4 in both languages, representing 4 key dimensions of intimate partner violence: (1) feeling controlled in an unwanted way; (2) being made to feel afraid of harm; (3) feeling pressured or forced to engage in sexual activity; and (4) experiencing nonsexual physical harm (see Table 3).
Intimate Partner Violence: Final 4 Items.
Of the first 1000 PLWH completing the intimate partner violence assessment in our clinics, the percentage of patients who reported feeling controlled in the past year was 9%; being afraid of harm, 5%; being pressured or forced to engage in sexual activity, 3%: and experiencing nonsexual physical harm, 4%.
Our development of the intimate partner violence items led to changes in clinic procedures across CNICS sites. On the same day that a patient reports violence in the PRO assessment, his or her clinician is informed via a feedback delivery mechanism just before seeing the patient. In addition, when a patient reports either physical or sexual violence, social workers receive a pager-based alert, and they evaluate patient needs and safety during the same visit. Because of this study, about 9000 assessments (including intimate partner violence items) have been completed as part of clinical care in CNICS.
With the assistance of a reference librarian, we performed a literature review of validated patient-reported measures of social support that yielded 31 measures and 324 items. We placed the item pool content into several dimensions (eg, belonging or inclusion, communication, emotional support, practical support, and whole-person acceptance). Two coders open-coded the content in these dimensions and created bins for similar content within each of them. The coders performed a qualitative item review75 that winnowed the pool from 324 items to 88, representing unique concepts within each dimension.
We used the dimensions to inform content development for a concept elicitation interview guide. We conducted 32 patient interviews in English or Spanish. The Spanish interviews were transcribed into English, and 2 coders matched excerpts from interviews to item content, reconciling differences through consensus discussion. The excerpts were matched to 82 of the 88 items in the pool; in addition, we identified 5 new concepts. We assessed the matched and new items for prevalence, eliminating items that matched to 2 or fewer patients. This reduced the item pool to 62 items—too many to rank-order or discuss in a focus group.
To further reduce the large number of items, we performed a literature review to identify aspects of social support that are most closely linked to health outcomes. On the basis of this review, we narrowed our focus to include emotional support–related items that measure the patient perception of support and excluded items that measure actual support received. This reduced the number of items to 14.
In focus groups, 24 patients ranked the 14 items in order of importance. The top items included “How much do you feel loved, liked, or cared about?” and “How much do you feel you can trust those in your personal life?” Key themes discussed included the importance of having peers who support one's ability to stay healthy (eg, supporting sobriety) and the importance of being accepted for who we are. Patients suggested removing some lower-priority items that seemed vague, confusing, or irrelevant, such as an item that asked if the person felt as though he or she belongs to a community. After removing some of the lower-priority items and concepts, we performed cognitive interviews with 30 patients on the 9 remaining items; all but 1 of these items was well understood, which resulted in an 8-item measure (Table 4). We integrated this measure into the CNICS PRO assessment for PLWH at CNICS sites across the United States.
Social Support: Final 8 Items.
We administered our newly developed social support instrument to 680 patients. In addition, we administered a legacy instrument (the Berlin Social Support Scale [BSSS]) and the PROMIS Emotional Support Short Form. We found the newly developed social support instrument to be sufficiently unidimensional to represent a single construct well, with a median standard error of measurement of 0.70 (0.18-0.79). It was highly correlated with the BSSS (Ρ = .72) and the PROMIS measure (Ρ = .69).
Low levels of social support on the newly developed instrument correlated with higher levels of depression on the PHQ-9 (Ρ = −0.49), worse HRQOL according to the EQ-5D (https://euroqol.org/eq-5d-instruments/) from the EuroQol (Ρ = −0.27),62-64 lower 30-day self-rated adherence (ρ = 0.18), higher likelihood of having used methamphetamines (τ = −0.20) or street opiates (τ = −0.25) in the past 3 months, and higher levels of self-reported HIV-related stigma (τ = −0.22-0.33). Levels of social support did not correlate with the use of cocaine in the past 3 months. The correlations found with the newly developed social support instrument and other domains were similar to those found with legacy instruments and other domains. Although levels of social support did not differ by sex, patients who cited bisexual or other as their sexual orientation reported worse levels of social support than those who identified as heterosexual, lesbian, or gay (P = .02). The legacy instruments could not detect this distinction.
PRO data can be useful for patients who are making decisions about their care. The activities in aim 1 helped us prioritize the domains that PLWH and clinicians consider most important for clinical care; these domains might be especially useful for patients in making health decisions. The specific population for this research is PLWH. Study results inform future health decisions by facilitating optimized measurement of highly valued PRO domains. This work will facilitate data collection from large numbers of PLWH, and their data will be useful for patient-centered outcomes research, which in turn will ultimately enable patients and clinicians to make better data-driven and evidence-based clinical decisions.
Item banks in PROMIS have much to recommend them, including careful attention to intellectual property; generic rather than disease-specific item content (eg, “I feel tired” rather than “I feel tired because of my cancer”); common item formats; common response formats; use of modern psychometric theory for calibration; generation of short forms and CATs to facilitate brief assessments; and calibration to national norms. However, much of the PROMIS instrument development has been in the context of research studies. More needs to be learned about how these instruments work in the specific context of health care.
As an example of the relevance of the clinical care setting, our findings for the alcohol use domain suggested that items from the PROMIS SF resulted in more precise measurements than the AUDIT-C. However, the AUDIT-C has long been used to identify high-risk alcohol use. This construct has tremendous clinical relevance, and the PROMIS SF does not elicit it. Our findings provide strong evidence that the PROMIS measure cannot replace the AUDIT-C in the context of routine clinical care, because clinically important patterns of alcohol use can be detected by the AUDIT-C but not the PROMIS measure. In the CNICS cohort, we have based our decision to collect the AUDIT-C on the results of this study.
We conducted similar cocalibration analyses for other domains, such as global health. For this report, however, we highlighted the alcohol analyses because these findings had the greatest relevance for clinical care.
Critically, the diversity of our patients is not reflected in the diversity of those who were used to develop PROMIS items or to derive item calibrations for most PROMIS domains. We can and should do a better job of ensuring that PROs reflect the voices of all patients.
This study fits in the larger context of the literature to date by addressing (1) gaps specifically related to those PRO domains, including PROMIS domains, that PLWH and other stakeholders value most highly; (2) psychometric relationships between legacy instruments and PROMIS domains; and (3) the usefulness of PROMIS domains in the context of care, including understandability, psychometric characteristics, and construct validity among PLWH.
This study's results have tremendous potential to be implemented in clinical care settings. The overall aim of the project was to improve the implementation and measurement of PROs in clinical care of PLWH to improve both care and clinically relevant patient-centered outcomes research.
In aim 1, much of the work on PRO implementation translated directly into improvements in the approach to PRO collection across CNICS sites; this included adding the Amharic language. An overarching goal of this aim (and of all of our work to date) has been to improve PRO collection in CNICS to enhance clinical care. In 2016, PLWH in CNICS completed 9955 PRO assessments as part of routine clinical care visits.
The aim 2 findings that we value most are those that influence clinical care. The alcohol analyses are the most relevant example. They drove our decision not to switch from AUDIT-C to PROMIS-SF in the CNICS assessment, which is completed almost 10 000 times a year by PLWH in clinical care across CNICS sites.
The work in aim 3 resulted in the development of brief, clinically relevant measures of intimate partner violence (4 items) and social support (8 items). Using the intimate partner violence measure as an example of the development process, we designed it to be responsive to stakeholder priorities, including those of case managers and clinicians in HIV clinical care settings. The measure asks patients about feeling controlled in an unwanted way, being made to feel afraid of harm, being pressured or forced to engage in sexual activity, and experiencing nonsexual physical harm. This measure also addresses limitations in the existing instruments, including that they are often too long for clinical care settings and that they make assumptions about the directionality of violence (ie, that it is always men on women, rather than possibly men on men or reciprocal violence between 2 men).
Initial collection and validation work on the intimate partner violence measure was conducted under the auspices of this project. However, the measure's value was immediately clear to CNICS investigators. Thus, although data collection for this project is complete, this 4-item measure has remained in the CNICS assessment and is a routine part of clinical care across CNICS sites. We expected that it would be completed more than 9000 times in 2017, and current efforts are focused on appropriate clinical responses to patients who report intimate partner violence.
The various studies presented here evaluated only PLWH. We believe generalizing to other settings or patient populations would be inappropriate without further investigation. Nevertheless, we suspect that many of the findings likely apply to other populations; for example, the psychometric limitations of the PROMIS alcohol measure might limit its effectiveness as a screening measure in other clinical care settings as well.
Subpopulation considerations are particularly relevant for this project. Critically, the diversity of our patients is not reflected in the diversity of those used to develop PROMIS items or to derive item calibrations for most PROMIS domains. We need to be sure that PROs reflect the voices for whom patient–clinician discordance has been shown to be highest; this is particularly important for African American patients and those recently diagnosed with HIV.
Subgroups of interest for our project included patients living in rural areas, women, transgender women, African Americans, monolingual Spanish speakers, English-speaking Hispanic PLWH, patients with multiple comorbidities, and patients aged 55 years or older. Our previous research had found that recently diagnosed (<5 years) PLWH had considerable discordance with clinicians on domain importance; for that reason, we also oversampled this group of relatively newly diagnosed patients. In addition, PLWH who were poorly engaged in HIV care were of interest. Women were more likely to rank social support and stigma as important; men ranked physical function higher. Hispanic and African American PLWH were both much more likely than White PLWH to rank social support and stigma as more important. Additionally, in comparisons between patients who were poorly engaged in HIV care and those who were well engaged, the former were much more likely to rank HIV stigma as more important than the latter. These examples demonstrate the importance of careful consideration and focus on key subgroups of interest, especially insofar as some of these groups have different priorities.
A key factor that enabled us to include adequate representation of subgroups is that we frequently enrolled participants beyond our initial preliminary targets. For example, for the aim 2 alcohol analyses, our initial target was at least 500 PLWH who reported at least 1 drink in the past 12 months; however, we ended up including 2497 PLWH. Only 12% of the 2497 were women. This percentage is consistent with the lower rates of alcohol use among women with HIV than men and with demographic data about the HIV epidemic in the United States, which consists of a much higher percentage of men than women. Because we enrolled so many PLWH, we were able to have adequate representation of women and other key groups despite their low overall percentage in the group.
Limitations varied by individual study within this project; for example, several limitations related specifically to the alcohol analyses. We evaluated only PLWH; therefore, generalizability to other populations and settings is uncertain. Nevertheless, alcohol use is common among PLWH, making this a particularly relevant group to study, and we have no evidence that the results from PLWH do not apply in other settings.
In addition, although the CNICS assessment has been expanded to include additional languages, such as Amharic, the alcohol analyses included only English-speaking and Spanish-speaking PLWH. We suspect that the results would be similar in other languages, but we do not have data to support this assumption.
Moreover, alcohol use was patient reported as part of the clinical assessment, and this practice could lead to underestimates of risk behavior. On the other hand, electronic PRO collection might reduce social desirability bias relative to a face-to-face interview with a clinician.85 We were concerned that time frame differences between the AUDIT-C and the PROMIS Alcohol Use SF could affect our findings, but sensitivity analyses limited to people who had indicated that they drank 2 or more times a month produced similar results. Follow-up data were available for a sizable proportion of our cohort, but some people did not have a follow-up visit in the time frame we considered for this project. It is unlikely but possible that additional follow-up data might have changed our conclusions regarding the ability of the PROMIS Alcohol Use SF to identify people who would continue to have or who would develop at-risk alcohol use by the time of their next clinical visit.
Limitations of specific analyses differed for each of the 3 aims; however, the overall limitations of this project include the fact that it was focused on PLWH in clinical care. This orientation is in many ways a key strength of these studies, but it does limit generalizability. The study included those who were well engaged in HIV care and specifically overenrolled those who were poorly engaged in HIV care. It included many PLWH in clinical care, but this might not represent individuals in care with other or no chronic conditions.
Areas of future research differ by specific PRO domain. For example, our next planned steps pertaining to social support are to measure the impact of low social support in our population and to analyze its associations with other health outcomes. Depending on the findings, we might explore the development of interventions to enhance social support for PLWH.
As another example, we have integrated the intimate partner violence measure into the CNICS clinical assessment as a screening tool; it is now completed more than 9000 times a year by PLWH across CNICS. We plan to use our intimate partner violence data as a basis for a better understanding of the prevalence of and associations with intimate partner violence in our population. Most important, we plan to use these data to design and test interventions to address intimate partner violence among PLWH.
Studies integrated into clinical care are the most important focus for future research in this area. By focusing on clinically relevant PROs integrated into care, this kind of research can have large positive effects on the clinical care of PLWH and other patients and on the ability to conduct high-quality, relevant research that leads to better outcomes. We found that PRO feedback for some domains—such as depression and inadequate medication adherence—led to increased clinician awareness and intervention, but this was not the case for every domain. Further investigation of factors associated with the gap between clinician awareness of PRO domains such as depression and clinician interventions is needed to bridge this divide.
This study evaluated PRO domain priorities, examined measurement properties and cocalibrated key PROMIS and legacy PRO domains, and focused on the development and integration of PRO domains for clinical care. Given the wide approach to PROs in clinical care for PLWH addressed in this study, it is not surprising that we reached a wide variety of relevant conclusions.
The study resulted in key findings related to PRO domain priorities, and we were able to draw clear conclusions about the importance of including a wide variety of stakeholders, given the substantial differences we found between patients and clinicians. This work also highlighted the importance of the careful inclusion of subgroups; of particular importance were the relevant differences between men and women and between patients who are less engaged vs more engaged in HIV care. PLWH expressed a desire for clinicians to demonstrate familiarity with the impact of HIV on the social domain and on the broader context of their lives.
We can emphasize several conclusions from our measurement work. For example, the consumption version of the AUDIT-C is well established as a clinically relevant threshold; because a high proportion of PLWH with at-risk alcohol use were not detected with the PROMIS Alcohol Use measure, we cannot advocate that it should replace the AUDIT-C in clinical settings for screening. Furthermore, the PROMIS Alcohol Use measure does not reliably identify large proportions of people who will continue to have at-risk alcohol use and especially does not identify large proportions of those who will develop at-risk alcohol use by the time of their next clinical visit. These results highlight the importance of assessing instruments according to the purpose of their use (eg, screening vs monitoring) and in the specific context of their planned use (eg, routine clinical care).
We do acknowledge, however, that PROMIS measures might appropriately become the gold standard in many instances, because of the general population norms and other favorable measurement properties. Cocalibration of legacy measures with PROMIS measures provides the opportunity to compare and combine data across different sites and studies that used different measures.
Finally, developing, testing, and implementing an intimate partner violence measure into clinical care for PLWH provided several insights, including the high prevalence rates (using within 1 year as the time frame) and the differences in rates depending on the type of intimate partner violence examined (eg, sexual violence vs nonsexual physical violence). Information about intimate partner violence is valued by clinicians and other stakeholders, such as clinic social workers, and it can be integrated into clinical care without negative consequences in terms of patient comfort, time burden, or effect on clinic flow. In the past, CNICS clinics have differed in their approaches to addressing intimate partner violence. We look forward to the next steps, which will include evaluating, comparing, and expanding these approaches.
We thank the patients, providers, and staff across all CNICS sites.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#ME-1403-14081) Further information available at: https://www.pcori.org/research-results/2014/creating-survey-questions-measure-important-aspects-health-people-living-hiv
Rank Order Cards With Definitions (PDF, 74K)
Crane HM, Fredericksen R, Crane PK. Creating Survey Questions to Measure Important Aspects of Health for People Living with HIV. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/8.2019.ME.140314081
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
