Critical Appraisal of Included Publications

APPENDIX 3Critical Appraisal of Included Publications

Publication Details

Table A3

Strengths and Limitations of Randomized Controlled Trials using Downs and Black.

Table A4

Strengths and Limitations of Observational Studies using Downs and Black.

Table A5

Strengths and Limitations of the Included Qualitative Study.

Table A6

Strengths and Limitations of Economic Studies using Drummond.

Publication Details

Copyright

Copyright: This report contains CADTH copyright material and may contain material in which a third party owns copyright. This report may be used for the purposes of research or private study only. It may not be copied, posted on a web site, redistributed by email or stored on an electronic system without the prior written permission of CADTH or applicable copyright owner.

Links: This report may contain links to other information available on the websites of third parties on the Internet. CADTH does not have control over the content of such sites. Use of third party sites is governed by the owners’ own terms and conditions.

Except where otherwise noted, this work is distributed under the terms of a Creative Commons Attribution-NonCommercial- NoDerivatives 4.0 International licence (CC BY-NC-ND), a copy of which is available at http://creativecommons.org/licenses/by-nc-nd/4.0/

Publisher

Canadian Agency for Drugs and Technologies in Health, Ottawa (ON)

NLM Citation

Tamper-Resistant Oxycodone: A Review of the Clinical Evidence and Cost-effectiveness [Internet]. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health; 2015 Jun 25. APPENDIX 3, Critical Appraisal of Included Publications.

Table A3Strengths and Limitations of Randomized Controlled Trials using Downs and Black¹⁰

Strengths	Limitations
Colucci¹²
Objectives, interventions, patient characteristics clearly described. Exact P-values reported Subjects were blinded to what injection they were receiving. Study described as double blind – likely the outcome assessors (not those running statistical tests) were blinded. However, part of the inclusion protocol required patients to be able to identify the difference between OXY and placebo. Important treatment emergent adverse events reported Compliance reliable	Unclear if industry sponsorship had an effect on the results Population likely not representative of those with substance abuse disorders. Power calculation not reported; unclear if the study had sufficient statistical power to detect differences. Disposition of patients not clearly described, nor were principle confounders. Likely that men were overrepresented in the study (men are more often abusers of prescription opioids,¹ but unclear if 87% of abusers are men) Supervised use not representative of real-world use. Intended OXY/naloxone ratio was 2:1, however this may not be representative of a real-world setting due to tablet crushing and dissolving.
Harris¹³
Study subjects were blinded, however, part of the inclusion protocol required patients to be able to identify the difference between OXY and placebo. Important treatment emergent adverse events reported. Objectives, interventions, patient characteristics clearly described. Actual P-values reported Compliance reliable	Population likely not representative of those with substance abuse disorders. Supervised use not representative of real-world use. Power calculation not reported, unclear if the study had sufficient statistical power to detect differences. Likely that men were overrepresented in the study (men are more often abusers of prescription opioids,¹ but unclear if 87% of abusers are men)
Webster⁹
Objectives, interventions, patient characteristics clearly described Adequate washout period between treatments P-values adjusted for multiple comparisons using the Benjamini-Hochberg method. Reasons for discontinuation clearly described. Important treatment emergent adverse events reported Outcome measures were valid and appropriate	Power calculation not reported – authors state that the study is well-powered but do not present their calculation. Likely that men were overrepresented in the study (men are more often abusers of prescription opioids,¹ but unclear if 78% are men) While the outcome measures are valid and appropriate for research, they are surrogates for predicting the behavior of those abusing the drugs. Population likely not representative of those with substance abuse disorders. Supervised use not representative of real-world use, protocol required fasting – it is unclear if ‘full’ participants would have the same aversion to niacin (as a high fat meal is known to reduce the negative effect of niacin)
Webster¹⁴
Objectives, interventions, participants clearly described Participants were blinded – interventions were made to be identical. Primary investigator and research staff were also blinded – pharmacist was not. Power calculation presented – 88% power to detect a one half SD difference between interventions.	Population likely not representative of those with substance abuse disorders. Included males only. Supervised use not representative of real-world use, protocol required fasting – unclear if that would be important in real-world use. Actual P-values not reported (just P<0.05 or P≤0.001) Unclear if study was powered for secondary outcomes or for safety.
Setnik¹⁵
Objectives, interventions, participants clearly described. Washout periods likely adequate Power calculation presented Adjustment for multiple comparisons using Benjamini-Hochberg method. Losses to follow-up and withdrawals mostly described. Exact P-values reported. Participants were blinded, assessors likely blinded (unclear which group was blinded other than the participants) Drugs were administered under both ‘fasting’ and ‘full’ conditions.	Population likely not representative of those with substance abuse disorders. Men likely overrepresented Supervised use not representative of real-world use

Table A4Strengths and Limitations of Observational Studies using Downs and Black¹⁰

Strengths	Limitations
Cicero¹⁶
Objectives clearly described The large SKIP sample was likely reasonably representative of a study of opioid abusers, drug use representative of real-life abuse rather than controlled clinical conditions. Data was from a large database sample. Basic patient characteristics described	The smaller RAPID sample (more detailed questionnaire of 244 people) may not be representative of the full population as they self-selected into answering the survey. Exact interventions (dosing etc.) not described. Descriptive study – therefore, cannot determine causality. Unclear if the results generalize to a non-treatment seeking population
Degenhardt⁸
Multiple data sources used. Objectives, main findings clearly described.	Descriptive observational study – cannot determine causality – study was hypothesis generating. Those who visit safe injection sites or needle exchange programs may not be representative of all drug users. Time period examined may not be lengthy enough to determine long-term changes in use
Larochelle¹⁷
Patients were able to enter and exit the cohort over the 10 year period. Objectives, main findings, patient population clearly described. 95% confidence intervals presented	Insured patients represent only some of the patients taking non-medically necessary opioids. Not randomized.
Cassidy¹⁸
Methods clearly described. Outcomes were clearly described and defined Model adjusted for potential covariates Steps taken to minimize geographical, and other covariate confounding	Estimates using logistic regression Those seeking treatment may not be representative of the entire drug using population Power calculation not reported, however there was a large sample size so power was not likely an issue.
Havens¹⁹
Authors explored factors related to the time of the interview and whether that would result in confounding bias. Single interviewer was used so if events were misclassified, they were likely consistently misclassified. Self-reported drug use is considered a valid measure of drug use; 30 day time frame was likely short enough to reduce problems with recall.	Recall bias a potential issue due to being a self-reported survey – used anchoring to attempt to reduce recall bias. Population being studied lived in an area where opioid abuse was considered epidemic – may not be generalizable to other populations.
Michna⁷
Objectives, methods, outcomes clearly described. Classification of exposures well described. Sensitivity analyses performed in order to control for potential confounders (time, primary opioid, coding of abuse)	Commercially-insured continuous users may not be representative of the broader oxycodone-using population. Observational study – direct causation difficult to determine.
Rossiter²⁰
Objectives, measures, interventions clearly described. Same inclusion and exclusion criteria applied to all included patients Pre- post- formulation change acted as a way to match patients; population-based study allowed for analyzing a large portion of users; pre-post- acted as a quasi-experimental design. Authors allowed for a washout period. Cases and controls well-matched Outcomes unlikely to be misclassified.	No randomization. Assumed that continuous and non-continuous users were the same. Non-experimental design – cannot assume causation. Number of people abusing or misusing opioids likely underrepresented.
Sessler²¹
Sensitivity analyses conducted ( included prescription numbers, reporter type or source, formulation specificity, missing date of death, and reporting time lag) Authors examined non-fatal reports and comparator reports and determined that there was no temporal bias. Outcomes were validated, definitions of outcomes well reported.	No randomization Possibility that not all fatalities associated with oxycodone use were coded as such. Voluntary reports of adverse events do not capture all events. (However, there was a large magnitude of change, and reporting of other adverse events did not decrease, therefore, it is more likely that the change seen was a real change and not a change in the way events were reported) Longer-term follow-up may be needed to capture more permanent changes in death rates.
Butler²²
Sensitivity analyses conducted using a national database. Objectives, measures, main findings clearly described Selection bias unlikely to account for changes in oxycodone but not other opioids. Data was collected uniformly across time and collection sites. Steps were taken to reduce reporting bias (pictures of the different formulations were shown to patients in order to accurately identify which drug was taken) Confidence intervals, exact P-values reported. Steps taken to minimize geographical, and other covariate confounding.	Patients being assessed for substance abuse may be different from other oxycodone users Results may not be generalizable to the general population of prescription drug abusers. Relied on self-reported data. Not randomized May not reflect patterns of abuse of the new formulation after more time has passed.
Coplan²³
Objectives, methods, main findings clearly described. Exposure classifications well described Confidence intervals and exact P-values reported	Possible that changes in poison centre reporting, misclassification of original or ADF oxycodone occurred during the study period Unclear if there was a representative sample, however, it was a large national sample.
Severston²⁴
Large percentage of the US population was part of the surveillance. Therefore the results are likely generalizable to the US population. 95% confidence intervals and exact P-values reported. Demographic characteristics (such as gender, ethnicity, and age) of the source population did not change throughout the study.	Is a descriptive study, therefore it is difficult to determine causation. Depended on voluntary reporting and therefore it is likely that not all cases were reported. Misclassification of product exposure was possible. Other interventions (such as prescription drug monitoring programs) may gave accounted for some of the reductions in abuse.
Butler²⁵
Objectives, methods, main findings clearly described. Exact P-values reported Population likely fairly representative those seeking treatment.	Self-reported data – recall bias possible, however, patients entering treatment tend to report fairly accurately. May not generalize beyond patients seeking treatment and who have access to a treatment facility. Descriptive study, therefore cannot determine causality.
Rintoul²⁶
Objectives, methods, main findings clearly described. Confidence intervals and exact P-values reported.	Descriptive study – cannot determine causation. Possible that not all deaths were identified or that drug-related deaths were misclassified.
Roxburgh²
Objectives, methods, main findings clearly described Exposures well described	Descriptive study – cannot determine causation. Likely that not all exposures were identified – some were likely classified as ‘opioid’ or misclassified as another opioid event.

Table A5Strengths and Limitations of the Included Qualitative Study

Strengths

Limitations

Buer²⁷

Research objectives clearly stated and congruent with qualitative methods.
Sampling strategy clearly described – and was altered when it became clear that the youngest age group was underrepresented.
Participants described with sufficient detail.
Results consistent with the data.
Data collection clearly described and was congruent with the research objectives.

Ethics approval not reported, however it was likely sought as it was part of a larger quantitative study.
Small sample in a small geographic area – results may not generalize to other populations.
Self-reported data – therefore may be subject to recall bias.
Reporting socially undesirable behavior – therefore may be subject to responder bias.
Unclear how the data was checked for credibility (the interviews were audio-recorded and transcribed, but unclear if the transcriptions were checked or if there was another method of credibility checking)
Unclear if techniques were used to enhance the dependability

Table A6Strengths and Limitations of Economic Studies using Drummond¹¹

Strengths	Limitations
White²⁸
Multiple models used. Models checked for robustness Used real population data to inform estimates	Multiple data sources – exposures and drug formulations may be coded differently so data may not be consistent. NSDUH past-year or past-month measures were not available and would likely be better predictors than the lifetime use that they used. Small sample size, therefore results may not be generalizable. Misclassification of data possible while coding entries into database. Does not include general medical costs. Likely not generalizable beyond the single payer perspective
Rossiter²⁰
Their base case was set at 75% of the abuse rate found in the database and sensitivity analyses included 50% and 100% rates. ADF oxycodone was cost saving at all estimates. Used real population data to inform estimates.	Not a full economic analysis Indirect costs not considered Based on data from an observational study (evaluation of claims data) – we cannot conclude the relationship was causal. Cost estimates likely only relevant to the US population (however reductions likely in other jurisdictions, but monetary values not transferrable due to differences in price)

: ADF = abuse deterrent formulation

APPENDIX 3Critical Appraisal of Included Publications

Table A3

Table A4

Table A5

Table A6

Publication Details

Copyright

Publisher

NLM Citation

Table A3Strengths and Limitations of Randomized Controlled Trials using Downs and Black10

Table A4Strengths and Limitations of Observational Studies using Downs and Black10

Table A5Strengths and Limitations of the Included Qualitative Study

Table A6Strengths and Limitations of Economic Studies using Drummond11

Table A3Strengths and Limitations of Randomized Controlled Trials using Downs and Black¹⁰

Table A4Strengths and Limitations of Observational Studies using Downs and Black¹⁰

Table A6Strengths and Limitations of Economic Studies using Drummond¹¹