The problem of controlling for imperfectly measured confounders on dissimilar populations: a database simulation study

J Cardiothorac Vasc Anesth. 2014 Apr;28(2):247-54. doi: 10.1053/j.jvca.2013.03.014. Epub 2013 Aug 17.

Abstract

Objective(s): Observational database research frequently relies on imperfect administrative markers to determine comorbid status, and it is difficult to infer to what extent the associated misclassification impacts validity in multivariable analyses. The effect that imperfect markers of disease will have on validity in situations in which researchers attempt to match populations that have strong baseline health differences is underemphasized as a limitation in some otherwise high-quality observational studies. The present simulations were designed as a quantitative demonstration of the importance of this common and underappreciated issue.

Design: Two groups of Monte Carlo simulations were performed. The first demonstrated the degree to which controlling for a series of imperfect markers of disease between different populations taking 2 hypothetically harmless drugs would lead to spurious associations between drug assignment and mortality. The second Monte Carlo simulation applied this principle to a recent study in the field of anesthesiology that purported to show increased perioperative mortality in patients taking metoprolol versus atenolol.

Setting/participants/interventions: None.

Measurements and main results: Simulation 1: High type-1 error (ie, false positive findings of an independent association between drug assignment and mortality) was observed as sensitivity and specificity declined and as systematic differences in disease prevalence increased. Simulation 2: Propensity score matching across several imperfect markers was unlikely to eliminate important baseline health disparities in the referenced study.

Conclusions: In situations in which large baseline health disparities exist between populations, matching on imperfect markers of disease may result in strong bias away from the null hypothesis.

Keywords: Monte Carlo simulation; database research; misclassification bias.

MeSH terms

  • Adrenergic beta-Antagonists / therapeutic use
  • Algorithms
  • Atenolol / therapeutic use
  • Bias
  • Cardiac Surgical Procedures / statistics & numerical data*
  • Cohort Studies
  • Comorbidity
  • Computer Simulation
  • Data Interpretation, Statistical
  • Databases, Factual / standards*
  • Drug Therapy
  • Humans
  • International Classification of Diseases
  • Logistic Models
  • Metoprolol / therapeutic use
  • Models, Statistical
  • Monte Carlo Method
  • Observational Studies as Topic / statistics & numerical data*
  • Postoperative Complications / mortality
  • Prevalence
  • Propensity Score
  • Retrospective Studies

Substances

  • Adrenergic beta-Antagonists
  • Atenolol
  • Metoprolol