NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Hartling L, Hamm M, Milne A, et al. Validity and Inter-Rater Reliability Testing of Quality Assessment Instruments [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2012 Mar.
Validity and Inter-Rater Reliability Testing of Quality Assessment Instruments [Internet].
Show details- 1.
- Higgins PT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. 2009. 5.0.2 [updated September 2009]. Available from: http:
//cochrane-handbook.org. - 2.
- Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. [PMC free article: PMC2943439] [PubMed: 20877712]
- 3.
- Moher D, Jadad AR, Nichol G, et al. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16(1):62–73. [PubMed: 7743790]
- 4.
- Juni P, Witschi A, Bloch R, et al. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282(11):1054–60. [PubMed: 10493204]
- 5.
- West S, King V, Carey TS, et al. Systems to rate the strength of scientific evidence. Evid Rep Technol Assess (Summ). 2002;(47):1–11. [PMC free article: PMC4781591] [PubMed: 11979732]
- 6.
- Olivo SA, Macedo LG, Gadotti IC, et al. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88(2):156–75. [PubMed: 18073267]
- 7.
- Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9. [PubMed: 11730399]
- 8.
- Siersma V, ls-Nielsen B, Chen W, et al. Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med. 2007;26(14):2745–58. [PubMed: 17117373]
- 9.
- Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36(3):666–76. [PubMed: 17470488]
- 10.
- Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess. 2003;7(27):iii–173. [PubMed: 14499048]
- 11.
- Higgins PT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. 2011. (5.1.0. [updated March 2011]). Available from: http:
//cochrane-handbook.org. - 12.
- Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352(9128):609–13. [PubMed: 9746022]
- 13.
- Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12. [PubMed: 7823387]
- 14.
- Als-Nielsen B, Gluud LL, Gluud C. Methodological quality and treatment effects in randomised trials: a review of six empirical studies. 12th Cochrane Colloquium 2004; Oct 2-6; Ottawa, Ontario, Canada.
- 15.
- Balk EM, Bonis PA, Moskowitz H, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002;287(22):2973–82. [PubMed: 12052127]
- 16.
- Egger M, Juni P, Bartlett C, et al. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1–76. [PubMed: 12583822]
- 17.
- Pildal J, Hrobjartsson A, Jorgensen KJ, et al. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36(4):847–57. [PubMed: 17517809]
- 18.
- Savovic J. The association of three bias domains with treatment effect estimates in randomised control trials: combined analysis of meta-epidemiological studies. Z Evid Fortbild Qual Gesundhwes. 2008;102:29–30.
- 19.
- Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601–5. [PMC free article: PMC2267990] [PubMed: 18316340]
- 20.
- Melander H, hlqvist-Rastad J, Meijer G, et al. Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ. 2003;326(7400):1171–3. [PMC free article: PMC156459] [PubMed: 12775615]
- 21.
- Porta N, Bonet C, Cobo E. Discordance between reported intention-to-treat and per protocol analyses. J Clin Epidemiol. 2007;60(7):663–9. [PubMed: 17573981]
- 22.
- Tierney JF, Stewart LA. Investigating patient exclusion bias in meta-analysis. Int J Epidemiol. 2005;34(1):79–87. [PubMed: 15561753]
- 23.
- Abraha I, Duca PG, Montedori A. Empirical evidence of bias: modified intention to treat analysis of randomised trials affects estimates of intervention efficacy. Z Evid Fortbild Qual Gesundhwes. 2008;102(Suppl VI):9.
- 24.
- Nuesch E, Trelle S, Reichenbach S, et al. Empirical evidence of attrition bias in meta-analyses of randomized controlled trials. Z Evid Fortbild Qual Gesundhwes. 2008;102:9.
- 25.
- Chan AW, Hrobjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65. [PubMed: 15161896]
- 26.
- Chan AW, Krleza-Jeric K, Schmid I, et al. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735–40. [PMC free article: PMC517858] [PubMed: 15451835]
- 27.
- Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753. [PMC free article: PMC555875] [PubMed: 15681569]
- 28.
- Dwan K, Altman DG, Arnaiz JA, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE. 2008;3(8):e3081. [PMC free article: PMC2518111] [PubMed: 18769481]
- 29.
- Hahn S, Williamson PR, Hutton JL. Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract. 2002;8(3):353–9. [PubMed: 12164983]
- 30.
- Von Elm E, Rollin A, Blumle A, et al. Selective reporting of outcomes of drug trials? Comparison of study proptocols and published articles. XIV Cochrane Colloquium 2006; October 23-26; Dublin, Ireland.
- 31.
- Bassler D, Ferreira-Gonzalez I, Briel M, et al. Systematic reviewers neglect bias that results from trials stopped early for benefit. J Clin Epidemiol. 2007;60(9):869–73. [PubMed: 17689802]
- 32.
- Montori VM, Devereaux PJ, Adhikari NK, et al. Randomized trials stopped early for benefit: a systematic review. JAMA. 2005;294(17):2203–9. [PubMed: 16264162]
- 33.
- Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–65. [PubMed: 12533125]
- 34.
- Lexchin J, Bero LA, Djulbegovic B, et al. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326(7400):1167–70. [PMC free article: PMC156458] [PubMed: 12775614]
- 35.
- Sismondo S. Pharmaceutical company funding and its consequences: a qualitative systematic review. Contemp Clin Trials. 2008;29(2):109–13. [PubMed: 17919992]
- 36.
- Gotzsche PC, Hrobjartsson A, Johansen HK, et al. Constraints on publication rights in industry-initiated clinical trials. JAMA. 2006;295(14):1645–6. [PubMed: 16609085]
- 37.
- Savovic J, Harris R., The Brando Collaborators. [O16-61] The association of three bias domains with treatment effect estimates in randomised control trials: combined analysis of meta-epidemiological studies. Cochrane Colloquium Abstracts Journal. 2008 [Abstract]
- 38.
- Hartling L, Ospina M, Liang Y, et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009;339:b4012. [PMC free article: PMC2764034] [PubMed: 19841007]
- 39.
- Hartling L, Bond K, Vandermeer B, et al. Applying the risk of bias tool in a systematic review of combination long-acting beta-agonists and inhaled corticosteroids for persistent asthma. PLoS ONE. 2011;6(2):e17242. [PMC free article: PMC3044729] [PubMed: 21390219]
- 40.
- Wells G, Shea B, O'Connell J, Robertson J, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analysis. 2011. [Website]. http://www
.ohri.ca/programs /clinical_epidemiology/oxford.asp. Available from: URL: http://www .ohri.ca/programs /clinical_epidemiology/oxford.asp. - 41.
- Wells G, Shea B, O'Connell J, Robertson J, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analysis. 3rd Symposium on Systematic Reviews: Beyond the Basics; July 3–5; Oxford. 2000. [Abstract]
- 42.
- Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377–84. [PMC free article: PMC1756728] [PubMed: 9764259]
- 43.
- Wells G, Brodsky L, O'Connell D, Robertson J, et al. Evaluation of the Newcastle-Ottawa Scale (NOS): an assessment tool for evaluating the quality of non-randomized studies. XI Cochrane Colloquium: Evidence, Health Care and Culture; Oct 26–31; Barcelona, Spain. 2003. [Abstract]
- 44.
- Hopewell S, Dutton S, Yu LM, et al. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010;340:c723. [PMC free article: PMC2844941] [PubMed: 20332510]
- 45.
- Liebetrau A. Measures of association. Newbury Park, CA: Sage Publications; 1983.
- 46.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. [PubMed: 843571]
- 47.
- Fleiss JL, Levin BA, Paik MC. Statistical methods for rates and proportions. 3 ed. Wiley; 2003.
- 48.
- Rosenthal R. Parametric measures of effect size. In: Cooper H, Hedges LV, editors. The Handbook of Research Synthesis. New York: Russel Sage Foundation; 1994.
- 49.
- Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med. 2000;19(22):3127–31. [PubMed: 11113947]
- 50.
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88. [PubMed: 3802833]
- 51.
- Ip S, Chung M, Raman G, et al. Breastfeeding and maternal and infant health outcomes in developed countries. Evid Rep Technol Assess (Full Rep). 2007;(153):1–186. [PMC free article: PMC4781366] [PubMed: 17764214]
- 52.
- McAlister FA, Ezekowitz J, Dryden DM, et al. Cardiac resynchronization therapy and implantable cardiac defibrillators in left ventricular systolic dysfunction. Evid Rep Technol Assess (Full Rep). 2007;(152):1–199. [PMC free article: PMC4781294] [PubMed: 17764218]
- 53.
- Santaguida PL, Balion C, Hunt D, et al. Diagnosis, prognosis, and treatment of impaired glucose tolerance and impaired fasting glucose. Evid Rep Technol Assess. 2005;(128):1–11. [PMC free article: PMC4780988] [PubMed: 16194123]
- 54.
- Furukawa TA, Watanabe N, Omori IM, et al. Association between unreported outcomes and effect size estimates in Cochrane meta-analyses. JAMA. 2007;297(5):468–70. [PubMed: 17284696]
- 55.
- Buxton AE, Lee KL, Fisher JD, et al. A randomized study of the prevention of sudden death in patients with coronary artery disease. Multicenter Unsustained Tachycardia Trial Investigators. N Engl J Med. 1999;341(25):1882–90. [PubMed: 10601507]
- 56.
- Sanchez JM, Katsiyiannis WT, Gage BF, et al. Implantable cardioverter-defibrillator therapy improves long-term survival in patients with unexplained syncope, cardiomyopathy, and a negative electrophysiologic study. Heart Rhythm. 2005;2(4):367–73. [PubMed: 15851337]
- 57.
- Frost FJ, Petersen H, Tollestrup K, et al. Influenza and COPD mortality protection as pleiotropic, dose-dependent effects of statins. Chest. 2007;131(4):1006–12. [PubMed: 17426203]
- 58.
- Tseng MY, Hutchinson PJ, Czosnyka M, et al. Effects of acute pravastatin treatment on intensity of rescue therapy, length of inpatient stay, and 6-month outcome in patients after aneurysmal subarachnoid hemorrhage. Stroke. 2007;38(5):1545–50. [PubMed: 17413047]
- 59.
- Wisner KL, Sit DK, Hanusa BH, et al. Major depression and antidepressant treatment: impact on pregnancy and neonatal outcomes. Am J Psychiatry. 2009;166(5):557–66. [PMC free article: PMC4426499] [PubMed: 19289451]
- 60.
- Suri R, Altshuler L, Hellemann G, et al. Effects of antenatal depression and antidepressant treatment on gestational age at birth and risk of preterm birth. Am J Psychiatry. 2007;164(8):1206–13. [PubMed: 17671283]
- 61.
- Ancel P, Saurel-Cubizolles M, Di Renzo G, Papiernik E, Breart G. Very and moderate preterm births: are the risk factors different? Br J Obstet Gynaecol. 1999;106:1162–70. [PubMed: 10549961]
- 62.
- Schlienger R, Fedson D, Jick S, Jick H, Meier C. Statins and the risk of pneumonia: a population-based, nested case-control study. Pharmacotherapy. 2007;27:325–32. [PubMed: 17316144]
- 63.
- Blaas S, Mutterlein R, Weig J, et al. Extensively drug resistant tuberculosis in a high income country: a report of four unrelated cases. BMC Infect Dis. 2008;8:60–7. [PMC free article: PMC2413242] [PubMed: 18454863]
- 64.
- Condos R, Hadgiangelis N, Leibert E, et al. Case series report of a linezolid-containing regimen for extensively drug-resistant tuberculosis. Chest. 2008;134:187–92. [PubMed: 18628223]
- 65.
- Rahaman J, Narayansingh G, Roopnarinesingh S. Fetal outcome among obese parturients. Int J Gynaecol Obstet. 1990;31:227–30. [PubMed: 1969362]
- 66.
- Dayan J, Creveuil C, Herlicoviez M, et al. Antenatal depression, a risk factor for prenatal delivery. Presse Med. 1999;28(31):1698. [PubMed: 10554610]
- 67.
- Kim Y, Lee B, Park H. Risk factors for preterm birth in Korea: a multicenter prospective study. Gynecol Obstet Invest. 2005;60:206–12. [PubMed: 16088197]
- 68.
- Norman R, Masters L, Milner C, Wang J, Davies M. Relative risk of conversion from normoglycaemia to impaired glucose tolerance or non-insulin dependent diabetes mellitus in polycystic ovarian syndrome. Hum Reprod. 2001;16(9):1995–8. [PubMed: 11527911]
- 69.
- Wright A, Holberg C, Taussig L, Martinez F. Factors influencing the relation of infant feeding to asthma and recurrent wheeze in childhood. Thorax. 2001;56(3):192–7. [PMC free article: PMC1758780] [PubMed: 11182011]
- 70.
- Arvanitakis Z, Schneider JA, Wilson RS, et al. Statins, incident Alzheimer disease, change in cognitive function, and neuropathology. Neurology. 2008;70(19 Pt 2):1795–802. [PubMed: 18199831]
- 71.
- Agresti A. Categorical data analysis. New York: John Wiley & Sons; 2002.
- 72.
- Sterne JA, Juni P, Schulz KF, et al. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21(11):1513–24. [PubMed: 12111917]
- 73.
- Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. [PMC free article: PMC2844940] [PubMed: 20332509]
- 74.
- Hamm MP, Hartling L, Milne A, et al. A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010;10:96. [PMC free article: PMC3018376] [PubMed: 21176224]
- 75.
- Crocetti MT, Amin DD, Scherer R. Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics. 2010;126(2):298–305. [PubMed: 20624806]
- 76.
- Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25(9):603–5. [PubMed: 20652370]
- 77.
- Schunemann HJ, Brozek J, Oxman AD. GRADE handbook for grading quality of evidence and strength of recommendation. The Grade Working Group; 2009. Version 3.2 [update March 2009] ed. Available from: http://www
.gradeworkinggroup .org/index.html. - 78.
- Berkman ND, Viswanathan M. Development of a tool to evaluate the quality of non-randomized studies of interventions or exposures. AHRQ 2009 Annual Conference; Sept 15, 2009; Bethesda, MA. 2009.
- References - Validity and Inter-Rater Reliability Testing of Quality Assessment ...References - Validity and Inter-Rater Reliability Testing of Quality Assessment Instruments
Your browsing activity is empty.
Activity recording is turned off.
See more...