Broad peer review of draft evidence reports is critical to the validity and ultimate acceptability of systematic reviews from the evidence-based practice centers (EPC) of the Agency for Healthcare Research and Quality (AHRQ) and thus to the credibility of the AHRQ Evidence-based Practice Program more generally. It was equally important to satisfy the longer-term goals of the Social Security Administration (SSA) in this area of research and specifically for the evidence report on Criteria for Determining Disability in Speech and Language Disorders.

Thus, we requested review by 20 experts representing: (1) clinical areas including speech, language, and voice disorders, neurology and neuropsychology (both in children and adults), otolaryngology, and developmental pediatrics; (2) professional societies and organizations; (3) development, validation, and use of psychometric tests and other instruments; and (4) likely users of this EPC report, such as the SSA and other entities interested in the appropriate evaluation of speech and learning disorders (Table C1). We invited all 10 TEAG members (Appendix B) to provide peer review of the evidence report, a responsibility that exceeded their initial mandate.

On March 21, 2001, we sent letters to the 20 nominated experts and organizations explaining the peer review process and asking if they would be available to conduct the review in July and to return the comments by early August. Two experts declined our invitation to review the report because of time constraints and professional responsibilities. We could not reach the director of one organization nominated for review despite repeated attempts to contact her through September 2001. Unanticipated project delays led to a six-week delay in our delivery of the report to peer reviewers, and we thus sent the report to them on August 23, 2001, with a request for comments to be returned to us by September 21, 2001. Our review period coincided with the tragic national events of September 11, and we consequently extended our deadline for reviews until October 10, 2001. Of the 17 reviewers who agreed to perform the review, seven (four nominees from organizations and three individual experts) were unable to return reviews because of time constraints. Ultimately, 10 reviewers returned reviews. In addition, staff from both AHRQ and SSA reviewed the document and returned comments.

Peer reviewers provided comments on the content, structure, and format of the evidence report, paying particular attention the inclusion/exclusion of literature for the selected tests, to the analysis and interpretation of study results and evidence, and to the discussion of gaps and areas that should be targeted for future research. EPC staff took all comments into account in revisions to this evidence report; these are documented in a separate document recording the disposition of all peer review suggestions that was submitted to AHRQ late in 2001.