Population Assessment of Tobacco and Health (PATH) reliability and validity study: selected reliability and validity estimates

Introduction: This paper reports a study done to estimate the reliability and validity of answers to the Youth and Adult questionnaires of the Population Assessment of Tobacco and Health (PATH) Study.

Methods: 407 adults and 117 youth respondents completed the wave 4 (2016-2017) PATH Study interview twice, 6-24 days apart. The reinterview data were used to estimate the reliability of answers to the questionnaire. Kappa statistics, gross discrepancy rates and correlations between answers to the initial interview and the reinterview were used to measure reliability. We examined every item in the questionnaire for which there were at least 100 observations. After the reinterview, most respondents provided a saliva sample that allowed us to assess the accuracy of their answers to the tobacco use questions.

Results: There was generally a very high level of agreement between answers in the interview and reinterview. On the key current tobacco use items, the average kappa (the agreement rate adjusted for chance agreement) was 0.79 for adult respondents (age 18 or older). Youth respondents exhibited equally high levels of agreement across interviews. The items on current tobacco use also exhibited high levels of agreement with saliva test results (kappa=0.72). Rating scale items showed lower levels of exact agreement across interviews but the answers were generally within one scale point or category.

Conclusions: The PATH Study questions were developed using a careful protocol and the results indicate the answers provide reliable and valid information about tobacco use.