Using data from 335,249 Medicare beneficiaries who responded to the 2007 Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey, along with data from 22 cognitive interviews, we investigated the reliability and validity of an instrument designed to assess beneficiaries' experiences with their prescription drug plans. Composite measures derived from the instrument had acceptable internal consistency and sufficient plan-level reliability to inform consumer choice, quality improvement, and payor oversight. These measures were positively associated with members' overall rating of the plan and their willingness to recommend the plan. Moreover, each was independently useful in predicting beneficiaries' global ratings of their plan. This instrument can be an important tool for helping beneficiaries to choose a plan that best meets their needs.