Chinese herb medicine (CHM) is the most commonly reported traditional Chinese medicine (TCM) modality. This study aimed to assess the prevalence and associated factors of CHM use in cancer patients in southwestern China. Cancer patients from eleven comprehensive cancer centers were asked to complete a structured questionnaire. Of 587 available replies, 53.0% used CHM. Multiple logistic regression analysis showed that educational level, stage of disease, duration of cancer since diagnosis, marital status, and previous use of CHM were strongly associated with CHM use after cancer diagnosis. The source of information about CHM was mainly from media and friends/family. CHM products were used without any consultation with a TCM practitioner by 67.5% of users. The majority used CHM to improve their physical and emotional well-beings and to reduce cancer therapy-induced toxicities. About 4.5% patients reported side effects of CHM. This survey revealed a high prevalence of CHM use among cancer patients. However, these patients did not get sufficient consultation about the indications and contradictions of these drugs. It is imperative for oncologists to communicate with their cancer patients about the usage of CHM so as to avoid the potential side effects.