Penicillium species are fungal pathogens that infect crop plants worldwide. P. expansum differs from P. italicum and P. digitatum, all major postharvest pathogens of pome and citrus, in that the former is able to produce the mycotoxin patulin and has a broader host range. The molecular basis of host-specificity of fungal pathogens has now become the focus of recent research. The present report provides the whole genome sequence of P. expansum (33.52 Mb) and P. italicum (28.99 Mb) and identifies differences in genome structure, important pathogenic characters, and secondary metabolite (SM) gene clusters in Penicillium species. We identified a total of 55 gene clusters potentially related to secondary metabolism, including a cluster of 15 genes (named PePatA to PePatO), that may be involved in patulin biosynthesis in P. expansum. Functional studies confirmed that PePatL and PePatK play crucial roles in the biosynthesis of patulin and that patulin production is not related to virulence of P. expansum. Collectively, P. expansum contains more pathogenic genes and SM gene clusters, in particular, an intact patulin cluster, than P. italicum or P. digitatum. These findings provide important information relevant to understanding the molecular network of patulin biosynthesis and mechanisms of host-specificity in Penicillium species.