Low phosphorus (P) availability is a major constraint to crop growth and production, including soybean (Glycine max), on a global scale. However, 50% to 80% of the total P in agricultural soils exists as organic phosphate, which is unavailable to plants unless hydrolyzed to release inorganic phosphate. One strategy for improving crop P nutrition is the enhanced activity of acid phosphatases (APases) to obtain or remobilize inorganic phosphate from organic P sources. In this study, we overexpressed an Arabidopsis (Arabidopsis thaliana) purple APase gene (AtPAP15) containing a carrot (Daucus carota) extracellular targeting peptide in soybean hairy roots and found that the APase activity was increased by 1.5-fold in transgenic hairy roots. We subsequently transformed soybean plants with AtPAP15 and studied three homozygous overexpression lines of AtPAP15. The three transgenic lines exhibited significantly improved P efficiency with 117.8%, 56.5%, and 57.8% increases in plant dry weight, and 90.1%, 18.2%, and 62.6% increases in plant P content, respectively, as compared with wild-type plants grown on sand culture containing phytate as the sole P source. The transgenic soybean lines also exhibited a significant level of APase and phytase activity in leaves and root exudates, respectively. Furthermore, the transgenic lines exhibited improved yields when grown on acid soils, with 35.9%, 41.0%, and 59.0% increases in pod number per plant, and 46.0%, 48.3%, and 66.7% increases in seed number per plant. Taken together, to our knowledge, our study is the first report on the improvement of P efficiency in soybean through constitutive expression of a plant APase gene. These findings could have significant implications for improving crop yield on soils low in available P, which is a serious agricultural limitation worldwide.