Familiarity to the mother and the novelty afforded by the postnatal environment are two contrasting sources of neonatal influence. One hypothesis regarding their relationship is the maternal modulation hypothesis, which predicts that the same neonatal stimulation may have different effects depending on the maternal context. Here we tested this hypothesis using physical development, indexed by body weight, as an endpoint and found that, among offspring of mothers with a high initial swim-stress-induced corticosterone (CORT) response, neonatal novelty exposure induced an enhancement in early growth, and among offspring with mothers of a low initial CORT response, the same neonatal stimulation induced an impairment. At an older age, a novelty-induced increase in body weight was also found among offspring of mothers with high postnatal care reliability and a novelty-induced reduction found among offspring of mothers with low care reliability. These results support a maternal modulation of early stimulation effects on physical development and demonstrate that the maternal influence originates from multiple instead of any singular sources. These results (i) significantly extend the findings of maternal modulation from the domain of cognitive development to the domain of physical development; (ii) offer a unifying explanation for a previously inconsistent literature regarding early stimulation effects on body weight; and (iii) highlight the notion that the early experience effect involves no causal primacy but higher order interactions among the initial triggering events and subsequent events involving a multitude of maternal and nonmaternal influences.