A novel method for the segmentation of multiple objects from 3D medical images using inter-object constraints is presented. Our method is motivated by the observation that neighboring structures have consistent locations and shapes that provide configurations and context that aid in segmentation. We define a Maximum A Posteriori(MAP) estimation framework using the constraining information provided by neighboring objects to segment several objects simultaneously. We introduce a representation for the joint density function of the neighbor objects, and define joint probability distributions over the variations of the neighboring positions and shapes of a set of training images. By estimating the MAP shapes of the objects, we formulate the model in terms of level set functions, and compute the associated Euler-Lagrange equations. The contours evolve both according to the neighbor prior information and the image gray level information. We feel that this method is useful in situations where there is limited inter-object information as opposed to robust global atlases. Results and validation from various experiments on synthetic data and medical imagery in 2D and 3D are demonstrated.