Lateral inhibition of cells surrounding an excited area is a key property of sensory systems, sharpening the preferential tuning of individual cells in the presence of closely related input signals. In the olfactory pathway, a dendrodendritic synaptic microcircuit between mitral and granule cells in the olfactory bulb has been proposed to mediate this type of interaction through granule cell inhibition of surrounding mitral cells. However, it is becoming evident that odor inputs result in broad activation of the olfactory bulb with interactions that go beyond neighboring cells. Using a realistic modeling approach we show how backpropagating action potentials in the long lateral dendrites of mitral cells, together with granule cell actions on mitral cells within narrow columns forming glomerular units, can provide a mechanism to activate strong local inhibition between arbitrarily distant mitral cells. The simulations predict a new role for the dendrodendritic synapses in the multicolumnar organization of the granule cells. This new paradigm gives insight into the functional significance of the patterns of connectivity revealed by recent viral tracing studies. Together they suggest a functional wiring of the olfactory bulb that could greatly expand the computational roles of the mitral-granule cell network.