We present spectral domain phase microscopy (SDPM) as a new tool for measurements at the cellular scale. SDPM is a functional extension of spectral domain optical coherence tomography that allows for the detection of cellular motions and dynamics with nanometer-scale sensitivity in real time. Our goal was to use SDPM to investigate the mechanical properties of the cytoskeleton of MCF-7 cells. Magnetic tweezers were designed to apply a vertical force to ligand-coated magnetic beads attached to integrin receptors on the cell surfaces. SDPM was used to resolve cell surface motions induced by the applied stresses. The cytoskeletal response to an applied force is shown for both normal cells and those with compromised actin networks due to treatment with Cytochalasin D. The cell response data were fit to several models for cytoskeletal rheology, including one- and two-exponential mechanical models, as well as a power law. Finally, we correlated displacement measurements to physical characteristics of individual cells to better compare properties across many cells, reducing the coefficient of variation of extracted model parameters by up to 50%.