Insulin-stimulated translocation of the glucose transporter GLUT4 to the plasma membrane in muscle and fat cells depends on the phosphatidylinositide 3-kinase/Akt pathway. The downstream target AS160/TBC1D4 is phosphorylated upon insulin stimulation and contains a TBC domain (Tre-2/Bub2/Cdc16) that is present in most Rab guanosine triphosphatase-activating proteins. TBC1D4 associates with GLUT4-containing membranes under basal conditions and dissociates from membranes with insulin. Here we show that the association of TBC1D4 with membranes is required for its inhibitory action on GLUT4 translocation under basal conditions. Whereas the insulin-dependent dissociation of TBC1D4 from membranes was not required for GLUT4 translocation, its phosphorylation was essential. Many agonists that stimulate GLUT4 translocation failed to trigger TBC1D4 translocation to the cytosol, but in most cases these agonists stimulated TBC1D4 phosphorylation at T642, and their effects on GLUT4 translocation were inhibited by overexpression of the TBC1D4 phosphorylation mutant (TBC1D4-4P). We postulate that TBC1D4 acts to impede GLUT4 translocation by disarming a Rab protein found on GLUT4-containing-membranes and that phosphorylation of TBC1D4 per se is sufficient to overcome this effect, allowing GLUT4 translocation to the cell surface to proceed.