We propose a novel subwavelength terahertz (THz) waveguide based on the magnetic plasmon polariton mode guided by a narrow gap in a negative permeability metamaterial. Deep subwavelength waveguiding (<lambda/10) with group velocities down to c/21.8 is demonstrated in a straight waveguide, a 90 degrees bend, and a splitter. The proposed waveguiding system inherently has no cutoff for any core width and height, paving the way toward the deep subwavelength transport of THz waves for integrated THz device applications.