Ma Zhongying joined the Muslim militia in 1924 when he was 14 years old. Ma Zhongying was involved in the rebellion against Feng Yuxiang's Guominjun forces in Gansu, and even fought against his own relatives such as the warlord Ma Lin, who had continued to serve the Guominjun.