Establishing an effective method to improve stem cell differentiation is crucial in stem cell transplantation. Here we aimed to explore whether and how sodium butyrate (NaB) induces rat bone marrow mesenchymal stem cells (MSCs) to differentiate into bladder smooth muscle cells (SMCs). We found that NaB significantly suppressed MSC proliferation and promoted MSCs differentiation into SMCs, as evidenced by the enhanced expression of SMC specific genes in the MSCs. Co-culturing the MSCs with SMCs in a transwell system promoted the differentiation of MSCs into SMCs. NaB again promoted MSC differentiation in this system. Furthermore, NaB enhanced the acetylation of SMC gene-associated H3K9 and H4, and decreased the expression of HDAC2 and down-regulated the recruitment of HDAC2 to the promoter regions of SMC specific genes. Finally, we found that NaB significantly promoted MSC depolarization and increased the intracellular calcium level of MSCs upon carbachol stimulation. These results demonstrated that NaB effectively promotes MSC differentiation into SMCs, possibly by the marked inhibition of HDAC2 expression and disassociation of HDAC2 recruitment to SMC specific genes in MSCs, which further induces high levels of H3K9ace and H4ace and the enhanced expression of target genes, and this strategy could potentially be applied in clinical tissue engineering and cell transplantation.