Type 2 diabetes mellitus (T2DM) is a metabolic disease that seriously threatens the health of patients. Modern medicine is mainly used to control blood sugar stability and prevent T2DM complications. T2DM patients are required to take hypoglycemic agents for life. Although chemical drugs have a definite curative effect, they also cause several side effects after long-term use, which seriously limits their use. Traditional Chinese medicine (TCM) has a long history of blood glucose-lowering abilities, and it has unique advantages in the treatment of complex diseases ( i. e., T2DM), but its mechanism is not fully understood. Recent studies have shown that TCM can lower blood glucose by regulating gut microbiota to improve glucose metabolism, which provides new method and ideas for the prevention and treatment of T2DM. This article reviews the recent progress in the understanding of the roles and mechanisms of TCM, its extracts, and compound preparations to lower blood glucose by regulating gut microbiota. We also provide references and recommendations for future research on the blood glucose-lowering ability of TCM.