Immune checkpoint inhibitors (ICIs) have a major impact on cancer treatment. However, the therapeutic efficacy of ICIs is only effective in some patients. Programmed death ligand 1 (PD-L1), tumor mutation burden (TMB), and high-frequency microsatellite instability (MSI-high) are markers that predict the efficacy of ICIs but are not universally used in many carcinomas. The gut microbiota has received much attention recently because of its potential to have a significant impact on immune cells in the cancer microenvironment. Metabolites of the gut microbiota modulate immunity and have a strong influence on the therapeutic efficacy of ICI. It has been suggested that the gut microbiota may serve as a novel marker to predict the therapeutic efficacy of ICI. Therefore, there is an urgent need to develop biomarkers that can predict anti-tumor effects and adverse events, and the study of the gut microbiota is essential in this regard.