Effects of tempering temperature on the microstructure evolution and mechanical properties of Cr–Ni–Mo–V steel with banded structure were investigated in this study. It is indicated that the tempering temperature has a significant influence on the morphology of the martensite lath in the banded structure. Carbides M3C, M2C and M2C/M7C3 are identified at tempering temperature of 430 °C, 580 °C and 630 °C, respectively. The MC precipitate is a stable phase existing in a wide range of tempering temperature. As the tempering temperature increases, the tensile strength is gradually decreased, while the elongation and impact toughness are improved. The delaminated crack observed in the fracture surface is connected to the banded structure in Cr–Ni–Mo–V steel. It is suggested that the enhancement of ductility and toughness can be attributed to the transformation of the stress state and the blunting of the crack tip. The relationship between the microstructure and mechanical properties is explored, and a detailed insight into the precipitation processes of carbides at different tempering temperature are conducted.