Abstract The emergence of Deep Learning provides an opportunity for traffic flow prediction. However, uncertainty and volatility exhibited by nonlinearity and instability of traffic flow pose challenges to Deep Learning models. Therefore, a combined prediction model, VMD-GAT-MGTCN, based on variational modal decomposition (VMD), graph attention network (GAT), and multi-gated attention time convolutional network (MGTCN) is proposed to enhance short-term traffic flow prediction accuracy. In the VMD-GAT-MGTCN, VMD decomposes traffic flow data to obtain the modal components, the GAT and MGTCN are integrated to design the spatio-temporal feature model to obtain the temporal and spatial features of traffic flow. The predicted value of traffic flow modal components by spatio-temporal feature model are stacked to obtain the ultimate traffic flow prediction results. The simulation experiments with the compared models and the baseline models show that the VMD-GAT-MGTCN have superior prediction accuracy and effect. It also verifies the enhancement effect of the VMD algorithm on the prediction performance of the VMD-GAT-MGTCN and the good prediction results obtained by the VMD-GAT-MGTCN in the traffic flow mutation region.