Accurate prediction of future rice yield needs the precise estimations of rice yield response to climate change factors, of which the most important one is the increasing carbon dioxide (CO2) concentrations. Estimates of CO2 fertilization effect (CFE) on rice, however, still had large uncertainties. Therefore, using the rice planting areas in East China as the study area, we firstly compared the rice yields and CFE predicted by four state-of-the-art crop models, and found that the CFE predicted by these models had significant differences. We then quantified the CFE on rice yield using the field-controlled experiment conducted at Danyang site at Jiangsu province. Using CFE measurements from a field experiment as benchmark, we have developed an experiment–model integration approach aiming to reduce this variation. This study thus highlights the large CFE uncertainties of current crop models and provides us with a method to reduce this uncertainty, which is beneficial for the accurate prediction of future global rice yield in the context of climate change.