The novel carbon-based solid acid catalyst (S-800-CG) was synthesized by high-temperature carbonization of calcium gluconate followed by sulfonation with 4-diazoniobenzene sulfonate at room temperature. The catalyst was characterized by TEM, SEM, N2 adsorption-desorption, TGA, FT-IR, and XPS to reveal its physical and chemical properties. The S-800-CG was used as a catalyst for converting xylose or corncob into furfural via a one-step process. Furthermore, the effects of calcination temperatures of the carbon support, reaction temperature, residence time, catalyst loading, and substrate concentration were investigated. It was demonstrated that S-800-CG is an efficient solid acid catalyst for furfural production from xylose in 1,4-dioxane. 85.9% furfural yield can be achieved from 50 mg xylose at 140 °C in 40 min by using 100 mg catalyst. Moreover, 52.9% furfural yield was obtained from 200 mg corncob at 190 °C in 70 min using 100 mg catalyst.