We use methods from microlocal analysis and quantum scattering to study spectral properties near the threshold zero of the Kramers-Fokker-Planck operator with a decaying potential in $\mathbb R^n$, $n \ge 4$, and deduce the large-time behavior of solutions to the kinetic Kramers-Fokker-Planck equation. For short-range potentials, we establish an optimal time-decay estimate in weighted $L^2$-spaces when $ n\ge 5$ is odd. For potentials decaying like $O(|x|^{-\rho})$ for some $\rho > n-1$, we obtain, for all dimensions $n \ge 4$, a large-time expansion of the solution with the leading term given by the Maxwell-Boltzmann distribution multiplied by the factor $(4\pi t)^{-\frac n 2}$ corresponding to the decay for the heat equation. These results complete those obtained in [16, 22] for dimensions $n=1$ and $3$. The same questions for $n=2$ are still open.