High-precision displacement sensing serves as a cornerstone for measuring numerous physical quantities. Through plasmon-enhanced fluorescence, we propose a nitrogen-vacancy-based highly sensitive and microwave-free distance measurement scheme for metal tips with a sub-wavelength scale size. The best sensitivity is estimated to be $ 72\,\mathrm{fm/\sqrt{Hz}}$ with a high bandwidth of $ 21 \,\mathrm{GHz}$ through simulations and numerical analyses. The study reveals that the sensor performance remains insensitive to fluctuations in plasmon resonance wavelength, indicating the system’s inherent tolerance to structural deviations, potentially stemming from manufacturing imperfections. Furthermore, a greater quantum radiation efficiency leads to an improved sensitivity and a wider working range. This low-noise, no-damage, and robust scheme with a high frequency bandwidth has potential applications in on-chip inertial devices and basic physical research.