Vibrational properties of solids are key to determining stability, response and functionality. However, they are challenging to computationally predict at Ab-Initio accuracy, even for elemental systems. Ab-Initio methods for modeling atomic interactions are limited in the system sizes and simulation times that can be achieved. Due to these limitations, Machine Learning Interatomic Potentials (MLIPs) are gaining popularity and success as a faster, more scalable approach for modeling atomic interactions, potentially at Ab-Initio accuracy. Even with faster potentials, methodologies for predicting entropy, free energy and vibrational properties vary in accuracy, cost and difficulty to implement. Using the Covariance of Atomic Displacements (CAD) to predict entropy, free energy and finite-temperature phonon dispersions is a promising approach but thorough benchmarking has been hampered by the cost of Ab-Initio methods for sampling. In this work, we use a MLIP and the CAD to characterize the convergence of the predicted properties and determine optimal sampling strategies. We focus on solid lithium at zero pressure, showing that the MLIP-CAD approach reproduces experimental entropy, phonon dispersions and the martensitic transition while also comparing to more established methods.