Abstract Light-matter interaction is exploited in spectroscopic techniques to access information about molecular, atomic or nuclear constituents of a sample. While scattered light carries both amplitude and phase information of the electromagnetic field, the latter is lost in intensity measurements. However, often the phase information is paramount to reconstruct the desired information of the target, as it is well known from coherent x-ray imaging. Here we introduce a phase retrieval method which allows us to reconstruct the field phase information from two-dimensional time- and energy-resolved spectra. We apply this method to the case of x-ray scattering off Mössbauer nuclei at a synchrotron radiation source. Knowledge of the phase allows also for the reconstruction of energy spectra from two-dimensional experimental data sets with excellent precision, without theoretical modelling of the sample. Our approach provides an efficient and accurate data analysis tool which will benefit x-ray quantum optics and Mössbauer spectroscopy with synchrotron radiation alike.