Quantitative evaluation of the thermodynamic properties of materials—most notably their stability, as measured by the free energy—must take into account the role of thermal and zero-point energy fluctuations. While these effects can easily be estimated within a harmonic approximation, corrections arising from the anharmonic nature of the interatomic potential are often crucial and require computationally costly path integral simulations to obtain results that are essentially exact for a given potential. Consequently, different approximate frameworks for computing affordable estimates of the anharmonic free energies have been developed over the years. Understanding which of the approximations involved are justified for a given system, and therefore choosing the most suitable method, is complicated by the lack of comparative benchmarks. To facilitate this choice we assess the accuracy and efficiency of some of the most commonly used approximate methods: the independent mode framework, the vibrational self-consistent field, and self-consistent phonons. We compare the anharmonic correction to the Helmholtz free energy against reference path integral calculations. These benchmarks are performed for a diverse set of systems, ranging from simple weakly anharmonic solids to flexible molecular crystals with freely rotating units. The results suggest that, for simple solids such as allotropes of carbon, these methods yield results that are in excellent agreement with the reference calculations, at a considerably lower computational cost. For more complex molecular systems such as polymorphs of ice and paracetamol the methods do not consistently provide a reliable approximation of the anharmonic correction. Despite substantial cancellation of errors when comparing the stability of different phases, we do not observe a systematic improvement over the harmonic approximation even for relative free energies. We conclude that, at least for the classes of materials considered here, efforts toward obtaining computationally feasible anharmonic free energies should therefore be directed toward reducing the expense of path integral methods.