We propose using an optical cavity to enhance the sensitivity of a magnetometer relying on the detection of the spin state of a high-density nitrogen-vacancy ensemble in diamond using infrared optical absorption. The role of the cavity is to obtain a contrast in the absorption-detected magnetic resonance approaching unity at room temperature. We project an increase in the photon shot-noise limited sensitivity of two orders of magnitude in comparison with a single-pass approach. Optical losses can limit the enhancement to one order of magnitude, which could still enable room-temperature operation. Finally, the optical cavity also allows us to use less pumping power when the cavity is resonant at both the pump and the infrared probe wavelength.