Reinforcement learning from human feedback plays a crucial role in aligning language
models towards human preferences, traditionally represented through comparisons …