These human reactions are then used to train a reward model, which, the theory goes, will reflect human desires for the ultimate language model. Constitutional AI tries a different approach.