Reward Model
A small model that learns to predict which AI output a human would prefer — used to train the larger model.
Definition
A reward model is trained on human preference data (pairs of outputs ranked good/bad) and outputs a scalar score for any new response. The main language model is then trained to maximize that reward. It's a critical bottleneck — bad reward models produce bad RLHF.
Examples
- Training data: 10,000 (prompt, better_response, worse_response) triples
- Output: a model that scores "this answer is 0.73 good"
Related terms
RLHF
Reinforcement Learning from Human Feedback — the technique that made ChatGPT helpful instead of just plausible.
Preference Data
Pairs of AI outputs labeled "A is better than B" — the raw fuel for RLHF.
DPO (Direct Preference Optimization)
A simpler alternative to RLHF that skips the reward model and trains directly from preference pairs.
Looking for actual gigs that use Reward Model? Browse all live listings or filter by $100+/hr.