Preference learning
term_id: preference_learning
Category: training_techniques
Definition
Preference learning focuses on teaching models to distinguish between good and bad outputs based on human judgments rather than absolute labels. It typically involves collecting pairs of responses where humans indicate their preferred option. Algorithms then optimize the model to maximize the likelihood of generating preferred responses. This is crucial for aligning large language models with human values, improving safety, and enhancing relevance in conversational AI systems.
Summary
A technique that trains models to align outputs with human preferences using comparative feedback.
Key Concepts
- Human feedback
- Pairwise comparison
- Reward modeling
- Alignment
Use Cases
- RLHF for LLMs
- Recommendation systems
- Content moderation filtering