Supervised Fine-tuning

term_id: supervised_fine_tuning

Category: training_techniques

Definition

Supervised Fine-tuning (SFT) involves taking a large pre-trained model, such as a language model, and continuing its training on a smaller, high-quality dataset labeled for a specific downstream task. Unlike initial pre-training which learns general patterns, SFT aligns the model’s behavior with human preferences or specific instructions, significantly improving performance on niche tasks without requiring training from scratch.

Summary

The process of further training a pre-trained model on a specific dataset to adapt it to a particular task or domain.

Key Concepts

  • Pre-trained Models
  • Transfer Learning
  • Instruction Tuning
  • Domain Adaptation

Use Cases

  • Custom chatbot development
  • Specialized medical Q&A systems
  • Code generation assistants

Code Example

1
2
3
4
5
6
7

model.train()
for batch in dataloader:
    inputs, labels = batch
    outputs = model(inputs, labels=labels)
    loss = outputs.loss
    loss.backward()
    optimizer.step()

Related