P-Tuning
term_id: p_tuning
Category: training_techniques
Definition
P-Tuning (Prompt Tuning) is a technique designed to adapt large pre-trained language models to specific downstream tasks with minimal computational cost. Instead of fine-tuning all model parameters, it introduces trainable virtual tokens (embeddings) at the input layer. The pre-trained model’s weights remain frozen, and only these prompt embeddings are updated during training. This approach significantly reduces memory usage and training time while maintaining performance comparable to full fine-tuning on many NLP tasks.
Summary
P-Tuning is a parameter-efficient fine-tuning method that optimizes continuous prompt embeddings rather than updating the entire pre-trained model weights.
Key Concepts
- Parameter-Efficient Fine-Tuning
- Virtual Tokens
- Frozen Weights
- Embedding Optimization
Use Cases
- Few-shot learning adaptation
- Resource-constrained environments
- Rapid prototyping of LLM applications