Contrastive Language–Image Pre-training
term_id: contrastive_languageimage_pre_training
Category: training_techniques
Definition
Contrastive Language–Image Pre-training (CLIP) is a neural network architecture trained on images and their corresponding captions from the internet. It uses a contrastive objective to maximize the cosine similarity between matching image-text pairs while minimizing it for non-matching pairs. This allows the model to understand visual concepts through natural language, enabling zero-shot classification and powerful image-text retrieval capabilities without task-specific fine-tuning.
Summary
A multimodal pre-training method that aligns image and text representations using contrastive loss functions.
Key Concepts
- Multimodal Learning
- Cosine Similarity
- Zero-shot Classification
- Encoder Architecture
Use Cases
- Image search engines
- Text-to-image generation guidance
- Visual question answering