Proximal gradient methods for learning
term_id: proximal_gradient_methods_for_learning
Category: training_techniques
Definition
Proximal gradient methods are iterative optimization techniques used when the loss function includes a differentiable smooth term and a non-differentiable regularizer, such as L1 norm. The algorithm combines gradient descent steps on the smooth part with a proximal operator that handles the non-smooth part. This makes them particularly useful for sparse learning and regularization tasks where traditional gradient descent fails due to non-differentiability.
Summary
Optimization algorithms designed to minimize composite objective functions containing both smooth and non-smooth components.
Key Concepts
- composite optimization
- proximal operator
- L1 regularization
- non-smooth convexity
Use Cases
- Sparse feature selection
- Lasso regression
- Structured prediction models