Empirical risk minimization
term_id: empirical_risk_minimization
Category: basic_concepts
Definition
Empirical Risk Minimization (ERM) is the standard objective function for training supervised learning models. It involves selecting a hypothesis from a class of functions that minimizes the average error (loss) calculated on the available training dataset. While ERM aims to fit the data well, it must be balanced with regularization techniques to prevent overfitting, ensuring that the model generalizes effectively to unseen data rather than merely memorizing noise in the training set.
Summary
A fundamental principle in machine learning that seeks to minimize the average loss over the observed training data.
Key Concepts
- Loss function
- Training error
- Generalization
- Overfitting
Use Cases
- Training logistic regression models
- Optimizing neural network weights
- Support Vector Machine formulation