Softmax

term_id: softmax

Category: basic_concepts

Definition

Softmax is widely used in the output layer of neural networks for multi-class classification tasks. It takes a vector of raw logits and normalizes them so that each element represents a probability between 0 and 1, and all elements sum to 1. This allows the model to express confidence levels across mutually exclusive classes, making it essential for interpreting final predictions in classification models.

Summary

A mathematical function that converts a vector of arbitrary real-valued scores into a probability distribution.

Key Concepts

  • Probability Distribution
  • Logits
  • Normalization
  • Multi-class Classification

Use Cases

  • Image classification output layers
  • Language model token prediction
  • Multi-label categorization

Code Example

1
2
3
4
5

import torch
import torch.nn.functional as F
logits = torch.tensor([1.0, 2.0, 3.0])
probs = F.softmax(logits, dim=0)
print(probs)

Related