Sigmoid

term_id: sigmoid

Category: basic_concepts

Definition

The sigmoid function, defined as σ(z) = 1 / (1 + e^-z), is widely used in machine learning to model probabilities. It squashes input values into the range (0, 1), making it suitable for binary classification output layers. While historically popular in logistic regression and early neural networks, it suffers from the vanishing gradient problem during backpropagation, which can slow down training in deep networks compared to alternatives like ReLU or Leaky ReLU.

Summary

A mathematical function that maps any real-valued number into a value between zero and one, forming an S-shaped curve.

Key Concepts

  • Logistic function
  • Probability mapping
  • Vanishing gradient
  • Non-linearity

Use Cases

  • Binary classification output
  • Logistic regression
  • Activation in shallow neural networks

Code Example

1
2
3

import numpy as np
def sigmoid(z):
    return 1 / (1 + np.exp(-z))

Related