Tanh
Definition
The hyperbolic tangent (Tanh) function is a non-linear activation function commonly used …
Deep Learning
Terms tagged with Deep Learning
Similarity learning
Definition
Similarity learning focuses on training models to map inputs into a vector space where …
Sentence Transformers
Definition
Sentence Transformers are extensions of traditional Transformer models (like BERT) …
Reparameterization trick
Definition
The reparameterization trick is a fundamental method used in variational autoencoders and …
Pyannote Audio
Definition
Pyannote Audio is a comprehensive toolkit designed to facilitate the development and …
Product of experts
Definition
The Product of Experts (PoE) is a method for constructing complex probability …
Multimodal representation learning
Definition
Multimodal representation learning involves training models to process and integrate …
Mode Collapse
Definition
In GANs, mode collapse occurs when the generator learns to exploit weaknesses in the …
Manifold hypothesis
Definition
This hypothesis explains why deep learning works effectively despite the curse of …
Highway network
Definition
Highway Networks are designed to address the vanishing gradient problem in deep learning …
Hidden Layer
Definition
A hidden layer consists of neurons that receive inputs from previous layers, apply …
Hardware for artificial intelligence
Definition
AI hardware refers to specialized computing devices optimized for the massive parallel …
Gradient Accumulation
Definition
This optimization strategy allows deep learning models to be trained with effective batch …
Gated Recurrent Unit
Definition
A Gated Recurrent Unit (GRU) is a specialized recurrent neural network (RNN) cell …
Feature learning
Definition
Feature learning, often associated with deep learning, enables models to learn …
Energy-based model
Definition
Energy-Based Models (EBMs) define a probability distribution over input data using an …
Domain Adaptation
Definition
Domain adaptation addresses the challenge when training and testing data come from …
Double Descent
Definition
Double descent challenges the traditional bias-variance tradeoff by showing that highly …
Differentially private stochastic gradient descent
Definition
DP-SGD is a variant of Stochastic Gradient Descent designed to protect the privacy of …
Bert
Definition
BERT is a transformer-based machine learning technique for NLP pre-training developed by …
Batch Normalization
Definition
This method adjusts and scales activations to have zero mean and unit variance within …
AlphaChip
Definition
AlphaChip is a specialized AI system designed to automate and enhance the placement and …
Adversarial Attack
Definition
Adversarial attacks exploit the vulnerabilities of neural networks by introducing subtle …
Vision
Definition
Computer Vision (CV) is a branch of artificial intelligence that trains computers to …
Recurrent Neural Network
Definition
RNNs are designed to recognize patterns in sequences of data, such as text, genomes, …
ReLU
Definition
ReLU is widely used in deep learning neural networks due to its computational efficiency …
Residual Connection
Definition
Residual connections, also known as skip connections, allow gradients to flow through a …
Long Short-Term Memory
Definition
LSTM networks address the vanishing gradient problem common in standard RNNs by using a …
Dropout
Definition
In neural networks, dropout prevents overfitting by temporarily removing a random subset …
Activation Function
Definition
An activation function introduces non-linearity into a neural network, allowing it to …
pre-trained
Definition
A pre-trained model is a foundational AI model that has undergone extensive training on …
diffusion-based
Definition
Diffusion-based models are a class of generative AI that create new data samples by …
Transfer Learning
Definition
Transfer learning leverages pre-trained models to improve performance and reduce training …
Pre-training
Definition
Pre-training is a foundational technique in deep learning where a model learns broad …
Neural Network
Definition
A neural network is a series of algorithms that endeavors to recognize underlying …
Multi-Head Attention
Definition
Multi-Head Attention extends the standard attention mechanism by running it multiple …
Large Language Model
Definition
Large Language Models (LLMs) are advanced artificial intelligence systems based on …
Diffusion
Definition
Diffusion models are a class of generative AI that learn to reverse a stochastic process …
Adam
Definition
Adam (Adaptive Moment Estimation) is a popular first-order gradient-based optimization …
Fine-tuning
Definition
Fine-tuning involves taking a model already trained on a large, general dataset and …
Convolutional Neural Network
Definition
Convolutional Neural Networks (CNNs) are designed to automatically and adaptively learn …
Attention Mechanism
Definition
An attention mechanism enables a model to weigh the importance of different elements …