Grok 1
term_id: grok_1
Category: basic_concepts
Definition
Grok-1 is the inaugural release from xAI, launched in November 2023. It is a decoder-only transformer-based large language model with approximately 33 billion parameters. Notably, it utilizes a Mixture-of-Experts (MoE) architecture, which allows it to activate only a subset of its total parameters for each token, improving efficiency. It was trained on a diverse dataset including public web data and real-time posts from X, serving as the foundation for subsequent iterations like Grok-2.
Summary
The first version of the Grok language model released by xAI, featuring a Mixture-of-Experts architecture and a 33 billion parameter count.
Key Concepts
- Mixture-of-Experts
- Transformer Architecture
- Parameter Count
- Decoder-only Model
Use Cases
- Text generation and completion
- Code synthesis
- Complex reasoning tasks