Released on August 25, 2026, Granite 4.2 is a reasoning-focused iteration of the Granite family delivered in three sizes: 3B, 8B and 30B. The Granite team at IBM describes the models as dense, decoder-only transformers that are pre-trained from scratch and then refined through supervised fine-tuning and a multi-stage reinforcement learning pipeline. All models are distributed under the Apache 2.0 license.
Architecture and pre-training
The models use a decoder-only transformer with Grouped Query Attention (GQA) configured for 40 attention heads and 8 KV heads, Rotary Position Embedding (RoPE) with θ = 10,000,000, SwiGLU activations in the MLP, RMSNorm (ε = 1e-5), separate input/output embeddings, and bfloat16 precision. Each size shares the same design but differs in scale: embedding sizes (2560 / 4096 / 4096), layers (40 / 40 / 64), attention head sizes (64 / 128 / 128), attention head counts (40 / 32 / 32), KV heads (8 / 8 / 8), MLP hidden sizes (8192 / 12800 / 32768), sequence length (131,072), and parameter counts (3B / 8B / 30B).
Pre-training was performed on roughly 15 trillion tokens using a five-phase approach. Phases 1–2 establish a foundation, phases 3–4 progressively increase data quality, and phase 5 extends long-context training to a 512K token window.
Supervised fine-tuning and staged RL
Supervised fine-tuning (SFT) used about 7.2 million samples (≈100B tokens, ~65B trainable) with an agentic/non-agentic split of 31.6% / 68.4%. The agentic portion emphasizes software engineering (SWE 69%), tool calling (12.1%), terminal use (8.0%), and smaller shares for math, search and action. Agent trajectories and harnesses cited include OpenHands, OpenCode, Terminus-2, SWE-agent, OpenResearcher, MiniSWE, OpenSeeker, EnvScaler, Gemini CLI, Hermes, Codex and Goose. Non-agentic categories included instruction following (18.8%), coding (18.8%), math (14.6%), multilingual (7.0%), science (5.4%), reasoning (3.0%) and safety (0.8%).
Data quality control used LLM judges GPT-OSS-120B and Gemma 4, removal of hallucinated or invalid samples, heuristic filters, and SHA-256 deduplication over tools and messages. Final SFT runs used 32–128 nodes (by model size), 4× Grace/GB200 per node, packed sequence length 131,072, global batch 128, and learning rates of 1.0e-5 (constant after warm-up) with 3.0e-6 for Phase 2.
After SFT, a multi-stage reinforcement learning curriculum was applied via asynchronous GRPO. Stages include foundational RL (RLVR and skill boosters) for all sizes and an agentic RL block (SWE → Terminal → Search) for 8B and 30B only; the 3B model omits the agentic block. The pipeline uses verifiable, judge-based, and agentic-outcome rewards and concludes with RLHF. When served through an OpenAI-compatible endpoint such as vLLM, models emit OpenAI function-calling format and support native tool calling and SGLang serving recipes.
All Granite 4.2 models are released under the Apache 2.0 license. Published August 25, 2026.
Original source: Hugging Face Blog