Mistral Large 4 Overview
Mistral Large 4, also known as Le Chonk, is a multimodal, multilingual Mixture-of-Experts (MoE) model that accepts both text and images as input and generates text output. The model features approximately 1 trillion total parameters, with 49 billion active parameters and a context window of 1 million tokens. It is available in two official variants: FP8 and NVFP4, and is distributed under the Mistral Ecosystem Open-Weight License, which is a derivative of the MIT license that imposes restrictions on model-as-a-service offerings.
Architecture and Vision Encoder
The architecture of Mistral Large 4 includes a text decoder with 72 layers, a hidden dimension of 6,144, and 64 attention heads. The attention blocks utilize a combination of Multi-head Latent Attention (MLA) and a DeepSeek Sparse Attention (DSA) indexer, allowing DSA to select up to 2,048 key positions for each query. The model employs a 3:1 RoPE to NoPE pattern across its decoder layers, sharing the KV state across layers. The first decoder layer utilizes a dense feed-forward network, while the remaining 71 layers incorporate MoE blocks, where a softmax router selects four of 128 experts for each token, alongside one shared expert, resulting in five active experts per token. The model also implements LeRope, a variant of RoPE scaled by YaRN.
The vision encoder is a 1.6 billion parameter Pixtral encoder, consisting of 30 layers, a hidden dimension of 2,048, and 16 attention heads. It processes 14 × 14 patches and employs a patch-merge projector with a spatial merge size of 2.
Benchmark Performance and Cybersecurity Capabilities
As of October 29, 2026, Mistral Large 4 ranks among the top open-weight models based on benchmark results. In coding tasks, it achieves scores of 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, and 28.3% on Terminal-Bench 4, resulting in a Coding Agent Index score of 49.8%. Additionally, it scores 59.9% on AutomationBench and an Elo rating of 1,393 on AA-Briefcase.
In terms of cybersecurity, Mistral Large 4 excels with an 82% score on a vulnerability reproduction and patching test within the Artificial Analysis Cyber Index. It also demonstrates strong performance on a suite of 40 challenges in Cybench, solving 93% of them. The model exhibits a 93.3% resistance to attacks on Lakera’s B3 benchmark and has a higher refusal rate for harmful cyber prompts compared to other open-weight models, as evidenced by tests such as JailbreakBench, StrongREJECT, and AgentHarm.
Mistral positions this model primarily for defensive security work, emphasizing its utility in vulnerability research and incident response, which can often resemble attack scenarios in model inputs. This focus on defensive capabilities is crucial, as the model’s open-weight nature allows users to maintain control over usage policies and deployment.
Inference Support and Fine-tuning
Mistral Large 4 includes day-0 support for Transformers, vLLM, and llama.cpp, with GGUF quantizations produced by Unsloth and the llama.cpp team. Fine-tuning is supported via TRL, and deployment recipes are available, including an Inference Endpoints configuration optimized for 8 × NVIDIA H200 GPUs. Reported performance metrics for the endpoint indicate a time-to-first-token of approximately 0.6 seconds for a 1K-token prompt, with generation speeds around 65 tokens per second, increasing to 120–138 tokens per second when speculative drafts are utilized. Metrics for long prompts and performance under load are also detailed in the release notes.
Official checkpoints for Mistral Large 4 can be found in the Mistral Large 4 collection, with FP8 and NVFP4 as the sanctioned variants, alongside community GGUF conversions provided by independent teams.
Original source: Hugging Face Blog