Reflection has released Beam, an open 501 billion-parameter mixture-of-experts (MoE) language model that teams can run and adapt on their own infrastructure. Beam is text-only, optimized for coding and agentic workflows, and is available under the Apache 2.0 license.
Architecture and capabilities
Beam is sparse: each token is routed to 8 experts out of 256, so inference computes roughly 23 billion parameters per token while keeping all 501B parameters resident in memory. The model has 52 decoder layers, a hidden size of 5,120 and uses a hybrid attention scheme—sliding-window attention for most layers and global attention in nine layers—to support a 1,048,576-token (1M) context window.
The design includes a residual-weighting scheme that preserves a constant-scale residual stream across depth, and several implementation choices performed in FP32 (router, SwiGLU product, norms, RoPE, query scaling and a tanh softcap at ±30). Beam supports five reasoning settings—”low”, “medium” (default), “high”, “xhigh” and “max”—and includes a tokenizer with a response template and an XML-style tool-calling grammar.
Checkpoints, benchmarks and tooling
Checkpoints are provided in BF16 (~1 TB safetensors), FP8 and NVFP4 (the latter reduces BF16 size to ~280 GB for NVIDIA Blackwell GPUs). The Reflection team published benchmark results across precisions; examples include DeepSWE v1.1 (43.3% BF16), SWE Bench Pro v2‑Hard (76.5% BF16) and AIME 2026 (97.7% BF16). Results for many other tasks and precisions were also released by the team.
Beam ships with a Rust-based harness and day‑0 support in transformers (requires “transformers>=5.20.1”), vLLM and SGLang. vLLM and SGLang recipes support DFlash2 speculative decoding with a small drafter model to speed generation. The tokenizer provides parsing helpers that separate reasoning traces, answers and tool calls when used with the model’s chat template.
Hardware guidance from the Reflection team notes NVFP4 is most practical for serving on Blackwell GPUs; alternatives include FP8 checkpoints for older hardware. Examples: two B200 GPUs (180 GB HBM3e each) provide ~360 GB; four B200s (720 GB) leave room for activations and long contexts. Multi‑GPU expert‑parallel loading and vLLM or SGLang serving recipes are provided for deployment.
Beam is released under Apache 2.0, includes open tooling, and the Reflection team is offering $10,000 in Hub credits for selected projects that use the model. The model is also available through hosted Inference Providers and one‑click Inference Endpoint recipes. To enter the credit program, creators must publish a Space, model or dataset repo and share it on the Beam model page.
Original source: Hugging Face Blog