On November 5, 2026 NVIDIA published four open-weight checkpoints in its Cosmos 3 family aimed at generating interactive world simulations and multi-sensor driving scenes. The new checkpoints are post-trained examples of Cosmos 3 Nano and each contains 16B trainable parameters. NVIDIA states the weights are released under OpenMDW1.1, permitting commercial and non-commercial use under that license.
Four checkpoints and their intended uses
Cosmos3-Nano-Transfer-Auto: a bidirectional model for offline synthetic driving data generation. It can produce synchronized video across up to 11 camera views (including fisheyes) and joint RGB–LiDAR outputs. Per-camera World Scenario Map (WSM) controls and text prompts steer scene layout, traffic motion and appearance; aligned HD-map range controls support LiDAR.
Cosmos3-Nano-Sim-Auto: a causally distilled, streaming variant of Transfer-Auto. It produces successive chunks of synchronized camera video and LiDAR in few denoising steps, using generated history and updated driving controls to enable longer interactive rollouts. NVIDIA describes Transfer-Auto as the starting point for adapting a new sensor rig, followed by distillation into Sim-Auto for fast streaming.
Cosmos3-Nano-Sim-Transfer: converts simulator spatial controls and a text prompt into photorealistic RGB observations while keeping simulator physics and state as ground truth. It supports control channels such as depth, segmentation, edges and blur to guide rendered outputs.
Cosmos3-Nano-Sim-Bimanual: predicts video responses to bimanual robot or camera actions. It supports two-arm embodiments (YAM and AgiBot) and a mixed-action schema that includes coordinate frames, rotations, gripper calibration and validity masks. The published inference actions are normalized for direct use.
Architecture, training recipe and deployment notes
The models use a Mixture-of-Transformers backbone that links text processing and visual generation through attention. Generation runs on compressed video latents (each latent representing four RGB frames), with denoising to refine segments. Temporal-causal attention and a key/value cache maintain history while preventing access to future targets; Transfer predicts one step per chunk while the optimized Bimanual setup can predict two.
NVIDIA describes a four-stage post-training recipe: bidirectional adaptation, teacher forcing, consistency distillation, and self-forcing to expose the model to its own generated history. The company notes remaining challenges: measuring when the models reliably support policy learning and understanding their limits.
Published tooling includes notebooks and inference “skills” for each checkpoint. System requirements listed by NVIDIA are Linux, an NVIDIA GPU, Docker and NVIDIA Container Toolkit; the vLLM-Omni setup was tested on one GB200, a test configuration rather than a minimum memory specification.
Original source: Hugging Face Blog