NVIDIA Introduces Nemotron 3.5 Bolt 24GB
NVIDIA has launched Nemotron 3.5 Bolt 24GB, a streamlined version of the Nemotron 3.5 Super VL, specifically designed for systems with a 24GB GPU memory limit. This model compresses the original Nemotron 3.5 Super VL from 120 billion total parameters (12 billion active) to 74 billion total parameters (9 billion active), allowing it to operate efficiently within the specified memory constraints.
Technical Specifications
The Bolt variant maintains support for both text and vision tasks, ensuring versatility in its applications. The adjustments in parameter counts are aimed at optimizing performance for local execution on compatible NVIDIA hardware.
Performance Metrics
NVIDIA reports that the Bolt model achieves a throughput of 84 tokens per second when running on RTX Spark hardware. This performance metric highlights its capability for handling agentic tasks locally.
Availability and Resources
NVIDIA has made model checkpoints, GGUF files, and a speculative-decoding drafter available on Hugging Face, facilitating easy access for users. The release notes detail the packaging options and available resources, although they do not provide extensive information on deployment templates or downstream integrations.
The Nemotron 3.5 Bolt is positioned as a tailored solution for users requiring a 24GB memory profile, with all performance figures and specifications reflecting NVIDIA’s own measurements.
Original source: Product Hunt