According to a report by Gabriel Pimenta de Freitas Cardoso, Breno de Almeida Beleza, Francisco de Almeida Rocha Alves and Bruno Duarte of Dharma-AI (published August 17, 2026), a change in allocation order — not hardware — produced large utilization and value gains in GPU clusters.
Order as a capacity decision
The authors argue that scheduling is fundamentally a set of binary allocation choices across GPUs, jobs and timesteps. Four workload types compete for those slots: training, real-time inference, batch inference and quantization. Training, batch inference and quantization require contiguous GPU blocks until completion, while real-time inference is elastic and varies each timestep. Under contention, the team says, the sequence in which jobs are placed becomes a capacity decision rather than a mere tiebreaker.
Benchmarks and measured gains
In side-by-side benchmarks against a FIFO scheduler (with real-time inference served from fixed reservations), the Dharma-AI allocator increased utilization by as much as 33 percentage points and improved priority-weighted output in every scenario. Reported results include:
- Mixed control (8 GPUs, 10 jobs): utilization 51.6% → 72.4%, value +54.8% (1 ms latency).
- Training-heavy (8 GPUs, 16 jobs): utilization 53.6% → 87.0%, value +105.1% (2 ms latency).
- Real-time contention (8 GPUs, 8 jobs): utilization 75.0% → 80.2%, value +24.6% (1 ms latency).
- Scale test (64 GPUs, 30 jobs): utilization unchanged at 44.9% but priority-weighted value rose +15.9% (15 ms latency).
- Uniform priority (14 GPUs, 16 jobs): utilization 76.8% → 87.5%, value +23.1% (2 ms latency).
The authors state the allocator ran in 1–2 milliseconds on contended scenarios and 15 milliseconds at the 64-GPU scale, fast enough for per-request use.
Modeling constraints and forecasts
The report describes five hard constraints for legal allocations (one GPU per timestep, demand ranges respected, contiguous blocks for batch-like jobs sized to powers of two, caps on GPU swaps for real-time jobs, and no interruption after a start). The objective function combines priority-weighted rewards for batch allocation and steep penalties for unmet real-time demand; the real-time penalty weight is reported as 5–10× the allocation weight.
The authors emphasize that the scheduler relies on forecasts of job GPU-hours and real-time demand. To mitigate forecast error, the scheduler optimizes a 24-hour horizon but commits only the current timestep and re-runs every 30–60 minutes, so each run inherits what is actually running and adapts to fresh data.
These details describe the allocator’s design and measured performance as reported by the Dharma-AI team.
Original source: Hugging Face Blog