Multiverse Computing’s paper presents Quantization-Aware Healing (QAH), a recovery procedure for models that have undergone both structural compression and weight quantization. QAH trains a compressed, 4-bit student by distilling directly from the original full-size, full-precision teacher rather than from a recovered bfloat16 checkpoint, a choice the authors say removes an accuracy ceiling imposed by previous methods.
How QAH differs from existing methods
The standard pipeline compresses an architecture, quantizes its weights, then applies a healing step. The dominant healing method, quantization-aware training (QAT), inserts fake-quantization into the forward pass and fine-tunes with a task loss. An alternative, quantization-aware distillation (QAD), distills from a frozen full-precision teacher but assumes the teacher and student share the same architecture.
The paper argues that when compression changes architecture (fewer layers, heads, or neurons), distilling from the recovered bfloat16 checkpoint anchors the student to a degraded target. QAH instead uses the original pre-compression model as teacher and matches its output distribution with a KL-divergence loss on logits; under QAH teacher and student may differ in size or shape.
Reported results and training behavior
Applied to a GPT-OSS 120B model compressed to 60B and quantized to MXFP4, the QAH 60B MXFP4 model outperformed the recovered 60B bfloat16 checkpoint on 7 of 9 benchmarks, according to the paper. Reported gains include +7.4 on AA-LCR (long-context reasoning) and +5.6 on AIME 2025 (math). Two benchmarks, MMLU-Pro and SciCode, trailed the bfloat16 source by under 1.5 points.
The paper also compares QAH with QAT on a GPT-OSS 9B model quantized to MXFP4. QAH reached peak performance in roughly 100 steps versus about 700 for QAT, and remained stable thereafter; QAT collapsed after its peak, losing nearly 19 points by step 1,200, the authors report.
The authors state that at 4-bit precision the QAH model uses roughly four times less weight memory than the bfloat16 student and, at half the teacher’s parameter count, about half the compute per token compared with the 120B teacher. For families that ship in bfloat16, they note the combined reduction could approach eightfold compute savings.
The paper, published August 25, 2026, includes the full healing pipeline, the chunked KL implementation used for long-context (up to 32k tokens) distillation, and distributed-training findings.
Original source: Hugging Face Blog