Transformers maintainers have proposed Model Integration Contracts (MIC) to run fast, repeatable checks that exercise real model APIs on cheap hardware. The MIC concept packages a shrunk model fixture, a runnable usage path and human-reviewed baselines; the contract holds when the usage path on the fixture reproduces those baselines.
Why MICs were introduced
The proposal responds to regressions that reached main despite an extensive test suite. The project runs 88,000+ CPU tests on main and 200,000+ GPU tests in a daily run, but full integration runs on every PR are impractical. As an alternative, MICs use compact fixtures built from real checkpoint configurations to run targeted checks on CPU runners.
One motivating example was a change on July 30, 2026 that enabled PyTorch’s scaled dot-product attention (SDPA) for T5 and, as a side effect, allowed UMT5’s decoder to attend to future tokens. A small UMT5 fixture detected the regression on every replayed commit until the fix on September 28, 2026. The contract measured differences of 1.22 in label log-probabilities and about 0.20 in training loss. The regression was only visible when the test exercised SDPA, not when it used the eager attention variant.
How fixtures are built and used
A MIC has three parts: a fixture (a shrunk but structurally faithful model), a usage path (a runnable Markdown example such as generation or a training step), and baselines (recorded outputs reviewed by a human). Fixtures preserve tokenizers and processors byte-for-byte and keep the model’s layer kinds so the path goes through the same code paths as a full checkpoint.
The shrink recipe follows four steps: reduce configuration sizes while keeping layer types; preserve preprocessing and tokenization; build and validate weights (either seeded random or lightly overfit) and reject builds that do not respond; then freeze and record baselines on a known-good build. Example size contrasts include a UMT5 fixture at 66 MB versus a 1,227 MB source checkpoint, and a Llama 4 Scout shrink from 108.6 billion parameters to 44.7 million, with the vision stack reduced from 34 to 2 layers and the language stack from 48 to 4.
The CLI tool tmic drafts contracts, builds fixtures and records baselines; a local run is possible with pip install “transformers-mic[runtime]” and tmic run org/model –local-only –fixtures-only –no-history –bill-to ”.
Open questions remain: as of November 1, 2026 the MIC job reports failures without blocking PRs, and the project is evaluating how often MICs catch real bugs on PRs, how many false alarms they generate, and which usage paths need coverage. Contributors can try the tmic workflow and contribute usage paths or reviews for uncovered models.
Original source: Hugging Face Blog