What PyTorch 2.15 introduces
The PyTorch team at Hugging Face reports that PyTorch 2.15 expands the MPS backend on Apple silicon with several changes: FP8 support on MPS, faster language‑model decoding, and a broad migration of operations from MPSGraph to dedicated Metal kernels.
FP8 is exposed as torch.float8_e4m3fn on MPS, using one byte per value and about half the storage of FP16/BF16. The release adds F.scaled_mm for scaled matrix multiplication of FP8 tensors on the GPU.
Performance and workload highlights
Language‑model decoding improves across dense and mixture‑of‑experts (MoE) layers. The PyTorch team reports MoE models decode 5–6.5× faster than in 2.14: for example, Qwen3.6-35B-A3B increases from 9.4 to 61.3 tokens/s on an M3 Ultra, and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s when using torch.compile. Single‑row linear layers now use specialized matrix‑vector kernels, and softmax over large vocabularies runs on dedicated Metal kernels (an FP32 softmax over a [1, 128256] row drops from ~57 µs to ~7 µs).
Linear algebra and training primitives also moved to GPU paths: complex LU and triangular solves for complex64, faster small‑matrix inverses, QR improvements, and more efficient batched triangular solves for certain sizes. Convolutions, padding, dropout and upsampling backward receive Metal implementations intended to reduce driver memory and improve correctness.
A striking example cited by the team concerns Wan‑AI/Wan2.2‑TI2V‑5B‑Diffusers: switching constant padding to Metal reduced total driver‑memory peak from about 270 GiB to roughly 8 GiB during a tiled VAE decode that previously made 8,764 padding calls.
Control, compatibility and next steps
Python APIs gain more control over streams: torch.mps.Stream and the torch.mps.stream context manager allow explicit stream selection and synchronization, with Tensor.record_stream for allocator tracking.
Updated requirements for this release are concrete: MPS now requires macOS 15 or later, and PyTorch 2.15 requires Python 3.11 or later. The PyTorch team at Hugging Face also previews further work planned for 2.16, including moving attention fully onto Metal and additional MPSGraph→Metal migrations, with reported token‑throughput and latency improvements on several models and shapes.
Original source: Hugging Face Blog