OpenAI ran Parameter Golf to test how researchers approach a tightly constrained machine learning problem. Over eight weeks the challenge drew more than 1,000 participants and over 2,000 submissions. Entrants had to minimize held-out loss on the FineWeb dataset while keeping model weights and training code within a 16 MB artifact limit and a 10-minute training budget on 8×H100s, OpenAI reported.
Technical patterns and standout submissions
OpenAI said many top-performing entries arrived through careful tuning, compression, and inventive model or data tweaks. Examples highlighted by the organizers include @notapplica (#60), who combined prior improvements and used Muon weight decay, spectral embedding initialization, residual-mix scheduling, and compiled evaluation to make a deeper model work; @signalrush (#414), which applied GPTQ-lite for post-training quantization; and @dexhunter (#1060), who extended earlier work to use full Hessian GPTQ.
Organizers also flagged approaches that blurred the line between model advances and evaluation strategy. OpenAI cited @samacqua (#77) for a score-first, per-document LoRA test-time training method and @abaybektursun (#1019) for a self-generated GPTQ calibration technique. On the modeling side, submissions such as @romeerp (#1729) with the CaseOps tokenizer, @unnir (#265) with XSA, @aquariouseworkman (#65) with SmearGate and BigramHash, and @msisovic (#1204) with mini depth recurrence were singled out as creative data- or architecture-focused ideas.
Agents, community effects and operational lessons
OpenAI said the majority of participants used AI coding agents. According to the post, agents reduced setup cost, sped experimentation, and broadened access, while also producing new challenges for submission review, attribution, and scoring. The volume of entries required automated triage; organizers developed an internal Codex-based bot to flag submissions for human review.
The nonrecord track encouraged experimentation: OpenAI noted several favorites—CiprianFlorim-Ifrim’s combination state-space model and JEPA, ddavidgao’s Designator/Guided Attention, and DariusFeher’s Byte-Level H-Net—and reported that half of nonrecord entries beat the naive 1.22 BPB baseline, with the top nonrecord result reaching 1.12 BPB.
OpenAI also credited Runpod’s $1,000,000 compute sponsorship for widening participation. The organization said it is considering more challenges of this form and invited interested researchers to complete a participant form.
Original source: OpenAI News