OpenAI builds Pando to scale Git for Codex-driven development

OpenAI developed Pando to address rapid growth in its monorepo activity as Codex enabled more parallel development. The company reports quarterly commits rose sharply relative to Q1 2024—reaching an estimated +2,442% by Q4 2026—and that existing Git infrastructure increasingly experienced slow fetches, failed merges and reliability dips below 90%.

Design goals and trade-offs

Pando was designed with two primary goals: support horizontally scalable writes so many branches can be pushed in parallel, and maximize read throughput so clients can download data at line rate. OpenAI retained compatibility with the Git protocol and client so engineers and agents could continue existing workflows.

To meet those goals, Pando departs from assumptions common in many Git services. Rather than imposing a single global ordering for all ref updates or generating a bespoke packfile for each client, Pando adopts a relaxed consistency model and reuses packfiles. OpenAI says this approach provides serializability for updates to individual refs while allowing unrelated branches to progress concurrently.

How Pando works and how it differs from other approaches

Pando stores incoming packfiles as immutable blobs in blob storage and records ref versions in a horizontally scalable metadata store. Each ref version records the new tip, the pack that supplied it and any prior ref versions it depends on. Fetches reconstruct the client-visible state by replaying the relevant ref versions. OpenAI describes this as causal consistency: a reader sees the history produced by the writer that created a given ref version but cannot assume a single global point-in-time snapshot across all refs.

The post contrasts Pando with two conventional designs. In a three-phase commit (3PC) model, replicas keep full copies of the repository and require quorum coordination for ref updates, which serializes unrelated pushes and places replica I/O on the push path. In a write-ahead log (WAL) model, servers upload packfiles to blob storage and update a shared manifest via compare-and-swap, which makes the manifest update a bottleneck as write volume grows. Pando aims to avoid those bottlenecks by decoupling pack storage from ref metadata and reusing packfiles across readers.

OpenAI reports that Pando now powers all development in its monorepo, cuts median push latency nearly 5× in production and delivers nearly a petabyte of Git data per day. The company also says it will continue contributing improvements back to upstream Git.


Original source: OpenAI News

Leave a Comment