Improved Prompt Caching for GPT-6
OpenAI has introduced an enhanced prompt caching system for GPT-6, designed to increase the reuse of shared prompt context and significantly reduce computational costs for repeated requests. This new system offers cache discounts for eligible shared prefixes reused within a 30-minute window, potentially delivering discounts of up to 90% on cached input tokens.
Monitoring and Diagnosing Cache Behavior
The new Prompt Caching Dashboard provides insights into how much of an application’s input is served from cache, tracks hit rates over time, and includes an input composition chart for comparing cached and uncached tokens. Additionally, a dedicated prompt caching diagnostics tool allows developers to investigate unexpected cache misses by comparing a request with a recent response. This tool reports reasons for misses, such as changes in tools, model settings, or inputs, and includes an estimate of affected tokens to help teams assess the impact before making adjustments.
Controls to Preserve and Tune Cached Context
Developers can now specify which prompt prefixes to reuse through explicit cache breakpoints and follow guidelines on the eligibility duration of cached prefixes. On GPT-6 models, it is possible to adjust reasoning effort between responses without breaking the cache by appending a configuration_update. This allows for modifications in task complexity while maintaining reusable context.
The update also includes recommendations for preserving cache as tool usage evolves: maintaining stable tool definitions, schemas, and ordering; using allowed_tools to limit callable tools; setting tool_choice to none when no tools are required; and appending new instructions with developer messages at the end of the context to override older ones.
Prewarming Cache to Reduce Latency
OpenAI describes prewarming as a method to prepare known context—such as shared instructions or tool definitions—before the user’s first request, thereby reducing latency. These controls are designed to enhance the engine’s default performance and tailor caching to specific workloads.
Impact on Developers
Mario Rodriguez, Chief Product Officer, noted that GitHub Copilot has successfully reduced the share of prompt tokens requiring fresh processing by over 50% across billions of requests compared to previous baselines. Arian Hanifi, Chief Technology Officer at Strawberry Browser Manus Wordsmith, reported that the diagnostics and dashboard have improved cache hit rates by several percentage points and reduced costs by 20% in their operations.
Getting Started
Developers can monitor cache hit rates using the Prompt Caching Dashboard and utilize the diagnostics tool to investigate any misses. For implementation details, they should consult the prompt caching guide.
Original source: OpenAI News