OpenAI’s New Decisions API
At OpenAI’s Dev Day, CEO Sam Altman announced the launch of the Decisions API, which allows models to be presented with a predefined set of choices, enabling them to return probabilities quickly. Altman explained that this approach allows the lab’s Luna model to select between options, such as image categories or agent behaviors, while maintaining speed and capabilities like image understanding, broad language support, and safety protections. The API is currently available as a limited preview.
Comparison with Jev and Industry Response
The Decisions API has drawn immediate comparisons to Jev, a model released earlier this month by TypeSafe AI, designed for software automation and capable of outputting probability scores for choices at low cost and high speed. TypeSafe CEO Diogo Almeida, a former OpenAI engineer and co-inventor of reinforcement learning, humorously referred to the situation as a “clone wars” on X, suggesting that OpenAI’s interest in such technology could indicate a shift towards building in a “System One” compatible manner.
Applications and Calibration Questions
Observers have noted that decision-focused models like Jev and the Decisions API are cheaper and faster than general-purpose large language models (LLMs). However, a key question remains regarding how well these decision models’ probability outputs will align with real-world decisions. Almeida highlighted the importance of synthetic data in calibrating these outputs.
One practical application of these models is in monitoring agentic actions. OpenAI has indicated that it employs a separate model to monitor for problematic agent behavior at a “significant compute cost.” Shapor Naghibzadeh from QueryStory developed a demo during a recent hackathon using Jev to evaluate each agent action against its assigned task, blocking high-confidence bad actions, flagging others for review, and permitting the rest. He reported that this monitoring cost $2.94 with Jev compared to $372 with a frontier LLM in his tests.
As the Decisions API remains in limited preview, TypeSafe did not respond to inquiries regarding OpenAI’s announcement. However, discussions on X indicate ongoing developer interest, with multiple firms launching similar decision-focused models.
Original source: TechCrunch AI