International access: https://4sapi.com
Domestic access: https://4sapi.org
On October 22, OpenAI announced the general availability (GA) of the Decisions API, built on top of the GPT-6 Luna foundation model. This release marks a clear divergence from the design philosophy behind mainstream generative interfaces such as the Responses API. Instead of supporting open-ended text composition, the Decisions API is purpose-built for bounded classification tasks. It returns structured, quantitative outputs rather than natural language paragraphs, making it suited for routing logic, risk screening, scoring pipelines, and other decision points embedded deep within automated agent stacks.
The API defines three standardized output schemas for developers to select according to business requirements. The first output variant, predicate, returns a floating-point probability value ranging from 0 to 1, which quantifies the likelihood that a predefined condition evaluates to true. The second variant, choice, requires developers to supply a fixed list of candidate options in advance; the model will select one candidate and attach a probability distribution plus a confidence metric for every available option. The third variant is score, which produces a weighted composite score following a predefined rating scale from 2 to 10. All three return formats are machine-native and ready for downstream programmatic consumption. Applications can trigger branching logic directly using these numeric outputs, removing the overhead of parsing unstructured free text responses.
The most prominent performance metric of Decisions API is latency. Under identical input conditions, inference through Decisions API can run up to 10 times faster than invoking GPT-6 Luna via the Responses API. This massive speed gain comes from architectural optimization: the runtime skips the token-by-token autoregressive generation of verbose text, and directly emits constrained structured results from the model’s internal latent representations. This optimization is not a minor tweak. It unlocks a category of use cases that were previously impractical with conventional chat and generation APIs, especially low-latency, high-volume decision gates inside production agents.
Pricing for Decisions API follows an aggressive, usage-aligned model optimized for batch workloads. Billing applies only to input tokens, with all output tokens exempt from charges. The base rate sits at $0.10 per million input tokens. When a request hits the built-in prompt cache, the cost drops sharply to $0.01 per million input tokens. Each single API request supports packaging up to 400 independent decision questions in one batch call. This batching capability, combined with the cache discount, drastically reduces marginal cost for workloads that repeatedly run similar judgment rules.
The GA release expands the feature set compared with the beta preview. New native integrations include compatibility with GPT-Live streaming workflows. The service also supports reusable prompt caching for persistent context segments. Data residency controls are now available for inference workloads hosted within the United States and European regions, addressing data compliance requirements for regulated enterprise clients. Native JSON input parsing and inline image support within individual decision questions are enabled out of the box. OpenAI has also expanded compute capacity across multiple zones in Asia to lower regional latency for developers operating within this market.
It is critical to clarify the boundary between Decisions API and the existing Responses API. The two interfaces are complementary rather than mutually replacing. Developers should route requests to Responses API when workflows require flexible custom JSON schemas or tool calling initiated autonomously by the model, leveraging its Structured Outputs and Function Calling modules. Decisions API takes ownership of bounded judgment tasks where the space of valid outputs is fully predefined. This separation of responsibilities creates a clean layered architecture for agent development. Many teams route lightweight classification and routing checks through Decisions API, and reserve the heavier generation and tool-use work for Responses API.
Real-world benchmark data from early enterprise adopters demonstrates tangible performance improvements across different verticals. Box deployed the API for traffic selection logic and recorded approximately a 5× speedup on routing decisions. Datadog integrated Decisions API as a pre-execution risk evaluator for shell commands produced by internal agents; the p95 latency of the risk-check step improved by 40×. Sierra, a customer service automation platform, uses the API to pick the next action inside conversational agent threads. Its p95 latency for this decision step stays below 280 milliseconds, and the per-call operational cost falls by 90 percent. Speak, an interactive voice assistant product, reduced the end-to-end latency for next-action selection from 1.4 seconds down to 0.13 seconds after switching its decision gate to Decisions API.
These case studies illustrate a consistent pattern. The Decisions API is not built to replace the creative or reasoning capacity of large generative models. It acts as a lightweight judgment layer inserted before expensive generation stages. In a typical agent pipeline, the system first runs cheap, fast predicates and choice classifications. Only when the decision output confirms that natural language generation or external tool invocation is necessary does the workflow forward the context to a full Responses API call. This pattern cuts both latency and total token consumption, because expensive generation cycles are avoided for the majority of trivial classification cases.
From an architectural perspective, this design solves a long-standing pain point for agent orchestration platforms. Many agent workflows contain dozens of micro-judgment points: determining whether user input contains sensitive content, selecting which downstream model variant to invoke, prioritizing support tickets, deciding whether to escalate a conversation to a human agent, validating extracted field values, and so on. When these judgments run through standard chat completions, the system wastes resources generating explanatory prose that downstream code ignores. Decisions API strips away that overhead entirely, delivering only the numeric signal the application requires.
For teams managing multi-model stacks, consistent routing and credential handling become operational overhead at scale. An API gateway can standardize request formatting, cache reusable prompts, and consolidate observability across different OpenAI endpoints and third-party model providers. Teams building agent pipelines with mixed Responses and Decisions API calls can simplify traffic management with 4sapi, which unifies routing rules and logging for heterogeneous OpenAI workloads.
Three core strategic takeaways frame the significance of this launch for AI engineering teams.
First, the product positions itself explicitly as the lightweight decision tier inside agent orchestration stacks. Its core use cases cover routing, compliance screening, confidence-based traffic splitting, dynamic model selection, and action selection. The design principle is resource segregation: allocate costly autoregressive generation capacity exclusively to nodes where expressive natural language output is mandatory. All binary checks, ranked choices, and numeric scoring tasks are offloaded onto this specialized interface. This layered design aligns resource expenditure with business value.
Second, the pricing structure is engineered to capture high-frequency batch decision traffic. By charging solely for input tokens and waiving output fees, OpenAI removes the economic penalty for batch-heavy workloads that emit many tiny structured outputs. The cache discount further incentivizes workloads with repeated base prompts, which are extremely common in rule-based agent gates. This creates a clear price ladder: low-margin, high-volume micro-decisions run on Decisions API, while long-context, complex reasoning and content generation remain on higher-priced general-purpose models. This tiered pricing makes it economically feasible to embed hundreds of decision points into every agent session.
Third, the release signals a broader industry shift in AI application design. Development patterns are moving away from monolithic single-turn chat interactions. Modern AI systems embed large numbers of tiny, low-latency model judgments directly within existing business process flows. For these embedded decision points, latency and per-invocation cost matter far more than raw model parameter count or maximum context window size. A larger, more capable general model will not be the optimal choice if the task is simply selecting between three predefined actions with strict latency limits. Specialized inference interfaces will continue to proliferate to match these granular embedded workloads.
Developers evaluating whether to adopt Decisions API need to assess the constraints of its execution model. All valid outputs must be bounded and known in advance. If the business logic requires open-ended discovery, dynamic schema adaptation, or multi-step tool chaining triggered by the model itself, Decisions API is not the correct fit. It is also important to plan for batch grouping. The 400-question per-request batch limit is a powerful lever for cost reduction, but application logic must be adjusted to aggregate independent decision jobs before sending them upstream. Poor batching discipline will leave significant performance and cost savings on the table.
Data residency controls also deserve attention for compliance-focused teams. Workloads that fall under regional data governance rules can be pinned to US or EU inference zones. Teams handling user data in Asia can benefit from the recent capacity rollout, which reduces round-trip latency compared to routing all traffic to western regions. When building global agent systems, combining residency configuration with regional routing logic helps maintain both performance and regulatory alignment.
The prompt caching mechanism is another high-impact feature. For decision gates that reuse the same instruction template across thousands of requests, the fixed instruction segment can be cached. Only the variable input payload changes on each call. This cuts the billed input token count dramatically. For example, a ticket triage decision prompt with a 1200-token fixed rule set and a 300-token variable user message would only charge for the 300-token variable segment after the initial cache fill. This makes recurring classification pipelines extremely economical.
When designing hybrid agent workflows that combine Decisions API and Responses API, observability is a key consideration. Engineering teams need separate metrics for decision latency, decision confidence distribution, cache hit ratio, and downstream generation trigger rate. Tracking how often a positive predicate result leads to a full generation call reveals the efficiency gains of the decision layer. If the majority of decision calls route to a “no-generation” branch, the pipeline is operating as intended and eliminating unnecessary compute.
Looking ahead, this specialized API creates a blueprint for future model interface segmentation. Foundation model providers will likely continue splitting capabilities into purpose-built endpoints optimized for latency, cost, or specific output modalities, rather than forcing every workload onto a universal chat completion interface. Agent developers will increasingly build systems as composable stacks: fast bounded judgment layers, general reasoning layers, multimodal extractors, tool execution modules, and retrieval components, each served by the most suitable endpoint.
The Decisions API GA is not merely a feature update to GPT-6 Luna. It formalizes a new primitive for agent engineering: the bounded probabilistic micro-decision. It gives developers a standardized, low-cost, low-latency way to inject model judgment deep into application logic without surrendering control to unconstrained text generation. For teams building production-grade autonomous agents, customer service automation, risk screening pipelines, and intelligent routing systems, this interface delivers a practical way to reduce latency, cut token spend, and make agent behavior more deterministic and auditable.
As adoption expands, developer tooling around this pattern will mature. Standard libraries for batch question assembly, confidence threshold tuning, cache monitoring, and fallback routing will reduce integration friction. The separation between decision logic and generative logic will become a standard architectural pattern, much like the separation between retrieval and generation in RAG pipelines.
In summary, OpenAI’s Decisions API fills a previously underserved niche in the LLM ecosystem. It trades general expressive power for deterministic structured outputs, extreme latency reduction, and a pricing model built for high-volume embedded decisions. It works best as one component inside a broader agent stack, paired with general-purpose generation APIs for tasks that demand open-ended reasoning and text output. The benchmark figures from Box, Datadog, Sierra and Speak validate that the performance and economic gains are substantial for workloads that fit its bounded-output design. For teams designing agent workflows with dozens of inline classification gates, this new GA endpoint represents a major optimization lever.
