Optiver measures the effect of changes on trading within hours, so its AI platform must support fast decisions, internal evaluation, and quick deployment.
2
An inference gateway, internal golden sets, and a model- and harness-agnostic meta harness let Optiver adopt new frontier models without depending on one provider.
3
Optiver reduced waste by controlling default context, improving caching, exposing team-level spend, using open-source models where suitable, and routing work between models.
Summary
Noyan Tokgozoglu explains why Optiver built its own on-prem AI platform. The firm makes markets with its own money, and changes rolled out to traders can affect P&L within hours. That creates pressure to test new models quickly, measure them on internal workloads, and keep systems available across global regions. Optiver uses an inference gateway to route requests, track costs, set budgets, and switch providers. New models run against customer trading teams' golden sets before wider adoption. A meta harness gives teams portability across model and agent frameworks. Tokgozoglu also describes how spending rose after major model releases and how the company brought it under control by reducing oversized context, improving MCP and cache use, exposing budgets, testing open-source models, and routing tasks to cheaper models. The platform is built around self-service, but AI engineers work directly with traders and improve the shared tools based on their workflows.
Optiver is a market maker that trades with its own money and has no clients. Its systems process more than 10 million trades per day, price more than a million instruments, and handle petabytes of data daily. Unlike a typical technology company that may wait months or quarters for revenue data, Optiver can roll a change out to traders and quants and see its P&L impact by the end of the day. That short feedback loop makes missed opportunities expensive. Tokgozoglu says the firm therefore needs to decide quickly, measure changes efficiently, and keep a bias for action.
The platform is built for speed, measurement, and self-service
Optiver's platform principles are to move fast, stay modular, and measure each layer. New models and new patterns such as loops, swarms, or graphs can be tested and adopted when they add value. External benchmarks are insufficient because Optiver's internal data distribution differs from public benchmark data, so the company uses internal evaluations. Traders get self-service tools with sensible defaults. AI engineers also work directly with trading teams, learn their workflows, and bring useful patterns back into the platform. Because Optiver is regulated and protects its intellectual property, its infrastructure is on-premises, with limited external calls for third-party inference.
The inference gateway gives Optiver model choice and operational control
Optiver runs models from third-party providers, self-hosted models, and open-source models through an inference gateway. The gateway handles routing, records model spend, and supports budgets. It also gives the firm a fallback when a provider is unavailable, which matters because downtime can reduce trading activity. Above the model layer, Optiver has an agent platform with skills, plugins, tools, memory, and knowledge systems. An agent gateway provides a shared ledger for agents, communication, orchestration, and identity management. The company also provides web interfaces, an agent workbench, and a personal assistant so people can use the platform through familiar surfaces.
New frontier models can reach production on the day they appear
When a new frontier model becomes public, Optiver aims to use it in production flows the same day if its internal tests show value. The company partners with trading teams to collect golden sets and runs new models against those workloads immediately. The platform team then adds the model to the gateway, prepares its APIs and cost controls, updates harnesses, and arranges global support. Results and recommendations are shared with employees so they know how well the model works and where to use it. The gateway also protects the firm from regional availability problems, cloud-provider differences, and workloads that must stay in isolated on-premises zones.
A meta harness keeps teams portable across models and frameworks
Optiver wants to avoid coupling its workflows to one model provider or agent framework. Third-party models, on-premises models, and different harnesses pass through the inference gateway, where interactions are logged and sent to evaluation. A meta harness sits above the supported harnesses. Teams can define their own configurations there while keeping some portability between implementations. The meta harness connects to team knowledge bases, and the company uses traces to derive additional knowledge over time. Teams can deploy agents on the platform or access them through the web portal, IDE, and other interfaces.
Spend control starts with reducing unnecessary context
Optiver's AI spend rose sharply during major frontier model releases, then came back under control through operational changes. Tokgozoglu says fresh sessions sometimes carried at least 200,000 tokens of default context because teams were putting too much information into prompts. The company introduced MCP tool search, allowed agents to use a CLI that wrapped multiple MCPs, and examined cache invalidation and cache duration. Engineers also moved information into knowledge bases instead of keeping it in very large system prompts. These changes reduce token use while preserving access to information, and the savings increase when tokens become more expensive.
Visibility and model routing help teams spend with more discipline
Optiver gives teams visibility into their spending, alerts, and self-managed limits. Tokgozoglu says teams often set reasonable limits when they can see the cost and control it themselves. The company also tests open-source models where they offer good value and explores routing prompts to cheaper models. Another pattern is to use a frontier model for planning and smaller models for implementation. Better knowledge bases can reduce retrieval volume by returning only the needed context. The goal is to get more useful work from the same token budget, rather than simply making more models available.
Optiver builds interfaces around the tools traders already make
Teams had built different harnesses, developer-box setups, and small web interfaces to fit their workflows. Optiver responded by building a shared web portal with session management, chat windows, and artifacts. Traders now contribute code through pull requests when they find useful additions, including collaborative document editing. The company is also working toward rooms where a user can bring together their own sessions, several agents, and other deployed agents, with access through Slack as well. Tokgozoglu's point is practical: users with high agency will build missing tools themselves, so the platform team should provide a better supported version.
"If a new frontier model is made available publicly, we want to be able to leverage it in production flows the same day because there's immense value in that if it is valuable."07:10
Who should watch
You run AI infrastructure for a regulated company and need to support both hosted and self-hosted models without locking teams to one provider.
Your model releases are fast, but internal evaluation, cost controls, harness updates, or regional support slow adoption.
Your teams are building agent tools independently and you need to decide which capabilities belong in a shared platform.
