Open models can match larger proprietary models in narrow domains while reducing latency and inference cost.
2
Serving an open model in production requires choices across kernels, runtimes, hardware, deployment, scaling, and observability.
3
Applications increasingly combine specialized models with retrieval, function calling, and external tools, so the complete system matters more than one model.
Summary
Dmytro (Dima) Dzhulgakov argues that open models are becoming a practical choice for production applications because teams can adapt them to narrow domains, run them faster, and avoid paying for capabilities they do not need. Smaller models can work well for tasks such as customer support, coding, medicine, or function calling, while lowering latency for voice assistants and agentic systems. The hard part is operating them. Teams must choose GPUs, frameworks, runtime settings, deployment models, and scaling infrastructure, then repeat that work as models change. Fireworks AI addresses this with a custom serving stack, fine-tuning, model hosting, and deployment options from serverless inference to dedicated enterprise hardware. Dzhulgakov also describes applications as compound systems that combine models with retrieval, function calls, and external tools. FireFunction is presented as a model built for this pattern, with examples that query stock data, create charts, maintain context, and switch topics.
Smaller open models can fit a narrow task better than a general proprietary model
Dzhulgakov says large proprietary models are good across many domains, but applications often pay for abilities they do not need. A customer support chatbot does not need to know 150 Pokémon or write poetry. A smaller model adapted to the specific domain can produce similar or better quality while responding faster. He cites function-calling benchmarks where a fine-tuned Llama 3 reached similar performance at 10x the speed. Lower latency matters for voice assistants and agent applications because those systems call models repeatedly during one interaction.
Open model weights make domain adaptation practical
Open models can be specialized for particular uses because their weights are available for customization. Dzhulgakov points to models adapted for Indian languages, coding, and medicine, along with thousands of other variants on Hugging Face. Teams can fine-tune a model for the quality they need instead of accepting the same general-purpose behavior for every workload. This flexibility is one reason he expects open models to gain a larger share of production use.
The difficult work is turning an open model into a maintained service
Running an open model requires more than downloading weights. Teams need to find GPUs, select frameworks, optimize performance, tune the deployment, and repeat the process when a new model version arrives. Runtime choices also depend on the workload. Prompt length, latency targets, and throughput requirements change which settings are best. Dzhulgakov says production deployment adds further work around unreliable GPUs, scalability, telemetry, and observability.
Fireworks tunes the whole serving stack for a workload
Fireworks built its serving stack from the ground up, including custom CUDA kernels and service-level deployment and orchestration. The company tunes the stack for a customer's model, cost target, and latency requirement. For long prompts, common in retrieval-augmented generation, runtime and deployment settings can be changed to improve throughput without losing the required latency. Dzhulgakov frames the optimization problem as minimizing cost under a fixed latency constraint, such as producing a required number of tokens within two seconds.
Fireworks supports serving many fine-tuned variants on the same GPU. Dzhulgakov says that, with the right techniques, thousands of low-rank adaptation models can be deployed together. This allows token-based serverless inference even when a customer has many model variants, without paying a fixed cost for each one. The approach is intended for teams that want custom models while keeping the operational model of a hosted service.
Production applications combine models with retrieval and tools
Dzhulgakov says a model alone is increasingly not the product. Models can hallucinate, so applications need grounding through retrieval or external knowledge bases. Applications may also chain different model types and call external actions. He uses the term compound AI system for this combination of retrieval, function calling, and tools. In his description, a central function-calling model coordinates domain-specific models and external services in response to a user's request.
FireFunction handles multi-step tool use and conversational context
Fireworks' FireFunction models are designed for function calling while retaining general chat abilities. In the demo, the model can query stock quotes, plot charts, and generate images. To create a chart of the largest cloud providers, it identifies the companies, calls an API for stock prices, and passes the results to a plotting function. It also understands follow-up requests such as adding Oracle to the existing graph. When the user changes subjects, it can discard the older context. The demo is open source and uses FireFunction alongside other models, including SDXL.
Fireworks offers a path from experimentation to dedicated deployment
Dzhulgakov describes several hosting modes. Serverless inference provides pay-per-token access without hardware configuration. Teams can fine-tune models on the platform or bring their own LoRA adapter. On-demand deployments provide dedicated hardware and more control, while enterprise arrangements add longer-term contracts and help with hardware setup and performance tuning. He names Quora, Poe, Sourcetable, Cursor, and Upstage among companies using Fireworks for chatbots, coding assistants, and agents.
"The model is not the product by itself, you need a kind of bigger system in order to solve the target application."10:32
Who should watch
You are deciding whether a smaller open model can meet a narrow application's quality and latency needs.
Your team has a fine-tuned model but needs a practical path to GPU hosting, scaling, and observability.
You are building an agent that combines retrieval, function calls, several model types, and external APIs.
