Azure AI model catalog lets teams compare and deploy a wide range of foundation, flagship, small, open, multimodal, embedding, and language-specific models.
2
The Azure AI model inference API uses a shared REST and SDK structure, so teams can switch model endpoints without changing their application code.
3
Azure AI Studio combines prototyping, evaluation, deployment, data grounding, monitoring, privacy controls, security features, and content safety checks.
Summary
Sharmila Chokalingam presents Azure AI model catalog as a way to move from testing models to production without rebuilding an application each time. Teams can browse more than 1,600 models, filter them by task or collection, inspect model cards, review benchmarks, and deploy through managed compute or serverless APIs. The demos cover a Cohere deployment, the chat playground with customer data, function calling with Mistral Large, and a Prompt Flow shopping assistant that uses retrieval and query rephrasing. A shared API and SDK structure makes it possible to compare models within the same application flow. Chokalingam also describes privacy and security controls, including restrictions on provider access to prompts, model scanning, private networking, and Azure Policy. Customer examples from EY, CEVA Logistics, and Bridgestone show different models being used for workplace assistance, customer care, and demand forecasting.
Sharmila says teams usually answer three questions: whether AI can solve the use case, which model fits it, and how to scale it for production. The first stage is prototyping, where developers try several models, establish feasibility, compare benchmarks, and create a shortlist. They then optimize for cost, latency, regional requirements, and other constraints. Azure AI Studio supports prompt engineering, retrieval-augmented generation, and fine-tuning during this stage. After that, teams can operationalize the application with monitoring, scalability, capacity management, data privacy, and content filtering.
The catalog covers different model types and requirements
Azure AI offers flagship LLMs and SLMs, including Azure OpenAI models, GPT-4, Mistral, Llama, Cohere, Phi-3, and other open models. The catalog also includes models for image generation, embeddings, and other multimodal needs. Sharmila gives language-specific examples: Mistral is useful for European languages, Cohere has a multilingual embedding model, and Jais supports Arabic. Azure also partners with organizations such as Meta, Databricks, Snowflake, and NVIDIA to bring more models to the platform.
AI Studio makes model discovery and deployment concrete
In the demo, the model catalog at ai.azure.com contains more than 1,600 models. Users can filter by deployment type, inference task, or model family, then open a model card with customization details, tool-use capabilities, retrieval information, pricing, and inference samples. The samples include integrations with LangChain and LiteLLM. Deployment connects to Azure Marketplace for billing based on token usage. The user chooses a deployment name, enables content filtering if needed, and receives a URL and key ready for application code.
Grounding a model with an index changes its answer
The chat playground first answers a question about Microsoft's culture of giving with general information. Sharmila then adds a Microsoft Give index and asks the same question. The grounded response includes a specific detail, that employees get 50 USD to start a new hire giving credit. The example shows how a model can use an available index to answer from internal material rather than relying only on broadly available information. Sharmila also states that customer prompts and completions are not shared with model providers or used to train the models.
Managed compute and serverless APIs have different operating costs
Azure AI offers managed compute and serverless API deployment. With managed compute, the customer supplies GPUs, pays for virtual machines by the hour, and handles quota and capacity management. This option supports hundreds of open-source models. Serverless APIs are available for Azure OpenAI Service and models-as-a-service offerings. They provide ready-to-use APIs for more than 30 flagship or premium models, with charges based on input and output tokens. A common REST schema and SDK structure lets developers change endpoints while keeping the surrounding code largely the same.
Function calling connects a model to application logic
The function-calling demo uses Mistral Large as a shopping assistant. Instead of asking the model to guess a shop's prices, the application defines a get bill amount function with the shop's relevant information. The model recognizes that the request needs an external function, calls it with the right arguments, and returns the exact result. For 10 staplers, the response gives a total of 45 dollars including ongoing discounts. A system message also asks the assistant to give environmental guidance, which appears in the answer.
Prompt Flow supports retrieval, query rephrasing, and model comparison
The Prompt Flow demo builds a shopping assistant with retrieval and generation steps. An additional query transformation step expands a short question such as whether new hiking shoes are available into a more detailed request about materials and features. The lookup step retrieves information from an index, and the language model uses that context to recommend a product. Developers can create variants that use different generation or embedding models, then evaluate the same flow across models such as Cohere Command R, Llama 3, and Mistral Large.
Privacy and governance controls cover the deployment path
Sharmila groups the platform's protections into data privacy, security and compliance, and responsible AI and content safety. Prompts and completions are not shared with model providers or used to train models. Azure's model scanning checks for embedded malware, back doors, common vulnerabilities and exposures, tampering, and corruption. Private networking lets customers route ingress and egress traffic through virtual networks and control outbound access with fully qualified domain name rules. Azure Policy can allow or deny model collections and separate development, test, and production access.
Customer examples use different models for different jobs
EY built EYQ Chat for professional tasks and adopted it across 275,000 employees. The company uses Azure OpenAI models and is exploring Llama models for documentation and summarization, while continuing to evaluate performance and responsible AI practices. CEVA Logistics uses a Mistral model for a chatbot supporting customer care agents, with reduced response latency and increased customer satisfaction reported for that use case. Bridgestone uses Nixtla's TimeGPT time-series model to forecast monthly demand and reported nearly 30 percent lower average forecasting error.
"The three questions that we all try to answer when we try to build generative AI apps is can AI solve my use case, and then what is the best model for my use case, and then how do I go about scaling this for the production workloads?"01:33
Who should watch
You are comparing several LLMs and want a practical way to prototype, benchmark, and change models without rewriting your application.
Your team needs managed access to open and premium models with controls for private networking, model governance, content filtering, and customer data privacy.
You are building a grounded assistant or production AI workflow and want examples using function calling, retrieval, Prompt Flow, and model evaluation.
