Multimodal models can read images, translate handwritten text, and reason about visual content in one step.
2
Azure AI Studio offers a broad model catalog with serverless and infrastructure deployment options, plus model benchmarks and evaluation tools.
3
Small models can run locally in a browser, while agents can execute generated Python code to analyze files and create visualizations.
Summary
Cedric Vidal demonstrates Azure AI features through a long sequence of live demos. GPT-4o reads printed and handwritten menus, translates French text, and reasons about dietary options. The same approach can inspect damaged utility infrastructure or houses and return either natural-language answers or JSON for monitoring systems. A video translation service preserves the speaker's voice, language, and tone. Vidal then explores Azure AI Studio's model catalog, which includes models from many vendors and supports serverless or customer-managed deployment. He compares model performance and shows Microsoft's Phi-3 Vision model, including a small quantized version running locally in a browser. Other demos cover retrieval-augmented generation, evaluation metrics, and an agent that generates Python code to analyze a kite-surfing GPX file and draw a map. He finishes with a preview of GitHub workspaces generating a Java front end from an existing repository.
Multimodal models can reason over images and text together
Vidal uses GPT-4o to inspect a restaurant menu and answer whether it contains vegan options. The model extracts the printed text from the image, examines each menu item, and reasons about which dishes have vegan or vegetarian versions. He contrasts this with an older image-to-text pipeline that used OCR first and reasoning afterward. In the newer approach, the model understands pixels and text natively. He repeats the demonstration with handwritten French text on a chalkboard, translating the menu and answering questions about it in the same interaction.
Visual analysis can feed operational monitoring and insurance workflows
Vidal applies image reasoning to a photograph of utility poles and power lines lying on the ground. The model describes the likely infrastructure damage and answers whether electricity is probably working in the area. He says an energy company could apply this approach to video cameras monitoring infrastructure across a country. The response could be returned as JSON rather than natural language, allowing software to feed dashboards and real-time monitoring systems. A second example shows a damaged, tilted house and identifies possible causes such as a hurricane, earthquake, or landslide, which Vidal connects to insurance use cases.
Video translation can preserve a speaker's voice and delivery
The video translation demonstration turns Vidal's speech into German while keeping his voice. He points out that the service also follows his delivery: when he whispers, the translated video whispers, and when he raises his voice, the translated version does too. Vidal says he stitched the different video segments together himself during postprocessing, but the translation service handled the rest. The example shows speech translation as more than replacing words, since timing and vocal tone are part of the output.
Azure AI Studio offers many models and two deployment paths
Vidal opens the Azure AI model catalog and says it contains 1,600 models at the time of the demo. Developers can filter deployment options between serverless hosting and their own infrastructure. With serverless deployment, billing is based on tokens and the vendor manages the infrastructure. Bringing your own infrastructure means paying for GPUs whether the endpoint is being used or not. Vidal also points to model benchmarks, which compare many catalog models using accuracy and other metrics to help developers choose a model for a particular application.
Small multimodal models can run locally in a browser
Vidal demonstrates Microsoft's Phi-3 Vision family alongside GPT-4o. He uses Phi-3 Vision to inspect the damaged utility image and produce a simpler explanation that still correctly identifies a likely power outage or disruption. He then runs a 3.8-billion-parameter Phi-3 model, quantized to four bits and roughly two gigabytes, directly in a browser. WebGPU gives the browser access to the device GPU. On his MacBook Pro with an M2 chip, the model answers a question about Rivian vehicles quickly, and Vidal notes that it could continue working without Wi-Fi, which is useful when sensitive data should stay local.
Retrieval can ground an older model in private documents
Vidal asks a model about Rivian vehicles and first gets information limited by the model's previous knowledge update. He then selects an index containing Wikipedia pages about Rivian models and asks the question again. The grounded response includes newer information such as the R3 and R2. This demonstration shows how retrieval-augmented generation can connect an existing language model to a document collection. The documents provide current or application-specific information without requiring the model itself to have learned it during training.
Vidal describes the evaluation feature in Azure AI Studio for testing language-model applications. Developers can run metrics including coherence, groundedness, and relevance. Groundedness matters particularly for retrieval-augmented applications because answers should be supported by the supplied documents. He changes system prompts or models as examples of modifications that require checking. In his simple demonstration, the answer to a question about Rivian models receives a coherence score of four and is judged well grounded on a one-to-five scale.
Code interpreter agents can analyze unfamiliar files and draw results
Vidal uploads a GPX file from a kite-surfing session to an agent with code interpreter. When asked how long he was on the water, the language model generates Python code and runs it in a sandbox to analyze the XML file. It then estimates the number of turns by examining GPS coordinates and changes in angle. Vidal asks the agent to draw the session on a map, then adds red crosses for the turns. He emphasizes that he did not need to understand the GPX format or write the analysis code himself.
A preview tool can generate a front end from a code repository
In a GitHub workspaces preview, Vidal asks the system to add a Java GUI front end to a repository containing Python code. The language model examines the repository, works out what it contains, writes specifications, and produces a plan. The developer can inspect each step and request corrections when the system makes a mistake. Vidal presents this as an upcoming capability rather than a generally available feature. The demo focuses on repository understanding and iterative planning before code generation.
"Now the model understands natively both pixels and text and in its internal representation has the same vectors for the same concepts visual concept and textual concept."06:46
Who should watch
You are building an Azure application that needs to combine text, images, speech, or video and want to see concrete workflows rather than product descriptions.
Your application needs current or private documents, model comparisons, evaluation metrics, or local inference for sensitive data.
You want to understand what code-interpreter agents can do with files and how repository-aware code generation might fit into development work.
