Gemini 2.5 Flash supports text, images, audio, video, and documents, and its thinking budget lets developers trade reasoning capacity against cost.
2
Structured outputs turn multimodal inputs such as invoices into typed data, while function calling lets Gemini choose tools whose results are then returned to the model.
3
The Gemini SDK can connect directly to MCP servers, handling tool discovery, function calls, and responses so developers do not have to implement the loop themselves.
Summary
Philipp Schmid runs a hands-on workshop on Gemini 2.5 through Google AI Studio, Colab, and the Gemini SDK. He starts with text generation, token counting, streaming, chat history, system instructions, and generation settings. He then moves to multimodal input, showing how Gemini can process PDFs, images, and videos, including extracting information from an invoice. The workshop explains thinking tokens and the thinking budget, which can limit how much reasoning a request uses. Schmid then demonstrates structured output with Pydantic and function calling. Structured output turns an invoice into fields that an application can store or process. Function calling gives the model a declared tool and lets the application execute it. Native tools add Google Search, code execution, and URL context. The final section connects the Gemini SDK to local and remote MCP servers, where the SDK manages the tool-calling loop. Schmid is direct about current limits, including tool-count pressure, authentication, citation details, and the need to run evaluations on real data.
The workshop uses Gemini 2.5 Flash as a free API starting point
Schmid focuses on Gemini 2.5 Pro and Gemini 2.5 Flash, using Flash because it is available through the free API tier. Both models are multimodal by default. They can understand text, images, audio, video, and documents, while generating text. He also mentions Gemini models for image generation and new text-to-speech models. The practical setup uses Google AI Studio for an API key and Google Colab for the notebooks, so attendees do not need to install much locally. The workshop repository contains sections on text generation, multimodality, structured output, function calling, native tools, and MCP servers.
The Gemini SDK exposes a small set of repeated model-call patterns
The first notebook introduces the Google AI SDK through a client, its models abstraction, and methods such as generate_content and generate_content_stream. The model ID is a parameter, so developers can switch from Gemini 2.5 Flash to another available model. Contents carries prompts, conversations, and messages. Schmid uses a coffee-shop naming prompt as a first example, then asks attendees to change prompts and model IDs. The notebook includes partially completed cells and exercises, while a separate solutions folder contains finished versions. The structure is intended to make people practice the SDK rather than learn a different API for every example.
Token metadata and thinking budgets make Gemini requests easier to price
The count_tokens API estimates the input tokens for a prompt before generation. A generated response also includes usage metadata with input tokens, thought tokens, and candidate tokens. Gemini 2.5 is a thinking model, so it can spend tokens on internal reasoning before producing the candidate response. Schmid explains that thought and output tokens use output pricing, which is higher than input pricing. Developers can set a thinking budget to zero to disable thinking, or choose a maximum number of thinking tokens. He presents this as a way to test different budgets against an application's accuracy and cost requirements instead of assuming that every request needs the same amount of reasoning.
Streaming and client-side chat state improve interactive applications
Waiting for a complete response can create a poor user experience, so the SDK provides generate_content_stream, which returns an iterator that applications can send to users as chunks arrive. The SDK also has a chats API that keeps the conversation state on the client. A chat session can receive a trip-planning message, return an answer, and then use the stored history when the user asks for food recommendations in the same context. Schmid says this is only a client abstraction. The backend still receives the equivalent conversation as a request containing the turns. The session history can be retrieved with a history method for storage or later updates.
Gemini can process uploaded files and multimodal documents directly
Schmid uploads The Adventures of Tom Sawyer through the Files API and passes a reference to the file in a later request. The file is stored in a personal bucket associated with the user's AI Studio account for a limited time. This avoids sending the complete file with every request. He also demonstrates a supermarket invoice in PDF form. Gemini performs OCR and provides an image representation of the PDF, so developers do not have to convert the document and run OCR themselves. The invoice example extracts the total correctly, including German text. For files that exceed the model context, Schmid recommends counting tokens and then chunking, summarizing, or extracting important information before sending the smaller context.
Structured output turns document understanding into application data
Schmid uses Pydantic schemas to require a particular response structure. His recipe example contains a recipe name, a list of ingredients, and a list of recipes. Gemini returns JSON that the SDK can parse back into the Pydantic schema. The same approach works with multimodal inputs. When he gives the invoice to the recipe schema, Gemini returns an empty recipe list rather than inventing recipes. After replacing it with an invoice schema, the model extracts the date, purchased items, and prices. This gives an application fields it can insert into a database or pass to another API. The model's document and image understanding remains part of the same request.
Function calling gives the application control over external actions
Function calling uses a declaration with a name, description, and input properties. Schmid defines a get_weather function and asks for the weather in Tokyo. Gemini returns the function name and the location argument instead of writing the final answer. The application must inspect that structured output, call the function, and send the function result back into the conversation. Gemini then produces a user-facing response. The function could query a database, call an external API, read email, or send email. Parallel tool calling lets independent functions, such as starting music, powering a disco ball, and dimming lights, run together. Sequential calls are needed when one function's result supplies the input for the next.
Native tools add search, code execution, and URL context
Gemini's native tools do not require developers to write each function declaration or execute every operation themselves. Google Search turns a prompt into one or more search queries, runs them, and gives the results back to Gemini for a final answer. The response includes grounding metadata that points to source websites and portions of the answer. Code execution lets Gemini write and run Python, including producing a chart. URL context extracts information from supplied web pages, with up to 20 URLs in one request. Schmid says these tools can be combined, such as searching for a current Python version and then writing and running code. Search has a free allowance, while retrieved web content contributes tokens to the context.
MCP integration moves the tool-calling loop into the Gemini SDK
The Gemini SDK can connect directly to MCP servers. The developer starts an MCP server, creates a session, and passes that session in the tools argument of a Gemini request. Behind the scenes, the SDK obtains the server's tool schemas, gives them to the model, detects a tool call, invokes the MCP server, and supplies the result back to Gemini. Schmid demonstrates a local weather MCP server using the Open-Meteo API, then describes connecting to a remote GitHub MCP server over streamable HTTP. He sees MCP as a way to avoid rebuilding the same wrappers and declarations across teams. He also warns that making many tools available at once creates a selection problem, especially when a server exposes dozens of tools.
"The LLM sees at the end the same exact exact exact same thing."1:31:36
Who should watch
You are building a Python application around Gemini and need working examples for streaming, chat state, token accounting, or model configuration.
Your application needs to extract typed fields from PDFs and images, or needs Gemini to call APIs, databases, or other application functions.
You are evaluating MCP and want to see how a Gemini client can use local and remote MCP servers without manually writing the full tool loop.
