Heroku Managed Inference provisions curated language, embedding, and image models as application resources, with credentials and model configuration exposed through environment variables.
2
Heroku's agent endpoint can run commands, query a read-only Postgres follower, transform documents, and execute generated Python, Node.js, Ruby, or Go code on one-off dynos.
3
Model Context Protocol lets developers attach Heroku-hosted or external MCP servers to agents, then expose those tools to clients such as Cursor or custom applications.
Summary
Anush Dsouza and Julián Duque present Heroku Managed Inference and Agents through a browser-based workshop. Anush describes Heroku's goal of making AI easier to attach to existing applications, with curated models and an agentic control loop running inside Heroku's trust layer. Julián then provisions a model, loads a Jupyter notebook, and calls an OpenAI-compatible inference endpoint with both regular and streaming responses. The workshop's agent endpoint adds tools for dyno commands, database schema inspection and queries, document conversion, and code execution in Python, Node.js, Ruby, and Go. Examples include checking server time, calculating Fibonacci numbers, extracting an algorithm from Wikipedia, and asking questions about solar-energy data. The final section covers MCP deployment, attaching a Brave Search server, exposing MCP tools through a gateway, and calling them from outside Heroku. The demonstrations show how tool permissions, read-only database followers, and scale-to-zero dynos shape the platform's approach.
Heroku is applying its deployment simplicity to AI applications
Anush Dsouza compares Heroku's AI offering with the earlier problem of deploying Ruby on Rails applications. Developers wanted to build, operate, deploy, and scale those apps, but the work was difficult. Heroku addressed that with tools such as Git push and Heroku commands. The company is taking a similar approach with AI applications and agents. Its stated goal is to make every software engineer an AI engineer and make it simple to attach agents and AI to an existing app. Anush focuses on the operational questions that appear after the first prototype, including model choice, evaluation, tracing, safe tool execution, and scaling.
Managed Inference attaches curated models to an application
Heroku takes an opinionated approach to model selection instead of exposing a large set of configuration choices. Julián demonstrates provisioning Heroku Managed Inference and Agents as an add-on from an application's Resources page. The workshop uses Claude 4 for text generation, while the available options also include Claude 3.5 and 3.7, Cohere Embed for embeddings, and Stable Image Ultra for image generation. Provisioning the add-on supplies an inference URL, an inference key, and a model ID as environment variables. Applications can then use an OpenAI-compatible SDK, make an HTTP request, or build their own client.
The inference endpoint supports ordinary and streamed responses
The workshop first uses the basic chat completions endpoint. A request contains the model ID, an API key, and a user message, and the response has the familiar chat-completions structure with an assistant message. Julián then shows streaming through server-sent events. Instead of waiting for one JSON response, the client receives chunks containing deltas from the completion. His Jupyter example parses each delta and renders the completed answer as Markdown. He presents streaming as useful for showing real-time feedback while an application is waiting for inference.
The agent endpoint adds tools that the basic chat endpoint does not execute automatically. The dyno run command can run a trusted Unix command or an existing script on an application's dyno. Database tools can retrieve a Postgres schema and then run a query based on that schema. Document tools convert an HTML page or PDF URL into Markdown. Code execution generates and runs Python, Node.js, Ruby, or Go on a Heroku one-off dyno, with dependency installation supported through a packages array. A one-off dyno starts for the operation and scales back to zero afterward.
Generated code can be run in several languages with a streamed agent call
Julián asks the code execution tool to calculate the 30th Fibonacci number. The model generates JavaScript, Heroku runs it on a dyno, and the result is returned with the explanation. He changes the same example from Node.js to Go, where Heroku compiles and runs the generated code before returning the result. The agent endpoint streams its responses because tool execution can take time. Julián also combines HTML-to-Markdown with Python execution: the agent retrieves the Wikipedia page for the Euclidean algorithm, extracts the Python implementation, runs it, and reports the greatest common divisor for 252 and 105.
Database access is limited through application permissions and read-only followers
Before using the Postgres tools, Julián attaches the workshop application to a follower database belonging to another application. He explains that the maintained database tools work with followers because followers are read-only. This avoids giving an LLM write access to a production database. The agent first calls the schema tool, then generates a query that matches the discovered tables and columns. In the example, the database contains hourly energy-produced and energy-consumed metrics for a solar-energy company. The agent answers a question about energy saved, breaks the result down by system, and can pass the data to Python code execution to generate a chart.
MCP extends the agent endpoint with Heroku-hosted and existing tools
The workshop attaches a Brave Search MCP server through the Heroku configuration page. The server exposes web-search and local-search tools, which become available to the agents endpoint under an MCP namespace. Julián runs a search for recent AI-agent news and receives the tool result before the model formats the answer. He also shows how to deploy another MCP by adding an MCP process type to a Heroku Procfile and supplying its API key. Heroku runs these standard-input and standard-output MCP servers on dynos that scale back to zero.
MCP tools deployed on Heroku are also available through a toolkit integration endpoint. Julián describes it as a server-sent-events endpoint protected by a bearer token, with OAuth support being developed. He uses the MCP package from Anthropic to create a client, connects to the Heroku endpoint with an authorization header, lists the available tools, and runs Brave web search. The returned list includes the Brave search tools and the deployed Perplexity Ask tool. He says the same approach can be used with clients such as Cursor or with an agent built on another platform.
"We want to make sure that every software engineer right now is an AI engineer and it is simple as attaching agents and AI to your apps."04:16
Who should watch
You have an existing Heroku application and want to add model inference without sending data to an external model API.
You need an agent to run generated code, inspect application data, or execute application-specific commands with explicit access to Heroku resources.
You are building an MCP server or client and want to see how Heroku-hosted tools can be attached to agents and called from another platform.
