# We accidentally made an AI platform

Jamie Turner, Convex | AI Engineer World's Fair 2024 | 05:36

Source: https://www.youtube.com/watch?v=JbyictbPFV0
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/we-accidentally-made-an-ai-platform
Published: 2024-10-08
Tags: data-pipelines, embeddings, workflows

## TL;DR
- Convex was built to replace much of the backend glue work that teams do around data, systems, and application state.
- Convex extends React-style reactive data flow into the backend, so queries and mutations can update applications as soon as data changes.
- Generative AI applications pushed Convex toward vector indexes, startup support, and reusable components for complex backend workflows.

## Summary
Jamie Turner explains how Convex started as an attempt to replace routine backend engineering. Teams spend much of their time connecting systems, configuring services, and converting data between formats, even though this work does not affect the product users see. Convex took inspiration from React's state model and applied it to the backend. Its queries and mutations track data dependencies so applications can receive updates without custom polling, cache invalidation, or push systems. Turner says this design became especially useful for generative AI applications, where server-side steps such as transcription, summarization, embedding generation, and related-note search produce results at different times. Convex responded by adding vector indexes, launching a program for startups, and preparing reusable components that package sophisticated workflows. The talk is a short account of a backend product being pulled toward the needs of AI developers.

## Key ideas
### Convex aims to remove backend glue work from application development
[00:00](https://www.youtube.com/watch?v=JbyictbPFV0&t=0s)
Turner says Convex's true mission is to replace traditional backend engineering. Teams spend time connecting services, configuring systems, mapping data formats, and handling other integration work that has little to do with their product. He says this can take up half of a team's time, while users neither see nor benefit from it. Convex wants to replace that work with a higher-level functional interface that feels native to the application. Turner compares the goal to Firebase or Parse, while describing Convex's approach as a design for the way current applications manage data and state.

### React's state model provides the design pattern for Convex
[01:13](https://www.youtube.com/watch?v=JbyictbPFV0&t=73s)
For the design exercise, Convex drew heavily from React and from functional reactive data flow more generally. In React, a state variable has a setter, and places that depend on the state are updated, rendered, or refreshed when it changes. Turner says the same pattern breaks down when a server is involved. Developers still need to poll the server, invalidate caches, or build their own push mechanism. Convex extends the reactive model into the backend by tracking data flow and dependencies across its queries and mutations.

### Backend queries can become live data sources for applications
[01:52](https://www.youtube.com/watch?v=JbyictbPFV0&t=112s)
Convex has queries and mutations like other frameworks, but Turner says it tracks backend dependencies pervasively. Queries are universally subscribable entities that applications can receive updates from as soon as those updates are available. This lets the application stay connected to changing backend state without separately assembling polling, cache invalidation, and event-based update systems. The model matters for applications where server work produces data progressively and the interface needs to show each result as it arrives.

### Generative AI workflows create many reactive server-side steps
[02:33](https://www.youtube.com/watch?v=JbyictbPFV0&t=153s)
Turner connects the backend model to AI applications by treating some reacting entities as server-side actions. He gives the example of a note-taking application that performs automatic speech recognition, summarizes the result, generates embeddings, and finds related notes. The application may need to return to the user between those checkpoints to show a summary or related notes. He also describes an application that creates a project plan from a prompt, producing project names, feature requests, color palettes, and icon ideas through concurrent background chains. Their results flow into the application as they finish.

### Generative AI usage pushed Convex toward platform features
[03:48](https://www.youtube.com/watch?v=JbyictbPFV0&t=228s)
Turner says more than 90 percent of projects on Convex became generative AI projects after the ChatGPT boom. Developers repeatedly needed vector indexing as one step in their workflows, so Convex added vector indexes to its schema system. He says developers can define them alongside ordinary field indexes with TypeScript support. Convex also launched a startup program with discounts and startup-only forums and events, and Turner says the first group included many generative AI companies.

### Reusable components package the state machines behind AI workflows
[04:44](https://www.youtube.com/watch?v=JbyictbPFV0&t=284s)
The next feature Turner describes is the Convex Components framework. It encapsulates whole state machines in reusable building blocks that developers can add to an application. These components package sophisticated workflows that Convex developed with customers, allowing teams to put the backend pieces into an application more easily and quickly. The aim is to make complex, multi-step AI workflows easier to ship without requiring every team to build the same backend coordination from scratch.

## Notable quotes
- "What I really want to title it is we accidentally made an AI platform and what are we going to do about it." (00:00)
- "A lot of times teams are spending a lot of their time, like half their time, on this stuff that has nothing to do with your product." (00:34)
- "Convex fixes that." (01:52)
- "It ends up that Convex is kind of a combination of syncing state between these backend steps and the application." (03:28)

## Tools & references mentioned
- Convex
- React
- Firebase
- Parse
- ChatGPT
- Dropbox
- Google
- Convex Components

## Who should watch
- You are building a generative AI application with several server-side steps and need the interface to receive their results as they arrive.
- Your team spends time wiring together backend services, handling cache invalidation, or maintaining custom update mechanisms.
- You want to understand why a backend built around reactive data flow can fit AI applications with concurrent workflows.

## Related talks

- [Building Reactive AI Apps](https://aietalks.com/talks/building-reactive-ai-apps) (Matt Welsh, Fixie.ai, 17:02)
- [Agentic Workflows on Vertex AI](https://aietalks.com/talks/agentic-workflows-on-vertex-ai) (Rukma Sen, Google Cloud, 18:06)
- [Git push get an AI API](https://aietalks.com/talks/git-push-get-an-ai-api) (Ryan Fox-Tyler, Hypermode, 45:01)
- [AI Platform Engineering](https://aietalks.com/talks/ai-platform-engineering) (Patrick Debois, 28:18)
- [The Build-Operate Divide: Bridging Product Vision and AI Operational Reality](https://aietalks.com/talks/the-build-operate-divide-bridging-product-vision-and-ai-operational-reality) (Chris Hernandez, Chime & Jeremy Silva, Freeplay, 12:50)