# Why We Rebuilt Microsoft Office From Scratch for AI Agents

Yijia Liang, Compound AI | AI Engineer NYC 2026 | 15:45

Source: https://www.youtube.com/watch?v=v0UkV5qS9E4
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/why-we-rebuilt-microsoft-office-from-scratch-for-ai-agents
Published: 2026-11-01
Tags: agents, enterprise, human-in-the-loop, multi-agent, workflows

## TL;DR
- Personal agents work because they run reliably in the background, act proactively, and connect to personal context, but finance workflows need shared agents with controlled access.
- Compound rebuilt Excel, PowerPoint, and Word as a headless Office engine after evaluating desktop plugins, virtual machines, and existing software libraries.
- Agent swarms can process large data rooms, while a write-render-check-fix loop lets agents produce and validate financial models and presentations before people review them.

## Summary
Yijia Liang explains why personal AI agents do not directly fit institutional finance. A deal team needs several people to share one agent, keep work inside existing email threads, and control access to sensitive information. It also needs agents to run long tasks in parallel across Excel, PowerPoint, and Word. Compound evaluated desktop plugins, virtual machines, and existing Office engines, then rebuilt the applications from scratch with agents. The resulting headless engine can run parallel work, render files, and check formatting before fixing errors. Liang also describes using agent swarms to search data rooms with more than 15,000 files and cite findings back to source documents. The aim is to move analysts away from file hunting and formatting checks so they can spend more time applying judgment to deals. Liang is honest that many finance workflows remain in the copilot category, while larger tasks such as drafting investment committee materials still need better controls and deeper integration.

## Key ideas
### Personal agents succeed because they run reliably, act proactively, and know their user's context
[00:49](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=49s)
Liang attributes the recent rise of personal agents to three properties. They can handle background tasks such as booking a restaurant or flight, models and computer-use systems have become more reliable, and agents can act without a fresh request. They also connect to email, drives, and calendars. He gives the example of an agent remembering a user's wife's anniversary and buying flowers. Finance needs a different version of this experience because its context may include a data room, SharePoint, Box, or decades of firm records.

### Finance needs shared agents that work with the whole deal team
[02:28](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=148s)
A single user working with Claude or ChatGPT often has to download a file, upload it to email or Teams, and ask colleagues for feedback. Liang argues that everyone who matters on a deal should be able to join the same thread and talk to the same agent. The agent can then hold the context for a workstream instead of forcing each person to maintain a separate session. This matters for long workflows that turn a data room into models, slides, and investment committee memos.

### Each agent needs an inbox with access controls and human-like email behavior
[04:10](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=250s)
Compound gives every agent a built-in inbox because financial workflows often run through Outlook email chains. Liang says this involves authenticating the people on a thread and checking that they have read and write access to the agent. Privacy matters because deal work is covered by NDAs. The product also has to handle several messages arriving quickly. Rather than sending repeated acknowledgements, the agent can introduce some latency so the exchange feels more like a teammate responding.

### Existing Office integrations did not support large-scale parallel work
[05:30](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=330s)
Compound evaluated Excel, PowerPoint, and Word desktop plugins, virtual machines running computer-use agents, open-source packages such as OpenPyXL, and commercial engines. Liang says these options did not meet the requirements for running many tasks in parallel. The team therefore rebuilt Office from scratch with coding agents. It used 1.5 trillion input-output tokens and more than 8 million lines of code, with the goal of recreating Office in full fidelity.

### Compound used Office as an oracle to recreate Excel, PowerPoint, and Word
[06:04](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=364s)
The team began with the open-source IronCalc project for Excel. It compared outputs against real files from online corpora and internally created files, then ran repeated loops against Office as the judge. Liang says the Excel implementation reached 800,000 lines of Rust and became a one-for-one replica that was more performant than Excel. The same approach was applied to PowerPoint and Word. Rust rendering was used to check pixel-level output so downloaded files matched what users saw in Compound.

### A headless engine makes parallel work and recalculation faster
[07:38](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=458s)
Desktop Office applications are tied to a machine's CPU, which limits how many tasks agents can run at once. A headless engine can run work in parallel and build complex models faster. Liang says Compound can recalculate 20 MB Excel files four times faster. He connects this speed to the quality and cost expected from an agent doing an analyst's work. In his view, scaling agents with desktop Excel and PowerPoint would follow the wrong architecture.

### Owning the editor lets agents collaborate through comments and checks
[08:21](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=501s)
Liang says he built the presentation for the talk in Compound by asking for a first draft and then adding comments about formatting. He applied lessons from his consulting and Coatue work directly through those comments. Multiple team members can use the same collaboration setup and comment on one deck. He says a team of five could work through a 100-page deck in an afternoon instead of revising it across several weeks. The editor also enables a write, render, check, and fix loop for complex models.

### Agent swarms can search large data rooms while preserving citations
[09:55](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=595s)
Buy-side firms may have data rooms with more than 15,000 files, including large workbooks. For breadth, Liang recommends using coding agents to read files with tools such as grep, having a lead agent divide the directory, and sending groups of agents to return findings. The process can repeat until the output is satisfactory. Findings can be grounded in citations to source documents. For depth, Compound's Excel engine processes a large workbook into a format that is easier for the model to read, which reduces the cost and delay of exposing every detail as code.

### Finance can move from copilots to autopilots when checks are strong enough
[13:10](https://www.youtube.com/watch?v=v0UkV5qS9E4&t=790s)
Liang says some tasks, such as spreading comparable companies, already work with existing tools. Longer tasks remain in the copilot category, including drafting a 50-page investment committee presentation or building a model exactly to a professional's requirements. Compound's systems are intended to expand the amount of work that can be delegated. Liang compares this to coding agents, where users became more willing to delegate large tasks after adding sufficient tests and checks. Finance needs comparable controls before teams will trust a data room and its outputs at much larger scale.

## Notable quotes
- "But all the factors that make personal agents work so well doesn't really translate to institutional finance today." (01:27)
- "Because ultimately at the end of the day for institutional finance, most workflows run through Outlook." (04:20)
- "We decided to just build our own office from scratch." (06:04)
- "So, with an own engine, you actually own the write, render, check, and fix loop." (09:34)
- "They started realizing, maybe I can just truly vibe it. If I have sufficient checks, if I have sufficient tests, maybe I can just trust all the code it writes." (14:39)

## Tools & references mentioned
- Compound AI
- Muser Instinct
- Coatue
- Claude
- ChatGPT
- Microsoft Excel
- Microsoft PowerPoint
- Microsoft Word
- Outlook
- Teams
- SharePoint
- Box
- OpenPyXL
- IronCalc
- Rust
- Opus 4.5

## Who should watch
- You are building agents for finance or other workflows where several people need to collaborate with one agent inside existing communication systems.
- Your agents must create or edit Office files at scale and you need to understand why desktop plugins and virtual machines may limit parallel execution.
- You are designing document retrieval for large data rooms and want an approach that combines agent swarms, source citations, and structured processing of large workbooks.

## Related talks

- [Redesigning How Software Gets Built With AI Agents](https://aietalks.com/talks/redesigning-how-software-gets-built-with-ai-agents) (Tariq Shaukat & Ali-Reza Adl-Tabatabai, Sonar, 19:12)
- [Building AI Agents that Actually Automate Knowledge Work](https://aietalks.com/talks/building-ai-agents-that-actually-automate-knowledge-work) (Jerry Liu, LlamaIndex, 17:57)
- [How agents will unlock the $500B promise of AI](https://aietalks.com/talks/how-agents-will-unlock-the-500b-promise-of-ai) (Donald Hruska, Retool, 16:22)
- [The Age of the Agent](https://aietalks.com/talks/the-age-of-the-agent) (Flo Crivello, Lindy, 14:49)
- [How to Build Enterprise-Aware Agents](https://aietalks.com/talks/how-to-build-enterprise-aware-agents) (Chau Tran, Glean, 19:53)
