The Adversarial Path to the Personal Assistant

Sumit Agarwal, Ario AI18:46 · Feb 2025 · 1,116 views
Thumbnail for The Adversarial Path to the Personal Assistant Watch on YouTube
TL;DR
  1. 1

    Ario starts a personal assistant with adversarial ETL, collecting a user's data from services such as Google, Amazon, DoorDash, Strava, Facebook, and Instagram.

  2. 2

    A profile built from personal data lets Ario answer questions and make recommendations without forcing the user through an intake form or a long series of follow-up questions.

  3. 3

    Sumit Agarwal argues that teams should use ordinary data-processing tools first, reserve the LLM for tasks that need it, and present the model with processed data at the right time.

Summary

Sumit Agarwal presents Ario's approach to building a personal assistant from a person's online history. The system first gathers data from services such as Google, Amazon, DoorDash, Strava, and Twitter through what Ario calls adversarial ETL. It then turns that data into an editable profile and conversation memory, which can inform recommendations, routines, and calendar management. Agarwal contrasts these results with generic answers from Copilot and ChatGPT, which often require users to answer many questions before becoming useful. He also explains several engineering lessons from building the system. Teams should handle simple transformations with conventional software, use search and processed profiles alongside vector data, and limit the data passed through each part of the pipeline. Ario also offers a browser extension with an offline developer mode that downloads data locally without creating an online account.

Key ideas
00:00

A personal assistant should give ordinary people time back

Agarwal frames Ario's purpose as giving each person who wants it one real hour back every day. He quotes the idea that people would rather have AI do their laundry than write poetry, because that would leave them free for creative work. The product is aimed at ordinary people managing everyday life, rather than only wealthy executives who can hire an assistant. Ario announced that it had raised $16 million to build this version of a personal assistant. Its proposed benefit starts with reducing the routine work people still have to do.

01:42

Ario begins with adversarial ETL because personal data is trapped in many services

The foundation of Ario is what Agarwal calls adversarial ETL, meaning the system works to retrieve a user's data even when the companies holding it do not make that easy. The examples include every Google search, Facebook post, DoorDash order, Strava run, and Instagram activity. Ario aggregates this information for the user, then places an agent architecture, an LLM, retrieval-augmented generation, and data management on top of it. The assistant can only become personal if the system first obtains the material that describes the user's actual life.

03:18

The profile is generated from behavior instead of a stale intake form

After a user connects data sources, Ario creates a data portrait and a written profile. Agarwal shows a profile that identifies interests such as photography, hiking, Japanese restaurants, daycare appointments, Lego toys, and beach trips. The system can also make mistakes, such as guessing that a child is in kindergarten, and the user can edit or correct those details. Agarwal says users do not need to complete an essay or answer 20 questions. The profile keeps changing as new activity and conversations provide more information about the person.

12:14

Personal context changes generic recommendations into advice tied to a person's life

In one comparison, Copilot gives a list of warm places to visit, while Ario connects possible destinations to the user's interests and circumstances. Maui is linked to Asian food and relevant activities, San Diego to the user's Lego purchases for a son, and Miami to activities that resemble things the user has done in the Bay Area. A morning-routine example uses the person's actual wake time, exercise habits, and DoorDash food preferences. Agarwal says the system can eventually combine schedule, household preferences, and budget instead of answering with generic suggestions.

11:19

Calendar assistance combines extraction with knowledge of the user's conflicts

Ario can read a photograph of a densely packed school or household schedule and publish its contents to the user's calendar. The system then compares those entries with existing calendar conflicts and warns the user 24 hours, 72 hours, and seven days in advance. Agarwal shows a school schedule with two possible first days because the school has not settled the date. Ario keeps both entries until the correct date arrives. He says a planned version would monitor email for this narrow category of information, remove the incorrect event, and notify the user.

13:20

Simple data transformations should happen outside the LLM

Agarwal's first RAG lesson is to avoid the LLM when conventional software can do the work. He describes Strava data with a run start time of 1:05 p.m. and an end time of 2:00 p.m. The model initially treated the duration as unknowable, then discovered that subtracting the two columns produced 55 minutes after extra prompting. That consumed prompt space for trivial arithmetic. Ario therefore tries several ordinary methods before asking the LLM, using the model for tasks where its judgment or language ability adds value.

15:08

Processed profiles and selective retrieval are more useful than dumping raw data into a vector store

Agarwal says search remains central as the system grows. Simply embedding raw Amazon or DoorDash records does not give the model enough structure to plan a good answer. Ario instead converts raw records into profiles and other processed data, then gives that refined information to the LLM. The system can return to raw records when it needs more detail. He also warns that making every piece of data available everywhere can overwhelm the pipeline. The team spends effort presenting the right data to the right part of the system at the right time.

16:25

The browser extension can download personal data without an online account

Ario Boost lets users apply the company's data-collection system without fully joining Ario. In developer mode, a user skips login, creates no online account, and downloads data from 15 or 20 sources. The extension uses screen scraping, APIs, and automated GDPR data downloads where appropriate. The data remains on the local device, where the user can search it, send it to another LLM such as Llama, or build another application. Agarwal presents this mode as a way to give people direct control over the data gathered on their behalf.

"Our goal at Ario is very simple, it's to give every single person that wants it one hour a day back in their lives."00:51
Who should watch
  • Engineers building personal agents who need a way to gather useful user context before asking users to fill out profiles or answer setup questions.
  • Teams designing RAG systems over activity logs, calendars, purchases, or other personal records will find concrete advice about preprocessing and retrieval.
  • Developers concerned about data ownership can compare Ario's hosted workflow with its local, no-login browser-extension mode.