AI Engineer NYC Livestream

Thumbnail for AI Engineer NYC Livestream Watch on YouTube
TL;DR
  1. 1

    Production agents need bounded autonomy, runtime controls, identity, evaluations, and access to the right data.

  2. 2

    The useful unit of AI engineering is shifting from individual features to systems that continuously evaluate, monitor, and improve agent work.

  3. 3

    Financial firms can gain more from redesigning workflows around judgment and evidence than from adding copilots to existing tasks.

Summary

This livestream brings together talks on production AI in finance, wealth management, banking, infrastructure, security, and investing. Bridgewater describes Vault, a governed corpus of structured and unstructured research that gives humans and agents time-aware evidence. Two Sigma argues that agents need composable controls rather than either total lockdown or unrestricted access. Other speakers show how to run long-horizon business processes, let subject-matter experts define agent skills, and combine deterministic workflow engines with model judgment. The event also covers tabular foundation models for fraud detection, GPU timing side channels, task-scoped authorization, evaluation design, self-hosted open models, and infrastructure for large numbers of sandboxes. The closing talks widen the frame. Coatue treats a hedge fund as a function that transforms internal and external context into investment decisions. General Intuition presents models that map pixels to actions in games and robots. Nous Research argues that companies should build on open foundations they can modify and own.

Key ideas
07:36

Useful agents depend on governed, time-aware context

Bridgewater built Vault because public web search could return relevant-looking material without the right date, coverage, or evidence. Vault indexes external and internal unstructured data, aligns it to Bridgewater's ontology, and adds metadata such as countries, concepts, markets, and time periods. Sam Green shows the contrast between a web answer using one or two documents and a Vault-backed answer using a wider candidate set from a specified period. Aaron Linsky says humans need precision while agents need scale, so search must support both. The team also moved familiar email habits into saved searches and collections before adding proactive investor workflows.

24:07

Agent safety should preserve useful autonomy

John Feminella uses the 2020 Revlon payment error to explain why controls matter. A bank's system sent about $900 million to lenders instead of about $8 million after operators overrode a warning. His answer is not to sandbox everything or rely on policy text in prompts. Two Sigma uses an agent registry, risk-based review, declared tool access, shared evaluations, and runtime classification. The registry records ownership, endpoints, tools, and product context. Low-risk agents should not wait behind agents that touch production. Routine actions can follow deterministic rules, while ambiguous or dangerous actions reach a person.

01:01:13

AI engineers increasingly build the improvement loop

Aparna Dhinakaran describes Signal, an always-on agent that reads traces, identifies problems, and can create a pull request. Each run receives traces, repository access, and other telemetry inside a sandbox. Its findings are saved as memory while credentials and the sandbox disappear after the run. Signal can restart from saved state if a harness or sandbox fails. Dhinakaran says the AI engineer's work is moving toward tracing agents, writing evaluations, evaluating the evaluators against human judgments, and feeding production telemetry back into the system. The product supports multiple coding harnesses and models so teams can compare quality and cost.

01:55:28

Long-horizon business work needs signals, memory, and people at the right points

Arca's agents handle work that unfolds over days rather than in one long model run. Aman Singh describes a tax-planning example involving an adviser, trader, operations staff, settlement delays, client responses, and changing information. Background agents read emails, documents, meetings, custodian events, and market movement. A harness supplies tools, operational context, and persistent memory. Agents register interest in future events and wake when a response, task update, or external event arrives. People approve or perform the actions that require them, while agents track the rest. Singh's test for an AI-native firm is whether agents can drive business goals by preparing work for people.

04:16:32

Workflow systems should make judgment typed and side effects deterministic

Karthic Rao argues that regulated workflows need a deliberate boundary between model judgment and deterministic execution. A model translates a request into a typed semantic graph. A compiler checks that graph against the firm's allowed vocabulary and either produces an immutable plan, asks for clarification, or refuses. The state machine controls evidence, deadlines, approvals, retries, and external side effects. Models handle small judgments such as identifying a missing field or deciding whether a reply is complete. Sending an email, filing a form, or writing a record requires an explicit gate. Because model calls have fixed schemas, most judgments can eventually move to privately hosted smaller models.

05:06:58

Evaluation must measure real work, cost, and failure modes

Rayan Krishnan says model cards are insufficient for choosing systems for financial work. Evaluations need valid tasks, enough samples for reliable results, and enough difficulty to distinguish models. Partial credit can hide a serious problem if the output is passed directly to a client, so fully correct answers may be the better metric for some workflows. Agents can game tests by using other models, hedging with multiple answers, or exploiting the sandbox. Krishnan describes replaying historical GitHub work to compare tools and models, then using the results for routing, procurement, usage limits, and fine-tuning. Institutional knowledge becomes useful only when it is captured in reusable evaluation data.

06:02:01

The advantage in AI adoption comes from judgment and redesigned workflows

Sid Pednekar says access to AI is widespread, while business impact remains uneven. His framework has three parts: choose the right problem, build shared capabilities, and compound the advantage. He contrasts a firm that adds summarization to an existing research process with one that asks why important signals arrive late and redesigns the workflow around that bottleneck. Leading firms concentrate investment on high-value workflows, connect technology teams with business specialists, and standardize shared infrastructure while personalizing the edges. They also preserve evidence from successes and failures so an individual's learning becomes part of the system.

06:46:41

Agents are becoming software systems that combine code, models, data, and judgment

Frank Long frames a hedge fund as a function that takes external and internal context and produces a decision about selection, size, and timing. The work includes gathering, filtering, analysis, and decision formation, with different steps assigned to deterministic code, statistics, language models, or people. Coatue's historical notes, decisions, and outcomes provide internal context alongside news, prices, and filings. The right goal is not to turn every box into an LLM. It is to find the best runner for each part and evaluate the resulting system. The same pattern appears across the event's other talks: useful agentic applications combine deterministic components with model-based interpretation.

07:12:37

Open and purpose-built foundations can reduce dependence on rented intelligence

Pim de Witte presents General Intuition's models as continuous pixel-to-action systems for games and robots. They learn from video-game recordings that pair visual frames with controller actions, then predict actions while continuously observing the environment. Dillon Rolnick makes a related argument for open agent foundations. He says models and software are becoming easier to swap and reproduce, so companies should own the layer where their workflows, data, policies, and judgment live. Hermes Agent is offered as an open foundation that can run where the user chooses and remain replaceable at the model layer.

"The goal is not to take as many things and make them LLM as possible for us. The goal is about the function."Frank Long6:56:39
Who should watch
  • You are building agents for banking, investing, wealth management, insurance, or another regulated workflow and need concrete patterns for access control, evidence, approvals, and failure handling.
  • Your team has deployed copilots or coding agents but lacks a repeatable way to evaluate them, monitor production behavior, and turn failures into improvements.
  • You are deciding whether to buy hosted models, self-host open models, or build on open agent infrastructure while keeping control of data and workflows.