# Critical AI Inference Your CIO Can Trust

Sahil Yadav & Hariharan Ganesan, Telemetrak | AI Engineer World's Fair 2025 | 19:04

Source: https://www.youtube.com/watch?v=6Tpm4m1YxHk
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/critical-ai-inference-your-cio-can-trust
Published: 2025-07-22
Tags: guardrails, human-in-the-loop, observability, reliability

## TL;DR
- Trustworthy AI needs explanations, traceability, and guardrails that can stop or redirect risky decisions.
- XTOPS extends MLOps with trust-specific monitoring, adaptive controls, and planned human intervention.
- A worker-safety case study shows how GPS drift created false alerts, and how attribution telemetry could shorten resolution from months to days.

## Summary
Sahil Yadav and Hariharan Ganesan argue that enterprise AI adoption is moving faster than AI governance. They describe failures in telecom, gas sensing, and supply chains where opaque or incorrect inferences created financial and safety risks. Their framework has three parts: every important decision should have an understandable explanation, every data and model change should be traceable, and guardrails should slow, stop, or redirect the system when it drifts. They call the operational model XTOPS, an extension of MLOps with trust dashboards, adaptive controls, and defined human intervention. Two measures put the work in operational and financial terms: mean time to resolve explainable errors and trust-adjusted risk in dollars. A Guard Hat worker-safety example shows how GPS drift caused 70% false positives and led users to ignore alerts. The speakers say their later framework could reduce this type of resolution process to seven days, although the original incident took eight months to solve.

## Key ideas
### AI adoption is outpacing AI governance
[01:01](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=61s)
Sahil says 78% of companies are adopting AI and another cited study says 95% are investing in it, while only 11% focus on AI governance. He describes the difference as a 67% gap. The risk is practical rather than theoretical. An AI decision can disrupt a telecom network, cause a gas sensor to misread a dangerous situation, or scramble supply-chain SKUs. These failures may remain silent until their costs have accumulated. Telecom operators can spend millions of dollars for each minute of downtime, while safety failures can put lives and company liability at risk.

### Trustworthy AI needs explanations, traceability, and guardrails
[02:54](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=174s)
Sahil defines three pillars for trustworthy AI. Explainability shows why an inference was made and what evidence supported it. Traceability works like a flight recorder, preserving audit trails so a situation can be reconstructed. Guardrails set a threshold at which the AI must stop rather than continue making risky decisions. Hariharan adds that important decisions should come with a simple English explanation, so an operator or auditor can act without finding a data scientist to translate the result.

### Adaptive controls and planned human intervention limit harm
[04:43](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=283s)
Hariharan describes adaptive control as a smart guardrail. If a system starts to drift or makes a wrong decision, it should slow down, change course, or call a person for help. Human involvement also needs defined roles and playbooks. The right expert should receive the right information at the right time, without creating unnecessary work. Traceability supports this process by digitally signing and tracking data and changes, much like a software bill of materials or a FedEx package moving through a delivery network.

### XTOPS adds trust controls across the AI lifecycle
[06:05](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=365s)
The speakers call their approach XTOPS, an MLOps model with built-in oversight. At the data stage, teams record where data came from and how it changed. During training, models are developed to provide explanations so teams can detect reasoning drift. At deployment, adaptive controls respond to new data and can pause the system. Human-AI teaming supplies feedback and lets people intervene. Their test for the system is that every decision has a clear why, when, and who attached to it.

### XTOPS changes existing MLOps components instead of replacing them
[08:11](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=491s)
Hariharan says XTOPS builds on familiar MLOps modules while adding trust-specific behavior. Its guardrails understand AI context and can block a risky decision. Its dashboards report trust measures in a form that leadership and boards can understand. Human review moves from ad hoc intervention toward click-to-fix workflows for quick changes. The stated benefit is less time spent responding to unpredictable behavior and more time spent building products.

### Trust needs measures tied to time and business cost
[09:36](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=576s)
The speakers propose mean time to resolve explainable errors, or MTRE, as a measure of how quickly a team can understand an unexpected result and fix it. They also propose trust-adjusted risk in dollars, which estimates the business cost of a trust failure through fines, lost customers, or reputational damage. Hariharan says some incidents take months to resolve, giving a biased decision time to cause escalating harm. He cites a serious privacy or credit-card bias incident that could reach $700 million in combined costs.

### GPS drift made workers ignore safety alerts
[12:25](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=745s)
Sahil describes Guard Hat, a worker-safety platform using wearable and environmental sensors in hazardous workplaces. GPS was one input to the AI, but it contributed to false positives in 70% of cases. Workers began ignoring alerts, creating a safety and liability problem. Without the proposed framework, Sahil estimates that 70% of the response time would go to finding the problem and another 20% to finding a solution, with no system to identify the GPS drift. Attribution telemetry would flag the anomaly, after which a fix or GPS fallback could be deployed.

### The CIO case has to be expressed in financial terms
[15:55](https://www.youtube.com/watch?v=6Tpm4m1YxHk&t=955s)
To explain the system to CIOs, the speakers frame trust as reduced financial exposure. They cite approximately $2.5 million in risk exposure per site per year and $500,000 in annual fines avoided per site. The corrected system also reduced false alerts, increased the trust score, and let workers rely on the alerts again. A dashboard notified a person when intervention was needed, and that person could act and retrain the model. The speakers admit that the original GPS problem took eight months to solve, even though the completed framework could solve similar problems in seven days.

## Notable quotes
- "You have to understand why those inferences are being made and on what basis." (02:54)
- "XTOPS is about creating a system where every AI decision has a clear why, a when and a who attached to it." (07:28)
- "The faster your MTRE is, the team is more agile, less defects in the product and quicker to solve the problems." (10:04)
- "It took 8 months for us, but this was a model problem that actually helped us to build this framework." (15:35)
- "Reputational damage is again right on top of that list. So it's really hard to measure that, to be honest." (18:36)

## Tools & references mentioned
- McKinsey
- Guard Hat
- Telemetrak
- MLOps
- XTOPS
- software bill of materials
- FedEx
- AT&T
- Verizon

## Who should watch
- You are responsible for deploying AI in safety-sensitive or revenue-sensitive operations and need controls that operators can use when a model drifts.
- Your MLOps stack records model performance but does not explain decisions, preserve a full audit trail, or define when a human must intervene.
- You need to explain AI investment to a CIO using incident costs, fines, resolution time, and site-level risk exposure.

## Related talks

- [How to Build Trustworthy AI](https://aietalks.com/talks/how-to-build-trustworthy-ai) (Allie Howe, Growth Cyber, 24:22)
- [AI Frontiers in Trust and Safety: Combatting Multifaceted Harm on Tinder at Scale](https://aietalks.com/talks/ai-frontiers-in-trust-and-safety-combatting-multifaceted-harm-on-tinder-at-scale) (Vibhor Kumar, Tinder, 14:36)
- [From Copilot to Colleague: Trustworthy Agents for High-Stakes](https://aietalks.com/talks/from-copilot-to-colleague-trustworthy-agents-for-high-stakes) (Joel Hron, Thomson Reuters, 19:45)
- [Local Models: Trust, Control, Optimization](https://aietalks.com/talks/local-models-trust-control-optimization) (Carter Abdallah, NVIDIA & Vincent Weisser, Prime Intellect & Lucas Atkins, Arcee AI & Chris Alexiuk, NVIDIA, 43:21)
- [Understanding AI Stakes to Break Production Code](https://aietalks.com/talks/understanding-ai-stakes-to-break-production-code) (Philip Rathle, Neo4j, 23:24)