Your Agent Hates Your CLI

Ayush Bhardwaj, Allos AI12:20 · Nov 2026 · 1,785 views
Thumbnail for Your Agent Hates Your CLI Watch on YouTube
TL;DR
  1. 1

    Repeated pipes and filters in agent calls reveal missing CLI features and hidden assumptions about output.

  2. 2

    Documentation, help text, and error messages are part of the agent-facing interface, so stale instructions can make new flags ineffective.

  3. 3

    Real transcripts can become replay tests, but CLI changes should be judged by answer quality and agent turns, not token reduction alone.

Summary

Ayush Bhardwaj examines his own coding-agent transcripts to see how agents work around poor CLI interfaces. In roughly 20,000 calls across 14 CLIs, about a quarter of outputs were sent through filters such as head, grep, tail, Python, sort, or wc. For read-heavy commands, the rate was higher. These repeated workarounds point to missing features, such as searching around a match, returning compact results, and inspecting several sessions in one call. He also explains how pipelines can hide failures when the final command returns success or removes the useful error lines. A CLI change is ineffective if the skill file and help text still teach the old interface. Bhardwaj recommends turning recorded calls into replay tests, keeping some cases for evaluation, and measuring whether agents get better answers with fewer turns. Smaller outputs can save little when they do not change the work the agent performs.

Key ideas
00:00

Repeated pipes expose missing CLI features

Agents often work around awkward output by downloading or printing too much and then using head, tail, grep, Python, or other filters. Ayush gives examples from Claude Code, Codex, and Cursor, where agents cut large outputs to find settings, dependency totals, or test failures. A single workaround says little, but the same command and pipe appearing repeatedly shows that the CLI is missing a useful operation. Agents usually do not file a bug report. They write a small program around the tool instead.

01:37

Pipelines can hide failures from both agents and users

Ayush found that a quarter of calls in his sample fed output into filters, with more than half of read-heavy commands doing so. In Bash, a pipeline without pipefail returns the exit code of its last command. Piping cargo test into head can therefore report success even when tests failed. Tail can also remove the actual error and leave only harmless lines, leading the agent to report success. His fix is to make failures loud: put a short failure summary and a pointer to the full log in the final lines of the output.

02:59

The chat-history CLI showed where output was wasting agent effort

Ayush built a CLI that lets coding agents search sessions from Claude Code, Codex, and Cursor. In an audit of about 1,000 calls, four out of five times an agent opened an old session, its output went directly into another filter. The full session could contain 1.2 million characters, while the messages around a search match were about 3,000 characters. The repeated pipe was a tax: it kept junk out of context, but forced the agent to guess the right cut each time.

04:32

Small interface additions can replace repeated workarounds

Ayush changed the CLI based on the patterns in those calls. The read command could return messages around a match, only matching messages, or a limited final slice. A compact mode returned one line per hit and was about six times smaller than JSON with the same results. An inspect command replaced agent-written loops over several sessions with one batch request. He also cites an infrastructure-cost tool that moved filtering into the CLI and added a compact format and skill instructions.

06:33

Documentation and help text are part of the agent interface

A feature does little if the agent is not taught to use it. Ayush made his deep flag a no-op while leaving the old skill instructions in place. In 95 out of 95 searches, agents still passed the flag. After he rewrote the skill without that instruction, new sessions stopped using it and tried the new flags on their first read. He treats skill files, help text, and error messages as part of the interface. Stale documentation is therefore a functional bug.

07:36

Transcripts can become replay tests

The commands agents actually ran provide test cases for a CLI. Ayush recommends replaying those calls against the old and new builds, then checking what the agent did next. Output-format changes should be checked for missing information. Default changes should be checked for altered behavior. A retry with different arguments is often a failing case. The next action is only a rough answer key, so some examples should be held back and used to score the change rather than to guide it.

08:26

Start with slices and limits before adding complex features

Ayush says 55% of the measured pipelines used only tail or sed downstream, which points to demand for smaller slices. He would ship ranges and limits first. A GitHub CLI list command's limit flag was used by an agent 81% of the time. By contrast, cargo test's quiet mode was used on about one run in ten, while agents usually piped the output instead. A flag needs to be visible in help and skills, and large outputs should explain how to retrieve more.

09:19

Token reduction is not enough to prove an agent interface improved

An agent mode can format output differently when a CLI detects that an agent is calling it. Ayush describes DuckDB returning plain output, reporting omitted rows, explaining how to fetch them, and returning errors as JSON. One example cut the model's output reading by 59%, but the saved tokens were only about 0.5% of all tokens read. Cost and runtime did not improve. He says evaluations should measure turns and correct answers as well as tokens.

10:14

People should design the interface and use agents to test it

Ayush argues that asking an agent to design an agent tool often produces a tool aimed at an imagined person, built from common patterns that may be poor. He cites studies that found problems in most tool descriptions and agent-written context files. Agents are more useful when given a transcript and a test they can run. They can inspect what really happened and find improvements. The human should decide what the tool ought to do, then use the agent to check that design against real traces.

"Your agent transcripts are the product analytics you probably aren't reading. The workaround is the bug report."11:56
Who should watch
  • You maintain a CLI, MCP server, or developer tool that coding agents call and want to find usability problems without waiting for bug reports.
  • Your agents frequently pipe, filter, retry, or repeat calls, and you need a practical way to turn those traces into interface changes.
  • You are building evaluations for tool use and need to distinguish smaller responses from genuinely better answers and shorter workflows.