Agentic AI: When It Genuinely Helps and When It Is the Wrong Tool

It worked in the demo. It works six times out of ten.

Here is the situation that usually precedes someone reading an article like this. An agent was built. In the demo it took a request, called five tools, and produced a correct answer while everybody watched. It was convincing enough to get funded. Now it is in front of real users, and the same request produces a different sequence each time. Sometimes it is right. Sometimes it calls the wrong tool, gets a confusing result, and spends nine more steps recovering from it. Nobody can say in advance which run you are going to get.

The instinct at this point is to reach for the prompt, the model, or a framework with a better reputation. Those are occasionally the right levers. Far more often the problem is architectural: autonomy was applied to a task that did not need it, or applied without the two things that make it safe — a way to check the work and a way to undo it. Telling those cases apart before the build is much cheaper than after.

What “agentic” actually means

The word has been stretched to cover almost anything that calls an API. The most useful definition in circulation comes from Anthropic's engineering write-up Building effective agents, which splits the category in two. Workflows are “systems where LLMs and tools are orchestrated through predefined code paths.” Agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.”

The distinction is not about how many tools you have or how capable the model is — both kinds of system can call ten tools. It is about who decides the order. In a workflow your code does, and you can read the sequence in a file. In an agent the model does, at runtime, and the sequence is an output rather than a specification.

Anthropic names five workflow patterns that cover most production systems people call agents: prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer. If your system routes a request to one of four handlers and then runs a fixed three-call chain, that is a workflow, and calling it an agent mostly costs you clarity. The same post is blunt about the default — “we recommend finding the simplest solution possible, and only increasing complexity when needed” — and notes that “agentic systems often trade latency and cost for better task performance.” Its criterion for when an agent is warranted is worth memorizing: cases where “it's difficult or impossible to predict the required number of steps, and where you can't hardcode a fixed path.”

The arithmetic nobody runs before the pilot

Multi-step autonomy has a mathematical property that sinks projects quietly. Suppose each step succeeds 95 percent of the time, and pretend steps fail independently. Twenty steps gives roughly a 36 percent chance the whole run is clean; at 99 percent per step it is about 82 percent; ten steps at 95 percent is about 60 percent. That is arithmetic, not a benchmark — run the exponent yourself — and independence makes it a best case, because a bad retrieval at step three poisons every judgment after it.

It is also why a demo tells you so little: a demo is one sample from a distribution, usually one somebody selected. What matters is consistency across repeated attempts at the same task, and there is a benchmark built on exactly that. τ-bench (Yao, Shinn, Razavi, and Narasimhan) simulates conversations between a user and a tool-using agent under domain policy, compares the final database state against an annotated goal state, and introduces a pass^k metric “to evaluate the reliability of agent behavior over multiple trials.” Its reported result is the part to sit with: “even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail).”

Model quality has moved since that paper and will move again. The structural lesson does not: capability and consistency are different quantities, and the one users experience is consistency. If your evaluation reports one success rate from one run per task, you measured what impresses a room, not what decides whether the system is usable. Run every task k times and report the fraction that succeeded all k times. Our guide on how teams run agent evals covers building that harness.

The three conditions where autonomy pays

Autonomy earns its cost when all three of the following are true. Not one. All three.

1. The path cannot be enumerated in advance. Anthropic's criterion, and the one people over-claim. Most business processes are enumerable; you just have not written them down. The genuine cases are where the second step depends on what the first returned in a way you cannot branch on ahead of time — investigating an incident across systems you did not know you needed until you read the first log, or searching a codebase where the next file depends on what the last one said.

2. A cheap, machine-checkable verifier exists. This decides most projects and gets discussed least. Coding agents work as well as they do partly because a test suite and a compiler tell the model it is wrong without a human in the loop. If your task has no equivalent — no schema to validate against, no query that either returns rows or does not, no assertion that passes or fails — every autonomous step is an unchecked step. Where verification is expensive, keep the human and drop the autonomy.

3. A wrong action is recoverable. Reads are cheap to get wrong. Writes are not. An agent that drafts an email for review or opens a pull request is working in recoverable space. An agent that sends the email or merges the branch has turned a model error into an institutional fact. The design question is not “is the model good enough” but “what does the third-worst run do, and can we take it back?”

When all three hold, autonomy buys what a workflow cannot: the long tail of cases nobody enumerated, and the tedious middle of a task while a person reviews the ends.

Where a fixed workflow wins, and what to build instead

Four situations where the deterministic system is better engineering, and the agent is the expensive way to a worse result.

• The steps are the same every time. Ingest a document, classify it, extract eight fields, validate, route to a queue. That is a pipeline. Written as code with model calls at the judgment points, it is testable and debuggable. Written as an agent, you pay extra tokens for the model to rediscover your own process on every request, with a chance of getting it wrong.
• Volume is high and per-request margin is thin. An agent's token spend is a distribution, not a number, and the tail is where the budget goes. A fixed chain has a cost you can multiply. Our breakdown of what an AI project actually costs is the companion to this point.
• The action is regulated or irreversible. When someone eventually asks why a decision was made, you want a code path they can read, not a transcript to interpret. Determinism is an audit feature.
• The same input must produce the same output. If two identical requests producing two different sequences is a defect rather than a curiosity, autonomy is working against your requirements.

What to build instead is not “no AI.” It is the same models doing the parts that need judgment — classification, extraction, drafting — with the sequencing lifted out of the model and into code. You are taking back control flow, not intelligence. For tooling in either shape, the agent frameworks comparison lays out what the main options assume about your architecture.

The failure modes, named

Tool sprawl. Every tool widens the set of choices the model must get right at every step, and tools with overlapping descriptions are the ones it confuses. Few tools, narrow scope, typed inputs, error messages written for the model to act on. The Model Context Protocol, an open standard for connecting AI applications to external systems, settles how tools are exposed — not which ones you should expose, which stays your judgment. Our note on what MCP settled goes further.

The write path. Separate read tools from write tools at the permission layer, not in the prompt. Allow-list the actions the agent may take, and gate irreversible ones behind a human confirmation showing the exact action rather than a summary.

Prompt injection. This one is specific to agents, because an agent reads content it did not author and then acts on it. OWASP's entry for LLM01, Prompt Injection separates direct injection, where a user's own input changes model behavior, from indirect injection, where content pulled from a website or file does. Its assessment is the sentence to quote to anyone promising a filter solves it: “Given the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention for prompt injection.” The mitigations OWASP recommends are architectural — privilege control, human approval for high-risk actions, segregating untrusted content, validated output formats, adversarial testing.

The sharpest framing of the risk is Simon Willison's lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. Any two are usually survivable; all three in one agent means an attacker who can get text in front of the model may be able to get your data out. His conclusion — “we still don't know how to 100% reliably prevent this from happening” — is worth reading before you wire up a connector. Break the trifecta by design: deny the outbound channel, isolate the untrusted content, or scope down the reachable data. NIST's Adversarial Machine Learning taxonomy (NIST AI 100-2 E2025, March 2025) gives these attack classes common names for a security review.

Unexplainable behavior. If you cannot reconstruct a run — every tool call, its inputs, its outputs, the stated reason for choosing it — you cannot debug the failure or defend the decision. Log the trace from the first commit; retrofitting it later is the same job with worse data. What to monitor in an LLM system covers the specifics.

Federal readers face a further constraint on how much autonomy is available. OMB Memorandum M-25-21, issued April 3, 2025, rescinded and replaced M-24-10 and set minimum risk management practices for high-impact AI, including ongoing monitoring, additional human oversight and intervention, and consistent remedies or appeals. Unbounded autonomy over a consequential action is the design that memo constrains, which makes the human gate a requirement rather than a preference.

A checklist before you build

Answer the first three in writing before anyone writes code. The rest separate systems that hold up from systems that impress once.

1. Can you write down the steps? If yes, write them down — that is your workflow, and you are done deciding.
2. How does the system know it succeeded, without a human looking?
3. Does it touch private data, untrusted content, and an outbound channel? If all three, which one are you removing?
4. Bound the loop — a maximum step count, a maximum wall-clock time, and a defined escalation at the ceiling.
5. Add a verifier wherever one is cheap — schema validation, a test run, a second model checking output against source. Anthropic's evaluator-optimizer pattern is this idea in workflow form.
6. Log every step with inputs, outputs, timing, and token cost, structured for querying rather than reading.
7. Keep the deterministic baseline as your control. Build the workflow version, measure both with pass^k on a frozen task set, and require the agent to beat it by enough to justify its cost and variance. Often it does not — and learning that in week two is a good outcome.

If item one has a clear answer and item two does not, you do not have an agent problem. You have a workflow, and building it as one will be faster, cheaper, and easier to defend.

Sources: Anthropic — Building effective agents; Yao, Shinn, Razavi & Narasimhan — τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (arXiv:2406.12045); OWASP GenAI — LLM01: Prompt Injection; Simon Willison — The lethal trifecta for AI agents; NIST AI 100-2 E2025 — Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations; Model Context Protocol documentation; Wiley — analysis of OMB M-25-21. The compounding-reliability figures in this article are arithmetic from a stated assumption, not measurements. Analysis and framing by Precision AI Academy.

Common questions

What is the difference between an AI agent and a workflow? Anthropic's engineering guidance draws the line by who chooses the next step. Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. Both can call tools many times; only the agent decides the sequence at runtime.

Why do agents that work in a demo fail in production? Per-step reliability compounds across a long chain, and a demo shows one successful run rather than the distribution. The τ-bench paper introduced a pass^k metric for exactly this reason and reported that state-of-the-art function-calling agents such as gpt-4o succeeded on under 50 percent of tasks and were inconsistent across repeated trials, with pass^8 under 25 percent in the retail domain.

When is agentic AI the wrong choice? When the sequence of steps is the same on every run, when there is no cheap machine-checkable way to verify the result, when the actions are irreversible and cannot be gated by a human, or when cost and latency have to be predictable. In those cases a routed, chained workflow with the sequencing written in code is cheaper, faster, testable, and explainable.

Can prompt injection be fully prevented in an AI agent? OWASP states that given the stochastic influence at the heart of how models work, it is unclear whether there are fool-proof methods of prevention. The practical response is architectural: enforce privilege control, require human approval for high-risk actions, segregate untrusted content, conduct adversarial testing, and avoid combining private data access, untrusted content, and outbound communication in one agent.