A language model that answers a question is not an agent. A script that calls an API on a schedule is not an agent either, even if an LLM wrote the script. The term has a real technical meaning, and it's worth being precise about it, because the engineering effort (and the failure modes) of an actual agentic system are very different from a well-prompted chatbot.
The minimum definition
An AI agent is a system where a language model doesn't just generate text, it decides what action to take next, takes that action through a tool, observes the result, and decides again. That loop, observe, reason, act, observe, repeated until a goal is satisfied or a stopping condition is hit, is the actual distinguishing feature. Everything else (memory, multi-agent orchestration, planning) is built on top of that loop.
This pattern is often called ReAct (reason plus act) in the literature, though the exact prompting scaffold varies by framework. What matters architecturally is the four components underneath it:
- A reasoning core, the LLM, deciding what to do next based on the current state and goal.
- Tools, functions the model can call: a database query, an internal API, a search call, a file write. The model doesn't execute these directly, it emits a structured request, your system executes it, and returns the result.
- Memory, short-term (the current task's context and intermediate results) and sometimes long-term (facts or preferences persisted across sessions).
- A control loop, the code that actually runs the observe-reason-act cycle, handles tool execution, enforces limits, and decides when to stop.
None of this requires the model to be "smarter." It requires the surrounding system, tool definitions, execution sandboxing, state management, error handling, to be engineered correctly. Most agent failures in production are not reasoning failures, they're systems-engineering gaps: a tool that returns an ambiguous error, a loop with no termination condition, a missing permission check.
When an agent is the right tool
Agents earn their complexity when a task has three properties at once:
- Multi-step, the task can't be done in one call; it requires gathering information, then acting on it, then possibly correcting course.
- Variable path, the sequence of steps isn't known in advance and depends on what earlier steps return. If the path is fixed, you don't need an agent, you need a script.
- Tool-integrated, the task requires touching real systems (your database, an internal API, a ticketing system), not just producing text.
A support ticket that requires checking an order status, cross-referencing a shipping API, and drafting a resolution based on both, that's a legitimate agent use case, the path genuinely depends on what the first two calls return. A form that extracts three fields from a document and writes them to a database is not an agent use case; it's a well-scoped extraction pipeline, and wrapping it in agent tooling adds latency, cost, and failure surface for no benefit.
Single-agent vs multi-agent architecture
Most production systems that genuinely need agentic behavior are well served by a single agent with a well-defined tool set. Multi-agent architectures, where several agents with different roles coordinate, sound appealing but introduce real coordination overhead: message passing between agents, shared state consistency, and compounding failure probability, if each agent has a 95% success rate on its sub-task, a five-agent pipeline's end-to-end reliability drops fast.
Multi-agent patterns earn their complexity in narrower cases: when sub-tasks genuinely benefit from different tool access or different context windows (a research agent with broad read access feeding a writing agent with narrow write access, for example), or when running sub-tasks in parallel meaningfully reduces latency. Reaching for multi-agent orchestration as a default, before establishing that a single well-scoped agent can't do the job, is one of the more common ways we see agent projects become unreliable and expensive to debug.
The part that actually needs engineering rigor
| Concern | What it means in practice |
|---|---|
| Tool sandboxing | An agent with write access to production data needs the same access controls a human employee would have, not more. |
| Approval gates | High-stakes or irreversible actions (sending an email, issuing a refund, deploying code) should require explicit confirmation, at least until the system has a track record. |
| Loop termination | Every agent needs a hard step limit and a defined failure state. An agent that can retry indefinitely is a cost and reliability risk, not a feature. |
| Observability | Every tool call, every intermediate reasoning step, needs to be logged. When an agent does something wrong, you need to see the full trace, not just the final output. |
| Evaluation before production | A held-out set of realistic tasks, run repeatedly, with success criteria defined before you look at the results, not after. |
Where a human still belongs in the loop
The honest framing for most business-critical agent deployments in 2026 is: agents are excellent at gathering information, drafting actions, and handling the well-trodden 80% of cases, and a human reviewer is still the right call for the ambiguous or high-stakes 20%. Systems that route confidently-handled cases through automatically and flag uncertain or high-impact ones for review tend to outperform systems that try to remove the human entirely, both on reliability and on how quickly the business will actually trust the system enough to rely on it.
The interesting engineering problem in agent development isn't getting the model to reason well. It's building the tool interfaces, guardrails, and observability around it so that when it doesn't reason well, the blast radius is small and visible.