The recipe book we wish every company had before its first AI agent pilot: context engineering, evals, durability, and the team design that gets agents shipped.
Leonardo Piñeyro
CTO
The hard part of shipping an AI agent is the stretch between a demo that wows the boardroom and a system that survives real traffic on a Tuesday morning. That stretch, quiet and unglamorous, is where most projects stall.
The numbers back this up. By early 2026, practitioners at QCon London cited reports that as many as 80 percent of firms see no tangible benefit from their AI initiatives.1 Analyst syntheses put the failure rate of multi-agent pilots at roughly 40 percent within six months.2 Frontier models are extraordinary. The real gaps sit in the engineering and the organization around them, and both are solvable.
For anyone weighing where to spend the next million dollars of build budget, here is the reframe that matters most: An AI agent is a distributed system and an organizational commitment first, and a clever prompt a distant last. The teams shipping agents that move KPIs treat context engineering, evaluation, durability, and team design as the real work. The prompt is the easy part.
This is the recipe book we wish every company had before its first pilot. It's opinionated, and while we cite what Anthropic, OpenAI, and Google publish along the way, the insights come first from our own experience shipping agents for clients at Pento. Read it as a sequence of decisions, each one cheaper to get right early than to fix in production.
A demo proves an agent can do a task once. Production asks it to do the task 10,000 times, cheaply, safely, and in a way you can audit when a customer complains. Those are different problems.
The failures are rarely dramatic. They're death by a thousand cuts. A misrouted support ticket here. A contract clause the agent missed there. A token bill that triples the month a feature goes viral. An agent that loops on a malformed input until someone notices the cloud invoice. Each cut is survivable. Together they erode the trust and the unit economics that justified the project.
The good news is that the levers that prevent this are well understood in 2026, and most of them sit outside the model. They're about how you feed the model, how you measure it, how you contain it, and who owns it. Let's walk through them in the order you should make the decisions.
The most reliable advice across every credible source in this space is the same. Start with the simplest thing that works, and add complexity only when evaluation proves you need it. Anthropic frames the principle as building effective agents through "minimum viable complexity."3 OpenAI's practical guide makes the same point from the other side, telling teams to validate that a use case needs agentic reasoning before building an agent at all.4
The distinction that matters is between a workflow and an agent. A workflow is an LLM orchestrated through predefined code paths: classify, then route, then summarize. An agent is an LLM that directs its own tool use in a loop, deciding what to do next based on what it just saw. Workflows are cheaper, more predictable, and far easier to debug, because you only pay for model reasoning at decision points you chose. Agents are for problems where the path genuinely can't be predicted in advance.
For a leader, this is a budget decision before it's a technical one. Agentic reasoning costs real money per step, and most business processes are stable enough to encode as a workflow. The expensive mistake is reaching for an autonomous agent when a deterministic pipeline with one LLM call would have been faster, cheaper, and auditable. Reserve agents for the ambiguous, multi-step, high-value workflows where rules-based automation has failed.
The rule of thumb: If a competent engineer can draw the decision tree on a whiteboard, build the workflow. If they can't, you may have a real agent use case.
Here is where context engineering comes into the picture, and it's the single highest-return investment you can make in an agent.
Every model has a finite context window, the working memory it can attend to at once. It's tempting to treat that window as free space and stuff it with everything that might be relevant: the full document, the entire chat history, every tool result. That instinct backfires. Performance degrades as the window fills, a phenomenon Anthropic and others now call "context rot."5 Past a certain point, every extra token costs you accuracy as well as money.
Context engineering is the discipline of finding the smallest set of high-signal tokens that produce the outcome you want. It's the evolution of prompt engineering for the agent era, and it covers everything the model sees: the system instructions, the retrieved documents, the tool definitions, the running memory, and what you choose to leave out.
In practice it comes down to a handful of concrete techniques. Compaction summarizes a long conversation as it approaches the window limit, then reinitializes with the summary plus the most recently used files. Structured note-taking writes progress to external storage and retrieves it on demand instead of holding everything in active memory. Sub-agent isolation gives each helper its own clean window and passes back only a summary. And a good system prompt sits at the right altitude, specific enough to guide behavior yet general enough that it doesn't shatter the first time reality varies.
A concrete example makes it tangible. Picture an agent that reviews supplier contracts. The naive build dumps the entire 80-page agreement plus every past email thread into the window and asks for a risk summary. It's slow, expensive, and it misses things, because the signal is buried in noise. The engineered build retrieves only the clauses that match the risk policy, pairs each clause with the specific standard it should be measured against, and returns a citation for every flag. Same model, same question. The difference in accuracy and cost is the difference between a tool the legal team trusts and one they quietly ignore.
For your business, this is the lever that quietly determines cost, latency, and accuracy all at once. A well-engineered context is smaller, so it's cheaper per call and faster to first response. It's also more accurate, because the model isn't distracted by noise. In our client work at Pento, context engineering has moved agents from "impressive but unreliable" to "boring and dependable" without touching the underlying model. That move is the whole ballgame.
Multi-agent architectures are the most over-applied pattern in the field. They're genuinely powerful for a narrow band of problems and a costly mistake everywhere else.
The evidence is striking. Anthropic's own multi-agent research system, with an orchestrator delegating to parallel workers, beat a strong single agent by a wide margin on internal research tasks, but it consumed roughly 15 times the tokens of a normal chat interaction, and token usage alone explained about 80 percent of the performance difference.6 Separately, benchmarking synthesized by Beam.ai found a single agent matched or beat multi-agent systems on roughly two-thirds of tasks when given the same tools and context, with multi-agent adding only a couple of percentage points at roughly double the cost.2 Treat those specific figures as vendor and analyst estimates rather than settled fact, but the direction is consistent. Multi-agent is expensive, and it only wins when the task genuinely splits into independent parallel threads.
When you do need multiple agents, the production default in 2026 is the orchestrator-worker pattern, where one capable lead model decomposes the job, delegates to cheaper specialist workers, and owns the canonical state. Pairing a strong lead with cheaper workers is a standard way to cut cost by a meaningful margin. The patterns that get teams into trouble are the open-ended ones, where agents talk to each other freely. Coordination complexity grows roughly with the square of the agent count, so a 10-agent mesh has around 45 communication paths and becomes nearly impossible to observe or debug.
The discipline here protects your budget. Cap the number of specialists, enforce hard step budgets from the orchestrator rather than trusting agents to self-limit, set a mandatory iteration ceiling, and run an independent verification pass. If a single agent already passes your evaluations, ship the single agent.
Whatever the topology, an agentic workflow that runs unattended in production needs containment built in from the start. The failure modes are well catalogued: context fragmentation across steps, coordination overhead, cost blowup, and the nastier ones, like a hallucination from one step being treated as ground truth by the next, or a retry loop with no termination condition burning tokens until someone pulls the plug.
The controls are straightforward. Enforce hard step budgets in code rather than by asking the model nicely, set explicit termination conditions and a maximum iteration ceiling, require source citations so every claim can be traced, and put human approval gates in front of any action that is hard to reverse or touches money.
Containment and observability are two halves of the same design. A step budget only helps if you can see it being hit, and an approval gate only builds trust if the reviewer can trace how the agent got there. Design the workflow and its instrumentation together, so that when something trips a limit you know exactly which step to fix. We cover the tracing machinery later, in the hardening section, but the decision to make everything visible happens here, at design time.
The academic backbone is worth knowing. A 2025 study introduced a taxonomy of 14 distinct failure modes for multi-agent systems, built from more than 1,600 annotated execution traces.7 The lesson for leaders is that bounded autonomy is the only workable operating model. "The agent will figure it out" is wishful thinking.
The framework market has converged, which is good news. Tool calling is now commoditized, and the real differences are in state management, durability, and observability. You don't need to agonize over this choice. Pick based on the shape of your problem and how much of the system you want the framework to own.
A practical map of the current landscape:
The worst outcome is adopting a framework that fights your architecture, for example forcing a delegation problem into a graph tool, or building a complex stateful workflow on a framework with no persistence. Provider-native SDKs give tighter model integration at the cost of some lock-in. For most enterprises, the right move is to prototype fast, keep your business logic separate from the framework, and avoid betting the system on any one vendor's roadmap.
An agent is only as useful as the tools and data it can reach, and in 2026 the way you connect those has standardized around two complementary layers.
The Model Context Protocol (MCP), introduced by Anthropic and now governed under the Linux Foundation, is the standard for connecting agents to tools and data.8 It has seen rapid adoption across Anthropic, OpenAI, Google, Microsoft, and AWS, with thousands of public servers available, and for most teams it's now table stakes for production tool integration. We wrote about that rise, and what we learned running it in client work, in our year with MCP. The second layer, A2A (Agent2Agent), handles agent-to-agent coordination across vendors and organizations. The practical rule is to start with MCP for tools and add A2A only when you have genuine cross-organization coordination.
The tool layer is also where context engineering becomes concrete. Retrieval-augmented generation (RAG) is the clearest example: a retrieval tool fetches the handful of high-signal passages the model needs instead of the whole knowledge base, which is the same discipline we covered earlier, applied at runtime. Skills work in a similar spirit, packaging instructions and procedures the agent loads on demand rather than carrying them in every prompt. Multi-modality belongs in this conversation too. An agent that handles voice conversations, images, and scanned documents alongside chat needs transcription, vision, and document-parsing tools that a text-only assistant can skip, so decide which modalities matter to your users before you commit to a toolset.
For a leader, the protocol layer carries a real risk you should name out loud. Third-party MCP servers are third-party code with access to your systems, so treat them exactly as you would any external dependency. Review them, pin versions, and audit what they can touch. Documented attack vectors include tool poisoning, where a malicious tool description manipulates the agent, and the field has already seen high-severity disclosures. The security boundary has to be your architecture, never the model itself.
The deeper point for your roadmap is portability. Open standards like MCP, plus a clean separation between your business logic and the model layer, are what keep next quarter's model upgrade from becoming a rewrite.
If there is one practice that separates teams who ship reliable agents from teams who ship anxiety, it's this: They build the evaluation before they build the agent.
Eval-driven development treats the evaluation suite as the working specification. The reasoning is blunt. If you can't express what "correct" looks like as a repeatable test, you're not ready to build the thing. The recipe is a golden set of 50 to 80 on-spec cases built with a domain expert, a grader (often a capable model scoring outputs against a rubric, an approach validated by methods like G-Eval9), and a harness that runs in your CI pipeline so a regression blocks the merge. For retrieval-heavy agents, isolate retrieval quality from generation quality so you can tell which half broke.
Evaluation keeps earning its keep after launch, too. Score a sample of live traffic against the same rubric, watch the trend, and feed real production failures back into the golden set so it stays representative of what users throw at the system.
The decision-maker takeaway is that agent evaluation is your quality contract and your risk control in one artifact. It's what lets you upgrade a model next quarter without praying, and what turns "the agent seems worse this week" from a vibe into a number. Set a realistic baseline, gate every change against it, and budget for this work explicitly. Teams that skip it pay for it later, with interest, in production incidents.
This is the stretch where most of the engineering, and most of the budget discipline, lives. Taking a working prototype to a production system means adding the unglamorous machinery that keeps it cheap, safe, and observable.
Durable execution is now the mainstream answer to reliability. Long-running agents need checkpoint-and-replay so that a crash resumes from the last completed step rather than starting over or corrupting state. Platforms like Temporal, Inngest, Restate, and DBOS handle this, along with built-in checkpointers in frameworks like LangGraph. It's worth saying plainly what durable execution leaves unsolved. Hallucinations, runaway loops, and evaluation drift all need their own controls.
Model routing is the highest-leverage cost lever. Most production traffic never needed a frontier model. Routing easy requests to cheap models and reserving the expensive ones for hard cases can cut model bills by anywhere from 40 to 85 percent, because the price spread between model tiers runs to roughly 100x. Prompt caching and per-session token caps further protect the budget and stop runaway loops.
Layered guardrails wrap the whole thing. Deterministic filters screen the inputs, validators check the outputs, every tool gets a risk rating based on how reversible and how costly its actions are, and the high-risk ones sit behind human approval gates. Prompt-level instructions alone won't survive a determined injection, so the defenses have to be architectural. Run tool execution in isolated, least-privilege sandboxes, and make every action idempotent so a duplicate never charges a customer twice.
Observability and kill switches close the loop. You want structured tracing across every model call, tool invocation, and memory operation, with the parent-child relationships preserved across handoffs, plus a global switch to suspend autonomy instantly if something goes wrong. Pair the traces with a drift-monitoring loop that streams the logs, flags low-confidence cases for human review, and feeds the corrected ones back into your evaluation set so it keeps tracking reality. Roll out through shadow mode (the agent runs in the background, its outputs compared against humans, nothing shipped) before assist mode (the agent drafts, a human confirms) before any autonomy.
The business framing for all of this machinery is one word: risk. Evaluation, guardrails, and observability are how you cap the downside of a probabilistic system before it touches a customer.
One more pattern earns its keep with enterprises carrying legacy systems. Rather than a risky rewrite, wrap the existing system so the agent reads and writes through your current APIs, keeping all the validation and business logic you already trust in place. It's the agent-era expression of the well-known strangler-fig integration pattern.10 You surround the old system instead of replacing it.
Here is the finding that most surprises technical leaders, and the one with the highest leverage: The bottleneck is usually the org chart.
Software architecture mirrors the communication structure of the team that builds it. That's Conway's law, and it's back in force for the AI era. The teams that succeed build a small amount of organizational maturity before they scale: clear ownership, a shared vocabulary between the model engineers and the product engineers, and governance for what an agent is allowed to do on its own. That last piece connects directly to the engineering chapters. Governance only has teeth when the workflows are designed for containment and the observability is in place to enforce it, because you can only govern what you can see.
Two pieces of sequencing advice repay attention. First, build your data infrastructure and hire data engineers before you hire data scientists, because a data scientist without clean, accessible data spends most of their time wrangling instead of modeling. Second, put your model metrics and your business metrics on the same dashboard: accuracy, latency, and hallucination rate next to adoption, retention, and task-time saved. When those live on separate screens, the model team optimizes a number the business doesn't feel, and the business chases an outcome the model team can't see.
Most organizations eventually evolve toward a hub-and-spoke structure, with a central platform team that sets tooling and guardrails and applied teams embedded in the products. You don't need that on day one, but it helps to know it's where you're heading.
Principles are cheap. Here is how this plays out when the system has to work for real clients. (Details are anonymized.)
A global hospitality operator with more than 250,000 employees. This client needed two agents in document-heavy, compliance-sensitive corners of the business: a legal contract review copilot and an HR screening assistant. The temptation was full autonomy. The right answer was trust calibration, bounded autonomy earned in stages. We built the contract copilot in assist mode, where it reads a contract against the company's own policies and flags deviations with a justification trail and an inline pointer to the exact source clause, and a lawyer makes the call. The heavy lifting was context engineering, feeding the agent the right slices of policy and precedent rather than everything, plus guardrails and a clean audit trail. The screening assistant followed the same shadow-then-assist path. The business outcome was experts spending their time on judgment instead of on the first pass, with every recommendation traceable.
A healthcare software startup. The founder had vibe-coded a working prototype, the kind of thing that demos beautifully and breaks the moment a second user shows up. Our job was the unglamorous part, turning it into something a regulated healthcare customer would trust. We hardened it with evaluation, durability, and the production machinery covered earlier, and took it from founder prototype to production in four months, with the first paying client onboarded by month six. The lesson that generalizes is that the distance between "interesting demo" and "system a customer pays for" is mostly engineering discipline, and it's shorter than most teams fear when the right practices are in place from the start.
If you take one thing from this recipe book, let it be the sequence. Decide whether you need an agent at all, since a workflow often wins. Write the evaluation before the agent, invest early in context engineering, and harden with durable execution, model routing, guardrails, and observability before you scale. Go multi-agent only when your evals prove the extra cost pays for itself, and treat the organizational work as part of the build.
The gap between a demo that impresses and a production system that moves your KPIs is smaller than most teams think. The demo is maybe 20 percent of the work. The context engineering, evals, guardrails, and observability behind it are the other 80 percent, and that's where the ROI is won or lost.
That's the part we love. If you're mapping where agents fit on your 2026 roadmap, or you have a prototype that needs to become a product, we'd be glad to compare notes. Schedule a conversation with our team.
QCon London 2026: Team Topologies as the Infrastructure for Agency with AI (InfoQ). The "80 percent see no tangible benefit" figure is cited at the conference and should be read as a directional estimate. ↩
6 Multi-Agent Orchestration Patterns for Production (Beam.ai, 2026 synthesis). The roughly 40 percent pilot-failure rate and the single-agent-vs-multi-agent comparison are analyst and vendor estimates synthesizing benchmarks including Princeton NLP work; verify against primary benchmarks before quoting. ↩ ↩2
Building Effective Agents (Anthropic). ↩
A Practical Guide to Building Agents (OpenAI). ↩
Effective Context Engineering for AI Agents (Anthropic). ↩
How We Built Our Multi-Agent Research System (Anthropic). The 90-percent-plus improvement and roughly 15x token figures are from Anthropic's internal research-task evaluation and may not generalize. ↩
Why Do Multi-Agent LLM Systems Fail? (Cemri et al., arXiv 2503.13657; NeurIPS 2025 spotlight). ↩
Model Context Protocol (Anthropic / Linux Foundation). ↩
G-Eval: NLG Evaluation Using GPT-4 with Better Human Alignment (Liu et al., arXiv 2303.16634; EMNLP 2023). ↩
StranglerFigApplication (Martin Fowler). ↩
CONTACT US
Work with Pento to turn promising AI experiments into systems that perform reliably in production, with the right architecture, delivery model, and engineering support.
