How To Create An AI Agent

A large-scale benchmark built with input from more than 250 industry experts across over 100 institutions found that mainstream AI agents achieved only a 2.6% full pass rate on the hardest tier of real-world professional tasks, while the best-performing agent reached roughly 26% (RapidClaw benchmark analysis). That gap changes the question. Learning how to create an AI agent isn't mainly about writing a clever system prompt. It's about designing a controlled software system that can select tools, manage state, recover from errors, and prove that it works.

The business pressure is real. A 2025 McKinsey global survey found that 23% of organizations were already scaling an agentic AI system in at least one business function, while 39% had begun experimenting with AI agents. Yet, within any single business function, no more than 10% of respondents said their organizations were scaling agents. Adoption is moving beyond prototypes, but production maturity remains limited.

The practical path is failure-first. Define the job before choosing the model, constrain every tool call, build evaluations before inviting users, and deploy with cost limits and a kill switch. That approach produces a less theatrical demo, but a far more dependable agent.

Why Building an AI Agent in 2026 Is an Engineering Problem, Not a Prompt Trick

An AI agent combines a model with instructions, tools, data access, state, and an execution loop. The model may decide what to do next, but the surrounding system determines what it's allowed to do, how errors are handled, and when execution must stop.

That distinction matters because a prompt can describe an intention, but it can't enforce a database transaction, guarantee a valid API argument, or prevent a planner from repeating the same failed action. Those controls belong in code and infrastructure.

The adoption gap changes the build standard

The McKinsey findings show a market in transition. Organizations are scaling agents somewhere, but broad operational maturity hasn't arrived. A production team therefore needs to treat every deployment as a reliability problem, not as proof that a model can produce convincing text.

A 2025 PwC survey of 300 senior executives found that 79% said AI agents were already being adopted in their companies, 88% planned to increase AI-related budgets in the next 12 months because of agentic AI, and 66% of adopters reported measurable productivity gains (PwC AI agent survey). The same source also describes a separate industry survey in which 95% of respondents had started an agentic AI journey, but only 14% had advanced beyond early validation to expansion or enterprise-scale deployment. Interest is widespread. Reliable execution is much harder.

Treat the agent as a distributed system

A useful mental model includes:

• A planner, which selects the next action.
• Typed tools, which expose only approved operations.
• State management, which records what happened and what remains.
• Policies, which block unsafe or unauthorized actions.
• Evaluations, which measure behavior before and after changes.
• Observability, which lets engineers reconstruct a failed run.
• Budgets, which limit latency, tool calls, and model spend.

The most common production mistakes are predictable. Teams leave tool inputs loosely specified, skip adversarial tests, allow indefinite retries, and assume a model update won't alter behavior. The result is an agent that appears capable in a short demonstration but becomes unreliable when a user supplies incomplete information, an API returns partial data, or a workflow takes an unexpected branch.

Practical rule: If you can't state what the agent may do, what counts as success, and when it must stop, you haven't finished the design.

Define the Job, the Metric, and the Boundaries Before You Touch a Model

Start with a job that a human can describe in one sentence. “Help sales” is too broad. “Review selected LinkedIn activity, identify posts that match our ideal customer profile, and produce a ranked list for human follow-up” is testable.

A narrowly defined job also gives you a better way to choose autonomy. An agent that summarizes evidence can operate with fewer permissions than one that updates a CRM, sends outreach, or changes account records.

Write the success contract

Use a short design document with four parts:

1. Trigger: What starts a run? It might be a new post, an uploaded document, an inbound support request, or a scheduled batch.
2. Input: Which fields are required, optional, or rejected?
3. Outcome: What artifact must the agent produce?
4. Metric: How will a reviewer decide whether the output is useful?

For a buyer-signal monitoring agent, the outcome could be a ranked record containing the author, company context, matching topic, evidence from the activity, and a suggested follow-up angle. The metric might focus on qualified-lead conversion, false-positive rate, reviewer acceptance, or the completeness of the evidence. Pick one primary metric first. Secondary metrics can reveal trade-offs, but too many primary goals make the agent impossible to tune.

Don't let the model define “qualified.” Write the rule explicitly. For example, a signal may need to match a selected topic, come from a person or company that fits the ICP, and include enough source context for a human to verify the classification.

Turn permissions into a boundary document

List every system the agent can access and the exact operation permitted on each one. A first version might allow read-only access to selected social activity and enrichment data, but prohibit CRM writes and outbound messages.

The boundary document should answer:

• Which accounts, records, and fields can the agent read?
• Which tools can it call?
• Which actions are permanently forbidden?
• Which actions require confirmation?
• What data must never enter the model context?
• What happens when evidence is missing or contradictory?

For the monitoring example, the agent may retrieve a post, inspect approved engagement metadata, compare the content against configured topics, and create a draft signal. It shouldn't invent a company match, infer sensitive personal information, or send a message without approval.

A permission isn't a convenience. It's a production liability that needs a reason.

Keep the document versioned. Every new permission should trace to a specific job requirement and a test case. This discipline shortens the design loop because it prevents teams from adding tools merely because a model asked for them.

Pick an Architecture Pattern That Fits the Workload

Architecture should follow the shape of the work. A fixed classification task doesn't need an autonomous planner, while a branching workflow shouldn't be forced into a chain of prompts that hides state and recovery.

Pattern	Best Workload	Latency Cost	Failure Recovery
Router	Narrow intent classification with a fixed tool list	Usually lower, because it selects among known paths	Route to a fallback or human when confidence is insufficient
Planner-executor	Multi-step workflows with recoverable failures	Higher, because planning and execution add model and tool calls	Replan from recorded state, with bounded retries
Graph	Workflows where state, branching, and approvals matter more than autonomy	Depends on graph depth, but explicit transitions add control overhead	Move to a known node, pause for approval, or terminate safely

Use a router for predictable decisions

A router receives an input, classifies it, and sends it to a specialized path. This works well for support triage, document type detection, or selecting a retrieval source. The tool set stays fixed, which simplifies testing and makes latency easier to reason about.

The router shouldn't answer the request itself if downstream logic needs structured inputs. Return a validated intent, required fields, and a confidence or escalation state. If a request doesn't fit an approved route, send it to a human or a safe fallback.

Use planner-executor when steps can change

Planner-executor designs suit workflows such as researching a company, gathering evidence, checking records, and drafting an output. The planner proposes steps, while the executor calls tools and reports structured results.

This pattern gives the agent flexibility, but it also creates more failure surfaces. Store each step, result, error, and retry count. The planner must receive tool failures as explicit states, not as vague text that encourages it to guess.

Use graphs when control matters most

A graph makes transitions explicit. Nodes can represent retrieval, validation, enrichment, approval, writing, and completion. Edges can require conditions such as “evidence exists,” “policy check passed,” or “human approved.”

Add retrieval when the agent needs external or changing knowledge. Use a short-term scratchpad for intermediate results that matter during one run. Add long-term vector memory only when repeated access to durable user or organizational context measurably improves the task. Each layer increases storage, retrieval, debugging, and privacy obligations, so memory shouldn't be an automatic feature.

Wire Tools, Schemas, and Guardrails the Agent Will Actually Respect

Tools are interfaces, not suggestions. Expose each operation through a typed schema with required fields, allowed values, explicit errors, and an idempotency strategy. The model should request a function by name. A dispatcher should validate the request and decide whether the underlying code may run.

Design contracts that fail clearly

An HTTP fetch tool might accept a URL from an approved domain list, a method from an allowed set, query parameters, a page token, and a request identifier. Its response should distinguish success, authentication failure, rate limiting, timeout, invalid input, and partial data.

A database write tool needs stricter controls. Require the table or service operation to be selected from an allowlist, validate field types, reject unknown columns, and include an idempotency key. Return a preview of the proposed change before any irreversible write.

Pagination deserves explicit treatment. The tool should return whether more records exist and provide a continuation token. The planner must have a maximum page budget. If a page fails after earlier pages succeeded, return the partial result with a clear failure state rather than presenting incomplete data as complete.

Layer the defenses

Layer	What It Catches	Placement
Input validator	Missing fields, invalid types, disallowed values	Before dispatch
Permission policy	Unauthorized records, tools, or actions	Before authorization and execution
Rate limiter	Excessive calls and abusive loops	At the dispatcher and service boundary
Output filter	Sensitive data, malformed results, unsafe content	After tool response and before model context
Human approval	Risky changes involving money, PII, or production data	Immediately before execution
Circuit breaker	Repeated failures, loops, and unavailable dependencies	Around the execution loop

A policy classifier can block risky calls, but it shouldn't replace deterministic rules. For actions touching money, PII, or production data, show a human the exact diff, the source evidence, and the intended destination. Approval should apply to the specific action, not to the entire session.

The model can propose. The dispatcher decides.

Log every invocation with the tool version, sanitized arguments, result status, latency, and correlation identifier. Pin prompts to known schema versions so a model can't unexpectedly target an interface that changed underneath it.

Timeouts and retries need budgets. A timeout should return a typed failure to the planner. A retry should require a reason, use bounded backoff, and stop after the permitted budget. If the planner sees the same failure repeatedly, route to escalation or terminate. Never allow “try again” to be the only recovery strategy.

Build an Evaluation Harness Before the First Real User

An agent without an evaluation harness is a guess with a user interface. Raw model capability cannot show whether your tool contracts, policies, and recovery paths work. Build tests around the failures that would matter in production before inviting real users.

Build three test tracks

Golden trajectories represent normal user sessions with an expected path or an acceptable set of paths. Store the input, relevant context, tools the agent should select, valid arguments, expected intermediate states, and an acceptable final result. Include variations that test whether the agent reaches the same outcome through a different valid sequence.

Adversarial cases target predictable failure modes. Include missing fields, contradictory records, prompt injection in retrieved content, invalid tool arguments, repeated tool errors, pagination traps, and requests beyond the caller's permissions. Add cases where the correct response is to stop, ask for clarification, or request approval.

Reliability baselines define minimum behavior for core capabilities. A run may need to select the correct tool, pass schema validation, preserve evidence, stop after a bounded number of attempts, and escalate when authorization or confidence is insufficient. These checks turn vague reliability goals into release criteria.

Score the trace, not only the answer

A polished final response can conceal a dangerous execution path. Score:

• Tool selection and call order.
• Argument correctness and schema compliance.
• Evidence grounding.
• Plan length and unnecessary actions.
• Recovery after failures.
• Policy compliance.
• Final answer quality.

Use deterministic assertions wherever possible. Check exact IDs, required fields, allowed tools, approval states, and database outcomes in code. Reserve an LLM judge for fuzzy qualities such as clarity or usefulness, then calibrate it against human-reviewed examples. A judge should supplement execution checks, not replace them.

Run the suite in continuous integration whenever prompts, model versions, schemas, retrieval settings, or policies change. Keep a held-out set that contributors cannot use while tuning. Without that separation, the team can optimize for familiar examples and miss regressions in tool choice, authorization, or recovery.

The agent engineering guide recommends testing before launch, with logging, tracing, and explicit cost budgets in deployment. Apply that sequence to every meaningful change. It prevents a common production failure: discovering too late that nobody can explain why the agent chose a tool or whether a release improved the system.

Deploy with Logging, Cost Budgets, and Production Failure Modes in Mind

A production agent needs an execution envelope. The envelope defines how long a run may continue, how many model and tool calls it may make, how much context it may consume, and what happens when a dependency fails.

A simple containerized worker with a queue is often enough for an initial deployment. It gives the team control over retries, concurrency, secrets, and versioning. A managed agent runtime can reduce infrastructure work, but it may obscure execution details or impose constraints on model, state, and observability. Choose the simplest environment that exposes the controls your risk profile requires.

Capture the complete trace

A useful trace includes:

• The request and relevant context, with sensitive data redacted.
• The model and prompt version.
• The plan or state transition.
• Every tool name, validated argument, status, and latency.
• Token usage and cache behavior.
• Retry counts and failure reasons.
• The final user-visible response.
• Approval decisions and policy outcomes.

Avoid logging secrets or unrestricted personal data. Store enough information to reproduce a failure without creating a second security problem.

Put budgets around every run

Set limits at several levels. A request budget can cap total model calls, tool calls, elapsed time, and context growth. A user or workspace budget can prevent one automation from consuming shared capacity. A task budget can stop expensive research from expanding indefinitely.

Cache deterministic sub-tasks where the input and policy context are stable. Don't cache responses that depend on fast-changing authorization, private records, or current state unless the cache key includes the relevant version and expiry information.

Predict failure before it reaches users

Recent reporting on multi-step agent performance described success rates of only about 30% to 35% on certain multi-step tasks, while a post-mortem audit identified tool-call hallucination at 22%, context-window overflow at 18%, infinite loops at 15%, and missing guardrails or over-privileged scope at 14% (The Register's report on agent failures). These figures are specific to the reported evidence and shouldn't be treated as a universal failure profile, but they point to practical controls.

Failure Mode	Early Signal	Rollback Action
Repeated tool loop	Same tool and arguments recur without progress	Open the circuit, stop new runs, and route requests to a fallback
Context overflow	Rapid context growth or truncation warnings	Disable the affected workflow and reduce context sources
Schema drift	Validation failures after a model or vendor change	Pin the previous schema and model version
Dependency outage	Tool timeouts or elevated error states	Disable dependent actions and return a transparent escalation
Policy violations	Blocked calls or unsafe output samples increase	Suspend autonomous execution and require review

For operational visibility into buyer-signal workflows, SignalSpot is one example of an autonomous agent that monitors LinkedIn activity against configured ICPs, topics, competitors, and keywords, then surfaces contextual lead signals. Regardless of the product, the engineering requirement is the same: expose traces, enforce budgets, and give on-call staff a safe disable path.

The runbook should include a feature flag or service-level switch that stops new agent runs without a redeploy. Preserve in-flight state, notify the owning team, switch to a manual process, and record the trigger. A kill switch that only exists in application code isn't enough if the application itself is the component failing.

Roll Out, Monitor, and Handle the Legal and Safety Questions

Autonomy should expand only after observed behavior supports it. Start in shadow mode, where the agent produces outputs for internal comparison but doesn't affect users or systems of record. Review the traces, inspect false positives, and confirm that sensitive data handling matches the approved design.

Then expose the workflow gradually behind a kill switch. The rollout plan in the product brief uses shadow mode for days 1 to 10, gradual exposure for days 11 to 20, and full traffic for days 21 to 30. It also calls for 5% to 10% traffic during gradual exposure and requires reliability and cost thresholds to hold for seven consecutive days before full exposure. Those are rollout parameters for this plan, not universal guarantees.

Monitor behavior that changes decisions

Build dashboards for:

• Tool-call success and validation failure.
• Sampled hallucination and unsupported-claim rates.
• P95 latency.
• Cost per resolved task.
• Escalation frequency.
• Approval rejection and policy-block rates.
• Differences between agent output and human decisions in shadow mode.

A threshold should trigger an action, not merely create a chart. Define who receives the alert, which feature flag they change, and how the team restores service after the incident.

Complete the legal review

The legal checklist depends on the use case and jurisdiction, but it should cover GDPR, CCPA, and applicable EU AI Act risk classification. Document the lawful basis for processing, data retention windows, deletion handling, access controls, audit-log completeness, vendor processing terms, and data residency requirements.

Copyright exposure also deserves review. Retrieval and generation can reproduce material that the organization doesn't own, while user-provided content may have usage restrictions. Define permitted sources, attribution rules, output review requirements, and retention limits before launch.

For privacy expectations, publish a clear policy and connect the agent's data handling to it through SignalSpot's privacy policy. The policy shouldn't be a substitute for internal controls. Engineers still need to redact logs, restrict context, and verify deletion behavior.

Short FAQ

Who is liable when an agent makes a harmful decision?
The organization deploying the system usually remains responsible for its operational controls and decisions. Assign an accountable owner, document approval requirements, and keep an audit trail.

Does data residency matter?
Yes, especially when personal, confidential, or regulated data crosses borders. Confirm where prompts, tool data, logs, backups, and support artifacts are processed and stored.

When is human review mandatory?
Require it when an action affects money, employment, access, legal rights, sensitive personal data, production systems, or an irreversible customer record. If the team can't explain the impact in a clear diff, the agent shouldn't execute autonomously.

---

SignalSpot helps teams monitor LinkedIn posts, comments, likes, shares, and follows for configured buyer signals, then prioritize activity with ICP-fit and intent context. If you're building an agent for signal-led prospecting, visit SignalSpot to see how continuous monitoring and CRM-connected lead context can fit into a controlled workflow.