AI Agent Architecture Diagram: Model, Tools, Memory and Guardrails

Most AI agent diagrams show a robot icon next to an LLM logo with an arrow labeled "tools". That picture hides the questions that decide whether an agent is safe to run: who chooses the next step, what the agent is allowed to touch, where untrusted text gets in, and what stops a run that is going nowhere.

This post breaks an agent into its parts, walks through the loop that drives it, covers tools and the Model Context Protocol, memory, guardrails, limits and tracing, and draws a customer support agent that can look up orders and issue refunds.

Workflow or agent?

Anthropic's guide to building effective agents draws a useful line. In a workflow, LLMs and tools are orchestrated through code paths you wrote in advance. In an agent, the model directs its own process and decides which tools to use, step by step, until the task is done.

The same guide recommends the simplest solution that works. If a task always follows the same three steps, write those steps in code and call the model inside them. Reach for an agent when the path depends on what the agent finds along the way: a support case where the next lookup depends on the last answer, or a coding task that needs several rounds of edit and test.

The parts of an agent

The model is only one box. Most of the engineering, and most of the risk control, lives in the code around it.

The loop

Stripped down, the runtime is a loop:

def run(task, user):
    messages = [system_prompt(user), task]
    for step in range(MAX_STEPS):
        reply = llm.generate(messages, tools=allowed_tools(user))
        if not reply.tool_calls:
            return check_output(reply.text)
        for call in reply.tool_calls:
            result = execute(call, user)   # permission check, approval, timeout
            messages.append(tool_result(call, result))
    return stop("step limit reached", messages)

The model never runs anything itself. It returns either an answer or a request to call a tool with certain arguments, and the runtime decides whether and how to run it. That is where control lives: permission checks, approvals, timeouts and limits all sit in execute and in the loop, not in the prompt.

Tools, and where MCP fits

A tool is a function the model can request: a name, a description and a JSON schema for its arguments. The descriptions matter as much as the code, because they are all the model knows about when to use a tool.

The Model Context Protocol is an open standard for connecting AI applications to external systems. An MCP server exposes tools, data and prompts; the agent runtime acts as the client. Instead of wiring every integration into the agent's code, each system ships a server, and any MCP-capable runtime can use it.

Good tools share a few traits:

• Few and specific. refund_order(order_id, amount, reason) beats a generic call_api(url, body).
• Read and write separated, so a role can get read tools without write tools.
• Clear errors the model can act on, such as "amount exceeds order total".
• Idempotency keys on writes, so a retried step does not refund twice.

Memory

Agents have two kinds of memory, and diagrams often blur them:

• Working memory is the context sent to the model on each step: instructions, the conversation, and tool results so far. It is limited by the model's context window and by cost, so long runs summarize older steps or drop raw tool output once it has been used.
• Long-term knowledge lives outside the model: documents, policies, past tickets. The agent retrieves the relevant pieces, often from a vector index, and adds them to working memory. The RAG architecture diagram covers the retrieval pipeline in detail.

Task state that must survive a crash, such as which steps already ran and which approval is pending, belongs in a normal database, not in the model's context.

Guardrails and permissions

The OWASP Top 10 for LLM applications lists prompt injection first. Another entry, excessive agency, covers damaging actions an LLM system can take because it has more functionality, permissions or autonomy than it needs. Agents are exposed to both: they read untrusted text (user input, web pages, retrieved documents, tool results) and they can act.

Rules that hold up in practice:

• Treat every tool result and retrieved document as data. Text inside a web page cannot grant permissions or change the task.
• Act with the user's permissions, not a service account that can see every customer.
• Gate writes. Reads run freely; writes above a threshold, or any irreversible action, wait for a person.
• Sandbox code execution: no credentials and no network by default.
• Check outputs before they reach users or other systems, for leaked internal data or unsafe content.

The threat modeling data flow diagram is a good way to find where untrusted text crosses into a component that can act.

Limits and cost

An agent that loops without progress burns money and patience. Put limits in the runtime, not the prompt:

• Steps: a maximum number of model calls per task.
• Time: a wall-clock limit per task and a timeout per tool call.
• Money: a token or dollar budget per task and per user per day.
• Rate: per-user request limits at the API, as in the rate limiter design.

When a limit is hit, stop cleanly: return what was found so far, say which limit stopped the run, and keep the trace for review.

Tracing and evals

Record every step: the context sent, the model and its settings, each tool call with arguments and results, tokens, latency and cost. OpenTelemetry has semantic conventions for generative AI, so these traces can flow into the same backends as the rest of your system; the observability architecture diagram shows that pipeline.

Traces answer "what happened in this run?". Evals answer "is the agent getting better or worse?": a fixed set of realistic tasks, scored automatically, run on every change to prompts, tools or models.

The architecture diagram

How to read it:

• Left: the user sends a task through the chat app, which handles sign-in and sessions and passes the task, with the user's identity, to the guardrails.
• Center: the agent loop sends context and the tool list to the LLM (solid) and gets back an answer or a tool call (dashed). It reads and writes working memory; the Markdown block lists the limits.
• Right: tool calls go to web search, a code sandbox and the orders API. A refund waits for the approver (dashed).
• Bottom: the knowledge index feeds retrieved documents into working memory, and every step is recorded in traces and evals (dotted).

A worked example: a refund

A customer writes: "Order 4411 arrived broken, I want my money back."

1. Guardrails check the message and attach the customer's identity.
2. The model asks for get_order(4411). The runtime checks that the order belongs to this customer and returns: delivered three days ago, 180 dollars.
3. The runtime retrieves the refund policy from the knowledge index and adds it to working memory: damaged items are refundable within 30 days.
4. The model asks for refund_order(4411, 180, "damaged"). The amount is above the 100 dollar threshold, so the runtime pauses the run and sends an approval request to a support lead.
5. The lead approves. The runtime executes the refund with an idempotency key and returns the confirmation.
6. The model writes the reply, output checks pass, and the customer gets an answer.

Three model calls, two tool calls, one retrieval, one approval, and a trace that shows every step.

What to put on an agent diagram

• Who decides the next step: the model, or code you wrote.
• Every tool, with read or write access and whose permissions it uses.
• Approval points for risky actions.
• Limits: steps, time and budget.
• Where untrusted text enters: user input, web pages, documents, tool results.
• Tracing, so a reviewer knows runs can be inspected.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. Add two Person nodes from the C4 Model pack for the user and the approver, and a Container for the chat app.
3. Add a Frame from the Core pack for the agent runtime, with a Container for the agent loop and two Component nodes for guardrails and working memory.
4. Add a Model and a Vector index from the ML pack, a second Frame with three Service nodes from the Stack pack for the tools, and Traces from the DevOps pack.
5. Draw the model's replies dashed and the tracing edge dotted, and add a Markdown block from the Core pack with the limits. To let an agent draw it for you, see creating diagrams with AI agents over MCP.

The example uses 15 diagram nodes, within the Free plan.

FAQ

What is the difference between an AI agent and a workflow? In a workflow, your code decides the sequence of steps and calls the model inside them. In an agent, the model decides the next step and which tools to use, inside a loop your code controls.

What is MCP in an agent architecture? The Model Context Protocol is an open standard for connecting AI applications to tools and data. Systems expose MCP servers, and the agent runtime connects to them as a client.

How do I stop an agent from taking harmful actions? Give it only the tools it needs, run them with the user's permissions, require human approval for writes above a threshold, and treat all tool results and documents as untrusted data.

How do agents remember things? Within a run, through the context sent to the model on each step. Across runs, through stored knowledge that is retrieved when needed, and through task state kept in a database.

How do I keep agent costs under control? Set limits in the runtime for steps, time and tokens per task, add per-user budgets, and trace every run so expensive patterns show up in reviews.

Start free — one seat, no credit card.