AI Agent Architecture Patterns — From Chatbots to Autonomous Systems

AI agents are the next evolution beyond chatbots. Instead of just generating text, they can reason, use tools, remember context, and take actions autonomously. But building reliable agents is challenging — they need clear architectures that handle uncertainty, failure, and safety.

This post covers the patterns I use to build production AI agents: the tool-calling architecture, memory systems for context retention, multi-agent collaboration, and safety guardrails.

What Makes an Agent

The difference between a chatbot and an agent is agency:

┌─────────────┐         ┌─────────────┐
│  Chatbot    │         │   Agent     │
│             │         │             │
│  Input →    │         │  Input →    │
│  LLM →      │         │  Reason →   │
│  Output     │         │  Tools →    │
│             │         │  Action →   │
└─────────────┘         │  Output     │
                        └─────────────┘

A chatbot just responds. An agent reasons, plans, and acts.

Core Agent Architecture

The foundation of any agent is the loop: observe, reason, act, observe again.

interface AgentMessage {
  role: 'system' | 'user' | 'assistant' | 'tool';
  content: string;
  toolCalls?: ToolCall[];
}

interface ToolCall {
  id: string;
  name: string;
  arguments: Record<string, any>;
}

interface Tool {
  name: string;
  description: string;
  parameters: Record<string, any>;
  execute: (args: any) => Promise<any>;
}

class Agent {
  private tools: Map<string, Tool> = new Map();
  private systemPrompt: string;

  constructor(systemPrompt: string) {
    this.systemPrompt = systemPrompt;
  }

  registerTool(tool: Tool): void {
    this.tools.set(tool.name, tool);
  }

  async run(userInput: string, maxIterations = 10): Promise<string> {
    const messages: AgentMessage[] = [
      { role: 'system', content: this.systemPrompt },
      { role: 'user', content: userInput },
    ];

    for (let iteration = 0; iteration < maxIterations; iteration++) {
      // Get LLM response
      const response = await this.callLLM(messages);
      messages.push(response);

      // Check if agent wants to use tools
      if (response.toolCalls && response.toolCalls.length > 0) {
        // Execute each tool call
        for (const toolCall of response.toolCalls) {
          const tool = this.tools.get(toolCall.name);

          if (!tool) {
            messages.push({
              role: 'tool',
              content: `Error: Tool "${toolCall.name}" not found.`,
              toolCalls: [{ ...toolCall }],
            });
            continue;
          }

          try {
            const result = await tool.execute(toolCall.arguments);
            messages.push({
              role: 'tool',
              content: JSON.stringify(result),
              toolCalls: [{ ...toolCall }],
            });
          } catch (error) {
            messages.push({
              role: 'tool',
              content: `Error: ${(error as Error).message}`,
              toolCalls: [{ ...toolCall }],
            });
          }
        }
      } else {
        // No tool calls, agent is done
        return response.content;
      }
    }

    return messages[messages.length - 1].content;
  }

  private async callLLM(messages: AgentMessage[]): Promise<AgentMessage> {
    // Format tools for OpenAI
    const tools = Array.from(this.tools.values()).map(tool => ({
      type: 'function',
      function: {
        name: tool.name,
        description: tool.description,
        parameters: tool.parameters,
      },
    }));

    const response = await fetch('https://api.openai.com/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
      },
      body: JSON.stringify({
        model: 'gpt-4-turbo',
        messages: messages.map(m => ({
          role: m.role,
          content: m.content,
          tool_calls: m.toolCalls,
        })),
        tools,
      }),
    });

    const data = await response.json();
    const choice = data.choices[0];

    return {
      role: choice.message.role,
      content: choice.message.content || '',
      toolCalls: choice.message.tool_calls?.map((tc: any) => ({
        id: tc.id,
        name: tc.function.name,
        arguments: JSON.parse(tc.function.arguments),
      })),
    };
  }
}

This loop is the heart of any agent. It continues until the agent decides it has completed the task.

Defining Tools

Tools give agents capabilities beyond text generation. Well-defined tools have clear names, descriptions, and parameter schemas.

// Example: Database query tool
const databaseTool: Tool = {
  name: 'query_database',
  description: 'Query the SQL database for customer, product, or order information',
  parameters: {
    type: 'object',
    properties: {
      query: {
        type: 'string',
        description: 'The SQL query to execute (SELECT only)',
      },
    },
    required: ['query'],
  },
  execute: async ({ query }) => {
    // Validate: only SELECT queries
    if (!query.trim().toUpperCase().startsWith('SELECT')) {
      throw new Error('Only SELECT queries are allowed');
    }

    const result = await db.query(query);
    return result.rows;
  },
};

// Example: HTTP request tool
const httpTool: Tool = {
  name: 'http_request',
  description: 'Make an HTTP GET request to retrieve data from an API',
  parameters: {
    type: 'object',
    properties: {
      url: {
        type: 'string',
        description: 'The URL to request',
      },
    },
    required: ['url'],
  },
  execute: async ({ url }) => {
    const response = await fetch(url);

    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    }

    return await response.json();
  },
};

// Example: Calculator tool
const calculatorTool: Tool = {
  name: 'calculator',
  description: 'Perform mathematical calculations',
  parameters: {
    type: 'object',
    properties: {
      expression: {
        type: 'string',
        description: 'Mathematical expression to evaluate (e.g., "2 + 2 * 3")',
      },
    },
    required: ['expression'],
  },
  execute: async ({ expression }) => {
    // Safe evaluation using Function (sanitized input)
    const sanitized = expression.replace(/[^0-9+\-*/().\s]/g, '');
    return Function(`"use strict"; return (${sanitized})`)();
  },
};

Good tools are focused, safe, and return structured data. Avoid tools that do too much — a monolithic tool is harder for the agent to use effectively.

Memory Systems

Agents need memory to maintain context across conversations and reason about past events.

Short-term Memory

interface Memory {
  type: 'observation' | 'action' | 'result' | 'reflection';
  content: string;
  timestamp: Date;
  metadata?: Record<string, any>;
}

class ShortTermMemory {
  private memories: Memory[] = [];
  private maxMemories = 100;

  add(memory: Memory): void {
    this.memories.push(memory);

    // Keep only recent memories
    if (this.memories.length > this.maxMemories) {
      this.memories = this.memories.slice(-this.maxMemories);
    }
  }

  getRecent(count = 10): Memory[] {
    return this.memories.slice(-count);
  }

  search(query: string, limit = 5): Memory[] {
    const queryLower = query.toLowerCase();

    return this.memories
      .filter(m => m.content.toLowerCase().includes(queryLower))
      .sort((a, b) => b.timestamp.getTime() - a.timestamp.getTime())
      .slice(0, limit);
  }

  summarize(): string {
    return this.memories
      .map(m => `[${m.type}] ${m.content}`)
      .join('\n');
  }
}

Long-term Memory with Vector Store

For persistent memory across sessions, use a vector store:

class LongTermMemory {
  private vectorStore: VectorStore;
  private embeddingService: EmbeddingService;

  async storeMemory(
    content: string,
    metadata: { type: string; sessionId: string }
  ): Promise<void> {
    const embedding = await this.embeddingService.generateEmbeddings([content]);

    await this.vectorStore.insert({
      id: `${metadata.sessionId}-${Date.now()}`,
      text: content,
      embedding: embedding[0],
      metadata,
    });
  }

  async retrieveRelevant(query: string, limit = 5): Promise<Memory[]> {
    const queryEmbedding = await this.embeddingService.generateEmbeddings([query]);

    const results = await this.vectorStore.search(queryEmbedding[0], limit);

    return results.map(r => ({
      type: r.metadata.type as any,
      content: r.text,
      timestamp: new Date(r.metadata.timestamp),
      metadata: r.metadata,
    }));
  }
}

Agent with Memory

class AgentWithMemory extends Agent {
  private shortTerm: ShortTermMemory;
  private longTerm: LongTermMemory;
  private sessionId: string;

  constructor(
    systemPrompt: string,
    shortTerm: ShortTermMemory,
    longTerm: LongTermMemory,
    sessionId: string
  ) {
    super(systemPrompt);
    this.shortTerm = shortTerm;
    this.longTerm = longTerm;
    this.sessionId = sessionId;
  }

  async run(userInput: string): Promise<string> {
    // Store user input in short-term memory
    this.shortTerm.add({
      type: 'observation',
      content: userInput,
      timestamp: new Date(),
    });

    // Retrieve relevant long-term memories
    const relevantMemories = await this.longTerm.retrieveRelevant(userInput, 3);

    // Build context with memories
    const memoryContext = relevantMemories
      .map(m => `- ${m.content}`)
      .join('\n');

    const enhancedInput = `Relevant context:\n${memoryContext}\n\nUser: ${userInput}`;

    // Run the agent
    const response = await super.run(enhancedInput);

    // Store response in long-term memory
    await this.longTerm.storeMemory(response, {
      type: 'agent_response',
      sessionId: this.sessionId,
    });

    return response;
  }
}

Multi-Agent Orchestration

Complex tasks benefit from specialized agents working together:

interface AgentTask {
  id: string;
  description: string;
  status: 'pending' | 'in_progress' | 'completed' | 'failed';
  result?: any;
  error?: string;
}

class MultiAgentOrchestrator {
  private agents: Map<string, Agent> = new Map();
  private tasks: AgentTask[] = [];

  registerAgent(name: string, agent: Agent): void {
    this.agents.set(name, agent);
  }

  async executePlan(goal: string): Promise<any> {
    // Step 1: Planning agent breaks down the goal
    const planner = this.agents.get('planner');
    if (!planner) throw new Error('No planner agent');

    const planText = await planner.run(
      `Break down this goal into specific tasks: ${goal}\n\n` +
      `Respond with a JSON array of tasks, each with a "description" and "assigned_to" field.`
    );

    // Parse the plan
    const plan = this.parsePlan(planText);
    this.tasks = plan.map((task, i) => ({
      id: `task-${i}`,
      description: task.description,
      status: 'pending',
    }));

    // Step 2: Execute each task
    const results: any[] = [];

    for (const task of this.tasks) {
      const assignedAgent = this.agents.get(task.assigned_to || 'worker');

      if (!assignedAgent) {
        task.status = 'failed';
        task.error = `No agent found: ${task.assigned_to}`;
        continue;
      }

      task.status = 'in_progress';

      try {
        const result = await assignedAgent.run(task.description);
        task.status = 'completed';
        task.result = result;
        results.push(result);
      } catch (error) {
        task.status = 'failed';
        task.error = (error as Error).message;
      }
    }

    // Step 3: Synthesize results
    const synthesizer = this.agents.get('synthesizer');
    if (!synthesizer) throw new Error('No synthesizer agent');

    return await synthesizer.run(
      `Original goal: ${goal}\n\n` +
      `Task results:\n${JSON.stringify(results, null, 2)}\n\n` +
      `Synthesize these results into a final response.`
    );
  }

  private parsePlan(text: string): Array<{ description: string; assigned_to: string }> {
    // Extract JSON from LLM response
    const jsonMatch = text.match(/\[[\s\S]*\]/);
    if (!jsonMatch) return [];

    try {
      return JSON.parse(jsonMatch[0]);
    } catch {
      return [];
    }
  }

  getTaskStatus(): AgentTask[] {
    return [...this.tasks];
  }
}

// Usage
const orchestrator = new MultiAgentOrchestrator();

// Register specialized agents
orchestrator.registerAgent('planner', new PlannerAgent());
orchestrator.registerAgent('worker', new WorkerAgent());
orchestrator.registerAgent('synthesizer', new SynthesizerAgent());

// Execute a complex goal
const result = await orchestrator.executePlan(
  'Research the latest AI trends and write a blog post summary'
);

Specialized agents are more reliable than one generalist. A planner agent is good at breaking down tasks, a worker agent executes specific actions, and a synthesizer agent combines results into a coherent output.

Safety Guardrails

Autonomous agents need safety constraints:

interface SafetyRule {
  name: string;
  check: (action: ToolCall) => Promise<boolean>;
  errorMessage: string;
}

class SafeAgent extends Agent {
  private safetyRules: SafetyRule[] = [];

  addSafetyRule(rule: SafetyRule): void {
    this.safetyRules.push(rule);
  }

  async run(userInput: string, maxIterations = 10): Promise<string> {
    // Intercept tool calls for safety checks
    const originalCallLLM = this.callLLM.bind(this);

    this.callLLM = async (messages) => {
      const response = await originalCallLLM(messages);

      if (response.toolCalls) {
        for (const toolCall of response.toolCalls) {
          for (const rule of this.safetyRules) {
            const passes = await rule.check(toolCall);

            if (!passes) {
              // Return error message instead of executing
              return {
                role: 'assistant',
                content: `Cannot execute "${toolCall.name}": ${rule.errorMessage}`,
              };
            }
          }
        }
      }

      return response;
    };

    return super.run(userInput, maxIterations);
  }
}

// Example safety rules
const safeAgent = new SafeAgent(systemPrompt);

// Block database modifications
safeAgent.addSafetyRule({
  name: 'no-db-writes',
  check: async (toolCall) => {
    if (toolCall.name === 'query_database') {
      const query = toolCall.arguments.query;
      return !query.match(/(INSERT|UPDATE|DELETE|DROP|TRUNCATE)/i);
    }
    return true;
  },
  errorMessage: 'Database modifications are not allowed',
});

// Block external requests to unknown domains
safeAgent.addSafetyRule({
  name: 'allowed-domains-only',
  check: async (toolCall) => {
    if (toolCall.name === 'http_request') {
      const url = new URL(toolCall.arguments.url);
      return ['api.example.com', 'cdn.example.com'].includes(url.hostname);
    }
    return true;
  },
  errorMessage: 'Only requests to allowed domains are permitted',
});

Production Lessons

Building agents for production has taught me:

1. Start simple. A single agent with good tools beats a complex multi-agent system with poor tools.

2. Tool quality matters. The best agents use focused, reliable tools. A bad tool causes the agent to fail repeatedly.

3. Monitor everything. Track tool success rates, iteration counts, and failure modes. This data is invaluable for improvement.

4. Have human oversight. For high-stakes decisions, require human approval before the agent takes action.

5. Test extensively. Agents have many failure modes. Build test suites that cover edge cases, tool failures, and unexpected inputs.

6. Rate limiting is essential. Agents can get stuck in loops calling the same tool repeatedly. Implement per-tool rate limits.

Conclusion

AI agents combine LLM reasoning with tool use to create autonomous systems. The core loop — observe, reason, act — is simple, but building reliable agents requires careful architecture: well-defined tools, memory systems, safety guardrails, and sometimes multi-agent collaboration.

Start with a single agent and a few tools. Monitor how it performs. Add complexity only when you have specific problems to solve. The patterns in this post provide a foundation for building agents that are both capable and safe.