Skip to main content

AI Agents and Tool-Using Systems Guide

AI agents are Large Language Models augmented with the ability to observe their environment, plan actions, and use external tools to execute those actions. Unlike single-prompt completions, agents loop: they perceive state, decide on an action, execute it via a tool, observe the result, and repeat until they reach a goal. This guide covers the ReAct pattern, tool definition standards, planning vs execution tradeoffs, and production-ready failure handling.

Key Takeaways​

  • The agent loop: Observe state → reason about action → execute tool → observe result → repeat until goal reached
  • ReAct (Reasoning + Acting) pattern: Interleave reasoning steps with tool invocations; output a "Thought," then an "Action," then observe the result; 40-70% accuracy improvement over single-shot prompts
  • Tool definitions matter: Specify input schema (what data the agent must provide), constraints (what the agent cannot do), and error handling (what happens if a tool fails)
  • Plan vs. Execute tradeoff: Explicit planning (ask agent to plan first, then execute) reduces errors but adds latency; dynamic planning (decide next tool mid-execution) is faster but may loop excessively
  • Five failure modes: Hallucinated tool names, infinite loops, format breakage, authorization bypass, and wrong-tool selection
  • Production patterns: Use tool call counts to detect infinite loops; log all tool invocations for audit; version tool schemas; implement tool authorization checks

What is an AI Agent?​

Agents vs. Prompts​

A traditional prompt is stateless: you provide text, the model produces text once, and you are done. An agent is stateful and looping:

  1. Observe: Agent reads the current state (user query, previous results, available tools)
  2. Reason: Agent generates a thought process about what to do next
  3. Act: Agent calls a tool (API, database, calculator, search engine)
  4. Loop: Agent observes the tool result and repeats steps 2-3 until the goal is reached or a termination condition is met

Example: A user asks, "What is the weather in London and what flight has the lowest price from New York?"

A single-prompt model might hallucinate: "The weather is clear and United Flight 123 costs $299." It does not actually check anything.

An agent would:

  1. Call get_weather("London") → observes "Partly cloudy, 15°C"
  2. Call search_flights("NYC", "London") → observes a list of flights with prices
  3. Calls sort_by_price(results) → observes the cheapest option
  4. Returns a grounded answer: "Weather is partly cloudy. The cheapest flight is [actual flight details]."

Why Agents Matter in 2025​

Three trends make agents essential now:

1. Real-time requirements: Single-prompt models cannot access live data. Agents can call real-time APIs (weather, flight prices, stock quotes).

2. Complex tasks: Tasks requiring 5+ steps (research, validation, correction) fail with single prompts. Agents can loop, plan, and retry.

3. Audit and compliance: Regulated industries need to prove the agent's reasoning and tool use. Agents produce step-by-step logs that are inspectable.

The ReAct Pattern: Reasoning + Acting​

Core Concept​

ReAct (Reasoning + Acting) interleaves explicit reasoning with tool invocations. Instead of asking the agent to call a tool immediately, you first ask it to think about whether a tool is needed, what the tool should do, and how to interpret the result.

Format:

Thought: [Agent's reasoning about the next step]
Action: [Name of the tool to call]
Action Input: [Parameters for the tool, in JSON]
Observation: [Result of calling the tool]
Thought: [Agent's interpretation of the observation]
Action: [Next tool, or "Final Answer"]
...
Final Answer: [Grounded conclusion]
User: "Find me the cheapest flight from NYC to London for tomorrow."

Thought: The user wants to find a flight. I need to search for flights from New York to London tomorrow. First, I should determine today's date to find tomorrow's date.

Action: get_current_date
Action Input: {}
Observation: 2026-06-02

Thought: Today is 2026-06-02, so tomorrow is 2026-06-03. Now I'll search for flights.

Action: search_flights
Action Input: {"origin": "NYC", "destination": "London", "date": "2026-06-03"}
Observation: [
  {"airline": "BA", "flight": "BA112", "price": "$650", "departure": "08:00"},
  {"airline": "United", "flight": "UA156", "price": "$480", "departure": "10:30"},
  {"airline": "Virgin", "flight": "VS204", "price": "$520", "departure": "14:00"}
]

Thought: I have three flights. The cheapest is United UA156 at $480. I should return this to the user.

Final Answer: The cheapest flight from NYC to London tomorrow is United UA156 at $480, departing 10:30.

Why ReAct Works​

Research (Yao et al. 2022) shows ReAct improves accuracy by 40-70% on complex reasoning tasks because:

  1. Explicitness: The agent must articulate its reasoning before acting, reducing impulsive errors
  2. Feedback: The agent observes tool results and can adapt its reasoning in the next step
  3. Transparency: Humans can audit each step and correct the agent's reasoning if needed

Designing Effective Tool Definitions​

Tool Schema Specification​

Every tool your agent can call must have a clear schema. Use OpenAI-compatible format:

{
  "name": "search_flights",
  "description": "Search for flights between two cities on a given date. Returns a list of flights with price, airline, departure time, and duration.",
  "parameters": {
    "type": "object",
    "properties": {
      "origin": {
        "type": "string",
        "description": "Three-letter airport code (e.g., 'NYC', 'LAX'). Must be a valid IATA code."
      },
      "destination": {
        "type": "string",
        "description": "Three-letter airport code (e.g., 'LHR', 'CDG')."
      },
      "date": {
        "type": "string",
        "description": "Date in YYYY-MM-DD format. Must be a future date."
      },
      "max_price": {
        "type": "number",
        "description": "Optional. Maximum price in USD. If set, only return flights under this price."
      }
    },
    "required": ["origin", "destination", "date"]
  }
}

Tool Constraints and Authorization​

Specify what the tool can and cannot do:

CONSTRAINTS:
- This tool returns flights for tomorrow through 365 days in the future only.
- It cannot search for past dates.
- Results are limited to commercial airlines.
- Each call returns max 50 flights. Call with filters (max_price, airline) to narrow results.

AUTHORIZATION:
- The agent can call this tool only if the user has a valid booking account.
- The agent must NOT call this tool to scrape competitor pricing data.
- Log all calls with user ID and date for compliance.

Error Handling in Tool Definitions​

Define what happens when a tool fails:

{
  "error_responses": {
    "invalid_date": "Return error message: 'Date must be in YYYY-MM-DD format and be a future date.'",
    "invalid_airport_code": "Return error message: 'Airport code must be a valid IATA code (e.g., NYC, LHR).'",
    "no_flights_found": "Return: { flights: [], message: 'No flights found for this route on this date.' }",
    "service_unavailable": "Return error message: 'Flight search service is temporarily unavailable. Please try again in a few minutes.' Agent should stop and inform the user."
  }
}

Planning vs. Dynamic Execution​

Plan-First Approach (Explicit Planning)​

Before executing any tools, ask the agent to output a plan:

Prompt:

User query: [User's request]

Step 1: Think about the steps you will take to answer this query.
Output a numbered plan (3-5 steps).

Step 2: Execute the plan by calling tools in order.
Report the result of each step.

Example:

User: "Find the cheapest flight from NYC to London and book it."

Plan:
1. Search for flights from NYC to London tomorrow
2. Filter by price and select the cheapest option
3. Check passenger details from the user's profile
4. Book the selected flight
5. Return confirmation number

Execution:
[Agent calls tools in order according to the plan]

Advantages: Reduces wasted tool calls, prevents infinite loops, aligns with user intent upfront.

Disadvantages: Adds latency (two rounds of LLM inference), plans may miss edge cases, plans may become invalid mid-execution.

Dynamic Execution (Mid-Loop Decision)​

Omit the planning phase. Ask the agent to decide the next action after each tool result:

Prompt:

User query: [User's request]

Think step by step. At each step, decide whether you need to call a tool.
If yes, call it. If no, return the final answer.

Advantages: Faster (fewer LLM calls), agent can adapt to unexpected results, works for unpredictable tasks.

Disadvantages: May loop excessively (call the same tool repeatedly), harder to audit, can exceed context limits.

When to Use Which​

  • Plan-First: Long workflows (5+ steps), high-stakes decisions (booking, financial), regulated tasks (compliance logging)
  • Dynamic: Exploratory tasks (research), time-sensitive (real-time search), simple tasks (2-3 steps)

Production Failure Modes and How to Detect Them​

Failure Mode 1: Hallucinated Tool Names​

Problem: Agent calls a tool that doesn't exist.

Thought: I should look up the user's booking history.
Action: get_booking_history
Action Input: {"user_id": "12345"}
ERROR: Tool "get_booking_history" not found.

Detection: Log all tool calls and check against your allowed tools list. Flag calls to unknown tools immediately.

Prevention: In the system prompt, list only the allowed tools by name. Include a line: "You can ONLY use these tools: [list]. Do not invent tool names."

Failure Mode 2: Infinite Loops​

Problem: Agent calls the same tool repeatedly without making progress.

Thought: I should search for flights again to make sure I didn't miss any.
Action: search_flights
[Same call as before]
Observation: [Same result as before]
Thought: Let me search again...

Detection: Track tool call count per session. Alert if any tool is called more than 5 times, or if total tool calls exceed 20 without a "Final Answer."

Prevention: Add constraints to the prompt: "You can call each tool at most 3 times. If a tool returns the same result twice, do not call it again."

Failure Mode 3: Format Breakage​

Problem: Agent's output is not valid JSON, markdown, or expected schema.

Final Answer: The flight is United UA156 at price $480. It departs at 10:30 and arrives at 22:15. It has [MISSING_FIELD].

Detection: Parse agent output against a schema. If parsing fails, treat as an error.

Prevention: Include an output schema in the prompt and an example of a correctly formatted response. Use a schema validation library to reject malformed outputs and re-prompt the agent.

Failure Mode 4: Authorization Bypass​

Problem: Agent calls a tool without proper permission checks.

Thought: I'll check what flights the competitor is offering.
Action: search_flights
Action Input: {"origin": "NYC", "destination": "London", "api_key": "competitor_key"}

Detection: Log all tool calls with user context. Flag calls that don't have proper authorization.

Prevention: Implement tool-level authorization checks. The agent should not have API keys or credentials. Instead, the agent runtime checks permissions before executing any tool call.

Failure Mode 5: Wrong Tool Selection​

Problem: Agent picks a tool that doesn't solve the problem.

User: "What is the capital of France?"
Thought: I need to search for this information.
Action: search_flights  [WRONG TOOL]

Detection: Check tool-to-task alignment. Log cases where a tool is called but produces irrelevant results.

Prevention: Improve tool descriptions. Instead of "Search for flights", write "Search for commercial airline flights between two cities on a specific date. Use this tool only for flight booking or travel research." In the prompt, include examples of correct and incorrect tool usage.

Operational Checklist for Production Agents​

Step 1: Define Tool Suite​

  • List all tools available to the agent
  • Write clear, detailed descriptions (50-100 words each)
  • Specify input schema and constraints
  • Define error handling for each tool
  • Test each tool's behavior with edge case inputs

Step 2: Version Tool Schemas​

  • Use semantic versioning: v1.0.0 for initial release
  • Log which tool schema version each agent invocation used
  • Before upgrading a tool schema, test the agent's behavior with the new schema
  • Keep older versions available for rollback

Step 3: Instrument Logging​

Log the following for every agent step:

  • Step number and timestamp
  • Agent's thought process
  • Tool called and input parameters
  • Tool result (success or error)
  • Agent's interpretation of the result

Step 4: Implement Guardrails​

  • Set max tool calls per session (e.g., 20)
  • Set max calls per tool (e.g., 3)
  • Implement timeout per tool call (e.g., 30 seconds)
  • Require authorization checks before tool execution

Step 5: Canary Deployment​

  • Deploy agent to a small user group (5-10%)
  • Monitor: tool call patterns, error rates, infinite loop rate, user satisfaction
  • If metrics are healthy, expand to 25%, then 100%

Frequently Asked Questions​

What is the difference between an AI agent and a chatbot?​

A chatbot responds to a single user message with a single response. An agent loops: it may call multiple tools, observe results, and repeat until a goal is reached. Agents are stateful across multiple steps; chatbots are typically stateless (though they may have conversation history).

How many tools can an agent effectively manage?​

Research shows agents degrade when given more than 50 tools. For best performance, keep to 5-15 tools that are highly relevant to the agent's domain. Group related tools (e.g., all payment tools) and use a tool-selection step if you have many tools.

What should I do if the agent gets stuck in a loop?​

Implement a max-call guard and track tool call patterns. If you detect the agent calling the same tool twice with the same inputs, stop it and ask the user to clarify. Log these cases for analysis. In the prompt, add: "If a tool returns the same result twice, do not call it again. Try a different approach or return a Final Answer."

Can I use agents with smaller models like Phi-3 or Gemma 2?​

Yes, but expect to provide more explicit examples and tighter tool definitions. Smaller models are less good at reasoning, so be more prescriptive about when and how to use each tool. Test extensively before production.

How do I audit agent decisions in production?​

Log every step: thought, action, result, interpretation. Make these logs queryable by session ID, user, timestamp, and tool. When a user disputes a decision, you can replay the agent's reasoning step by step.

Further Reading​