Agent State Management: Tracking Tool Use and Memory
Agent state management ensures your LLM-powered tools and workflows behave consistently as they execute sequences of actions. Unlike simple chatbots that reply to a single question, agents make decisions, call functions, and accumulate state—and every step must be predictable and auditable.
Key Takeaways
- Define agent state explicitly: Track conversation history (what was said), tool calls (what was executed), results (what came back), and decision context (why actions were taken).
- Maintain a decision log: Record every tool call with input, output, and reasoning. This log is your source of truth for debugging when agents misbehave.
- Freeze decision logic: Separate policy rules (guardrails, tool allowlists) from execution (which tool to call next) from memory (conversation history) using clear delimiters.
- Budget and summarize: As conversations grow, summarize old turns to stay within context limits. Never let state explode.
- Test state transitions: Verify that your agent handles the full lifecycle—happy path, error recovery, rollback, and cleanup. State management failures are the #1 production issue in agent systems.
Why State Management Matters in Agents
Traditional chatbots are stateless: each message is independent. Agents are stateful: they maintain a model of progress, available tools, user permissions, and past decisions. When state drifts—a tool call isn't recorded, conversation history is corrupted, or a decision gets lost—the agent can make inconsistent choices or repeat work.
Consider an agent that books flights: it might call a tool to search flights (state: search_in_progress), then call a tool to check prices (state: pricing_fetched), then call a tool to book (state: booking_requested), then confirm with the user (state: awaiting_confirmation). If any transition fails silently, the user might book twice, or the agent might skip payment.
Explicit state management prevents these failures. The cost is moderate bookkeeping. The return is predictability and debuggability.
The Four Layers of Agent State
Layer 1: Conversation History
The transcript of all turns (user messages + agent responses). This is the context the model sees.
{
"conversation_id": "agent_123_2026_06_02",
"turns": [
{
"turn": 1,
"role": "user",
"message": "Book me a flight from NYC to SF next Tuesday for 2 people.",
"timestamp": "2026-06-02T10:00:00Z"
},
{
"turn": 2,
"role": "agent",
"message": "I'll search for flights leaving NYC next Tuesday. Let me check available options.",
"timestamp": "2026-06-02T10:00:05Z"
},
{
"turn": 3,
"role": "system",
"action": "tool_call",
"tool": "search_flights",
"params": {"origin": "NYC", "destination": "SF", "date": "2026-06-09"},
"timestamp": "2026-06-02T10:00:06Z"
}
]
}
Keep history in a database, not in the prompt context (to save tokens). Embed only the relevant recent turns in the prompt.
Layer 2: Tool Call Registry
Every tool invocation, its result, and any errors. This is your audit log.
{
"tool_calls": [
{
"call_id": "tc_001",
"turn": 3,
"tool_name": "search_flights",
"input": {
"origin": "NYC",
"destination": "SF",
"date": "2026-06-09",
"num_passengers": 2
},
"output": [
{"flight_id": "AA123", "departure": "08:00", "arrival": "11:00", "price": 250},
{"flight_id": "UA456", "departure": "10:30", "arrival": "13:30", "price": 320}
],
"status": "success",
"latency_ms": 850,
"timestamp": "2026-06-02T10:00:07Z"
},
{
"call_id": "tc_002",
"turn": 5,
"tool_name": "book_flight",
"input": {
"flight_id": "AA123",
"num_passengers": 2,
"payment_method": "credit_card_****1234"
},
"output": {
"booking_id": "BK_9876543",
"status": "confirmed",
"confirmation_email_sent": True
},
"status": "success",
"latency_ms": 2100,
"timestamp": "2026-06-02T10:01:30Z"
}
]
}
Log this in a database. Query it to answer "Was this flight booked?" or "What was the input to that tool call?"
Layer 3: Agent Decision State
The current state machine state: what is the agent working on right now, what's next, what decision is pending?
{
"state_machine": {
"current_state": "awaiting_payment_confirmation",
"previous_states": [
"idle",
"search_in_progress",
"results_presented",
"seat_selection_complete",
"price_confirmed"
],
"user_request": "Book flight from NYC to SF next Tuesday for 2 people",
"progress": {
"flights_searched": True,
"flight_selected": True,
"flight_id": "AA123",
"seats_assigned": True,
"price_confirmed": True,
"payment_authorized": False,
"booking_complete": False
},
"pending_decision": {
"type": "confirm_payment",
"amount_usd": 500.00,
"awaiting_user_response": True
}
}
}
Use a state machine (e.g., with the transitions Python library) to enforce valid state transitions. If the agent tries to book before payment, reject it.
Layer 4: Memory and Context Summaries
User preferences, previously learned facts, and periodic summaries of long conversations.
{
"user_context": {
"user_id": "user_456",
"preferred_airlines": ["American Airlines", "United"],
"seat_preference": "aisle",
"dietary_preferences": "vegetarian",
"loyalty_program": "AAdvantage_#123456"
},
"conversation_summary": {
"last_summarized_turn": 15,
"summary": "User searched for flights NYC to SF on 2026-06-09 for 2 people. Found AA123 ($250/person) and UA456 ($320/person). Selected AA123. Confirmed price ($500 total). Now awaiting payment authorization.",
"generated_at_turn": 16,
"tokens_saved": 800
}
}
Summarize every 10-15 turns to keep context manageable.
Building a State Management Template
Use this template and adapt for your agent workflow:
from dataclasses import dataclass, asdict
from enum import Enum
import json
import time
from datetime import datetime
class AgentState(Enum):
IDLE = "idle"
GATHERING_INFO = "gathering_info"
DECISION_PENDING = "decision_pending"
EXECUTING = "executing"
ERROR = "error"
COMPLETE = "complete"
@dataclass
class ToolCall:
call_id: str
turn: int
tool_name: str
input: dict
output: dict
status: str # "success" | "error" | "timeout"
latency_ms: float
timestamp: str
@dataclass
class ConversationTurn:
turn: int
role: str # "user" | "agent" | "system"
content: str
action: str = None # For system turns
timestamp: str = None
class AgentStateManager:
def __init__(self, conversation_id: str):
self.conversation_id = conversation_id
self.turns: list[ConversationTurn] = []
self.tool_calls: list[ToolCall] = []
self.current_state = AgentState.IDLE
self.state_history = [AgentState.IDLE]
self.user_context = {}
self.pending_decision = None
def add_turn(self, role: str, content: str, action: str = None) -> None:
"""Record a conversation turn."""
turn_num = len(self.turns) + 1
timestamp = datetime.utcnow().isoformat() + "Z"
turn = ConversationTurn(
turn=turn_num,
role=role,
content=content,
action=action,
timestamp=timestamp
)
self.turns.append(turn)
def record_tool_call(self, tool_name: str, input_params: dict, output: dict, status: str = "success") -> str:
"""Record a tool call. Returns call_id for reference."""
call_id = f"tc_{len(self.tool_calls) + 1:03d}"
turn_num = len(self.turns)
start_time = time.time()
latency_ms = (time.time() - start_time) * 1000
timestamp = datetime.utcnow().isoformat() + "Z"
tool_call = ToolCall(
call_id=call_id,
turn=turn_num,
tool_name=tool_name,
input=input_params,
output=output,
status=status,
latency_ms=latency_ms,
timestamp=timestamp
)
self.tool_calls.append(tool_call)
return call_id
def transition_state(self, new_state: AgentState) -> bool:
"""Transition to a new state. Returns True if valid, False otherwise."""
# Define valid transitions
valid_transitions = {
AgentState.IDLE: [AgentState.GATHERING_INFO],
AgentState.GATHERING_INFO: [AgentState.DECISION_PENDING, AgentState.ERROR],
AgentState.DECISION_PENDING: [AgentState.EXECUTING, AgentState.IDLE, AgentState.ERROR],
AgentState.EXECUTING: [AgentState.COMPLETE, AgentState.ERROR],
AgentState.ERROR: [AgentState.IDLE, AgentState.COMPLETE],
AgentState.COMPLETE: [AgentState.IDLE]
}
if new_state in valid_transitions.get(self.current_state, []):
self.current_state = new_state
self.state_history.append(new_state)
return True
else:
return False
def set_pending_decision(self, decision_type: str, details: dict) -> None:
"""Set a decision awaiting user input."""
self.pending_decision = {
"type": decision_type,
"details": details,
"awaiting_response": True,
"created_at": datetime.utcnow().isoformat() + "Z"
}
def clear_pending_decision(self) -> None:
"""Clear the pending decision."""
self.pending_decision = None
def get_context_for_prompt(self, max_recent_turns: int = 10) -> str:
"""Generate context block for prompt, including recent history + state."""
recent_turns = self.turns[-max_recent_turns:] if len(self.turns) > max_recent_turns else self.turns
context_lines = [
"=== CONVERSATION HISTORY ===",
]
for turn in recent_turns:
context_lines.append(f"Turn {turn.turn} ({turn.role}): {turn.content}")
context_lines.extend([
"",
"=== AGENT STATE ===",
f"Current state: {self.current_state.value}",
f"Turns completed: {len(self.turns)}",
])
if self.pending_decision:
context_lines.append(f"Pending decision: {self.pending_decision['type']}")
return "\n".join(context_lines)
def export_to_json(self) -> str:
"""Export full state to JSON for storage."""
return json.dumps({
"conversation_id": self.conversation_id,
"turns": [asdict(t) for t in self.turns],
"tool_calls": [asdict(tc) for tc in self.tool_calls],
"current_state": self.current_state.value,
"state_history": [s.value for s in self.state_history],
"pending_decision": self.pending_decision,
"user_context": self.user_context
}, indent=2)
# Example usage
def flight_booking_agent():
"""Example: A flight booking agent with state management."""
manager = AgentStateManager("flight_booking_001")
# Turn 1: User input
manager.add_turn("user", "Book me a flight from NYC to SF next Tuesday for 2 people.")
manager.transition_state(AgentState.GATHERING_INFO)
# Turn 2: Agent response
manager.add_turn("agent", "I'll search for flights. Let me check available options.")
# Turn 3: Tool call
manager.add_turn("system", action="tool_call", content="search_flights")
flights = [
{"flight_id": "AA123", "price": 250},
{"flight_id": "UA456", "price": 320}
]
manager.record_tool_call("search_flights",
{"origin": "NYC", "destination": "SF", "date": "2026-06-09"},
flights)
# Turn 4: Present results
manager.add_turn("agent", "Found 2 flights: AA123 ($250) or UA456 ($320).")
manager.transition_state(AgentState.DECISION_PENDING)
manager.set_pending_decision("flight_selection", {"options": flights})
print(manager.get_context_for_prompt())
print("\nFull state:")
print(manager.export_to_json())
if __name__ == "__main__":
flight_booking_agent()
Operational Checklist for Agent State
1. Define Your State Machine
List all states and valid transitions:
IDLE → GATHERING_INFO
GATHERING_INFO → DECISION_PENDING (have enough info)
GATHERING_INFO → ERROR (user input invalid)
DECISION_PENDING → EXECUTING (user approves)
DECISION_PENDING → IDLE (user cancels)
EXECUTING → COMPLETE (success)
EXECUTING → ERROR (tool failure)
ERROR → IDLE (user wants to retry)
COMPLETE → IDLE (ready for new task)
Enforce these transitions in code. If the agent tries an invalid transition, log it as a bug.
2. Log Tool Calls Comprehensively
For every tool call, log:
- What tool was called
- What inputs were passed
- What output was returned
- How long it took
- Any errors
Use this to debug: "The agent booked the wrong flight—let me check the search_flights output."
3. Summarize Long Conversations
After 10-15 turns, generate a summary and embed it instead of the full history:
def summarize_conversation(turns: list[ConversationTurn], max_summary_length: int = 200) -> str:
"""Generate a summary of conversation turns."""
prompt = f"""Summarize this agent-user conversation in {max_summary_length} chars:
{chr(10).join(f'Turn {t.turn} ({t.role}): {t.content}' for t in turns)}
Summary:"""
# Call Claude to summarize
response = client.messages.create(
model="claude-opus-4-1",
max_tokens=100,
messages=[{"role": "user", "content": prompt}]
)
return response.content[0].text
4. Test State Transitions
Write tests for every state transition path:
def test_booking_happy_path():
"""Test: User books flight successfully."""
manager = AgentStateManager("test_001")
# Transitions
assert manager.transition_state(AgentState.GATHERING_INFO)
assert manager.transition_state(AgentState.DECISION_PENDING)
assert manager.transition_state(AgentState.EXECUTING)
assert manager.transition_state(AgentState.COMPLETE)
# Invalid transition should fail
assert not manager.transition_state(AgentState.GATHERING_INFO)
def test_booking_error_recovery():
"""Test: User cancels, agent goes back to idle."""
manager = AgentStateManager("test_002")
manager.transition_state(AgentState.GATHERING_INFO)
manager.transition_state(AgentState.DECISION_PENDING)
manager.transition_state(AgentState.ERROR)
assert manager.transition_state(AgentState.IDLE) # Recovery
if __name__ == "__main__":
test_booking_happy_path()
test_booking_error_recovery()
print("All state transition tests passed!")
Common Pitfalls and Solutions
Pitfall 1: State Drift
Problem: The agent's internal state doesn't match the database. Example: agent thinks it booked a flight, but the booking_complete flag in the database is False.
Fix: After every state transition, write to the database. Query the database to check state before making decisions.
# Good: Write state after each transition
manager.transition_state(AgentState.EXECUTING)
database.update_agent_state(conversation_id, manager.current_state.value)
Pitfall 2: Lost Tool Call Results
Problem: The agent calls a tool, gets a result, but doesn't record it. Later, you can't debug why it made a certain choice.
Fix: Record tool calls in the database immediately after they return, before doing anything else.
# Good: Log before using the result
result = flight_search_tool(origin="NYC", destination="SF")
manager.record_tool_call("search_flights", {...}, result)
# Now process the result
Pitfall 3: Unbounded Context
Problem: Conversation history grows indefinitely, consuming all context tokens and slowing down inference.
Fix: Summarize and rotate out old turns. Keep recent turns verbatim, summarize older ones.
def prune_conversation(turns: list, recent_turns_to_keep: int = 5):
"""Keep recent turns verbatim, summarize the rest."""
if len(turns) <= recent_turns_to_keep:
return turns
old_turns = turns[:-recent_turns_to_keep]
recent_turns = turns[-recent_turns_to_keep:]
summary = summarize_conversation(old_turns)
return [
ConversationTurn(turn=0, role="system", content=f"Summary of earlier turns: {summary}"),
*recent_turns
]
Frequently Asked Questions
How do I handle concurrent agents?
If multiple agents can modify the same conversation state simultaneously, use database transactions and optimistic locking (version numbers). Before updating, check that the version matches. If it doesn't, you have a concurrency conflict—resolve it (merge, ask user, etc.).
Should I store state in the prompt or in a database?
Prompt: Pro—fast, no network latency. Con—limited size, no historical queries. Database: Pro—unlimited size, audit trail, queryable. Con—network latency, eventual consistency.
Best practice: Store state in database (source of truth), embed recent state in the prompt (for context).
How do I rollback if a tool call fails midway?
Design tool calls to be idempotent (calling twice = calling once). Example: "Create an order if it doesn't exist" instead of "Create an order." If a tool fails, the agent can retry without corruption.
For non-idempotent operations (transfers, deletions), use compensating transactions: "Book flight" + "If payment fails, cancel flight booking."
What's the right granularity for state transitions?
One transition per significant decision or action. Example: "search flights" is one transition, not five. Too many states = hard to test. Too few = you lose visibility into what happened.
Can I use an event log instead of a state machine?
Yes. Instead of tracking state, log every decision as an event: {"event": "flight_searched", "timestamp": "...", "data": {...}}. Replay events to reconstruct state. This is more flexible but harder to enforce constraints (you can't prevent invalid transitions).
Hybrid: Use state machine for enforcement, events for audit trail.