Skip to main content

Shared Scratchpads: Central State Management for Agents

A shared scratchpad is a centralized document or database that all agents can read and write to. Instead of passing messages point-to-point, agents update the scratchpad with their findings, and other agents read from it. This enables asynchronous coordination, reduces messaging complexity, and provides a single source of truth for the system's current state. Scratchpads are especially useful in swarm systems, complex orchestrations, and scenarios where agents need to see the full context (previous discoveries, decisions, and current constraints).

Think of a scratchpad as a whiteboard in a team meeting: one person writes a discovery, others see it immediately and build on it. Scratchpads solve problems like coordinating when multiple agents work on the same problem, tracking what has been attempted (avoiding duplicate work), and maintaining a detailed audit trail.

Types of Shared Scratchpads​

Document-based scratchpads​

A shared document (stored in memory, a database, or a file) that agents append to. Simple and human-readable:

# Problem Analysis

## Findings
- [agent_1, 2026-06-02 10:00] Claim X is true based on source A
- [agent_2, 2026-06-02 10:05] Claim X contradicts finding Y from last month
- [agent_3, 2026-06-02 10:10] Resolved: Claim X is true in context Z; finding Y applied to context W

## Current Best Solution
Path: [0, 1, 6, 7], Cost: 38, Discovered by: agent_2

## Attempted Approaches (to avoid duplication)
- Greedy shortest path: Cost 42 (agent_0)
- Dijkstra variant: Cost 38 (agent_2)

Structured scratchpads​

A JSON or database structure with well-defined fields that agents query and update:

{
  "problem_id": "shortest_path_001",
  "state": "in_progress",
  "candidates": [
    {"path": [0, 1, 6, 7], "cost": 38, "agent": "agent_2"},
    {"path": [0, 3, 5, 7], "cost": 42, "agent": "agent_0"}
  ],
  "explored_regions": [
    {"start": 0, "heuristic": "greedy", "result": "cost_42", "agent": "agent_0"}
  ],
  "pheromones": {"edge_0_1": 0.95, "edge_0_3": 0.6},
  "iteration": 5
}

Hybrid scratchpads​

Combine document and structured sections: structured data for queries, documents for narrative findings and reasoning.

Designing a Shared Scratchpad System​

Step 1: Define the schema​

List what information the scratchpad must contain:

  • Problem context: The task being solved, constraints, deadlines.
  • Discoveries: Findings made by agents, with timestamps and source attribution.
  • Candidates: Proposed solutions, ranked by quality.
  • Metadata: Iteration count, wall-clock time elapsed, agents active.
  • Locks/claims: Which agent is working on which part (to avoid duplicates).

Step 2: Choose a storage backend​

Options:

  • In-memory (dict or list): Fast but lost on restart; good for short-lived tasks.
  • File (JSON): Persistent; works across restarts; slow for high-concurrency.
  • Database (PostgreSQL, MongoDB): Scalable, concurrent access, transactions.
  • Git repo: Auditable, version history, but slower and requires custom merge logic.

Step 3: Implement read/write access patterns​

Agents need to:

  1. Read the current state without blocking others.
  2. Write updates atomically (no partial writes).
  3. Detect conflicts (e.g., two agents claim the same task).
  4. Reconcile conflicts (merge or choose winner).

Step 4: Add versioning and timestamps​

Every write should include a timestamp and agent ID. This allows replaying history and debugging.

Building a Shared Scratchpad System​

Here is a complete example:

import anthropic
import json
import time
import threading
from typing import Optional
from datetime import datetime

client = anthropic.Anthropic()

class SharedScratchpad:
    """A thread-safe shared scratchpad for multi-agent coordination."""
    
    def __init__(self):
        self.state = {
            "problem": "Find the shortest path from 0 to 7",
            "candidates": [],
            "explored_regions": [],
            "iteration": 0,
            "last_updated": datetime.utcnow().isoformat(),
            "lock": None  # Which agent is currently writing
        }
        self.lock = threading.Lock()
    
    def read_state(self) -> dict:
        """Agents read the current state. Non-blocking."""
        with self.lock:
            return json.loads(json.dumps(self.state))
    
    def claim_task(self, agent_id: str, task: str) -> bool:
        """Agent attempts to claim a task to avoid duplicates."""
        with self.lock:
            if self.state["lock"] is None:
                self.state["lock"] = {"agent": agent_id, "task": task, "timestamp": datetime.utcnow().isoformat()}
                return True
            return False
    
    def release_task(self, agent_id: str):
        """Agent releases its claimed task."""
        with self.lock:
            if self.state["lock"] and self.state["lock"]["agent"] == agent_id:
                self.state["lock"] = None
    
    def write_discovery(self, agent_id: str, discovery: dict):
        """Agent writes a discovery (solution, finding, etc.)."""
        with self.lock:
            # Add to candidates
            discovery["agent"] = agent_id
            discovery["timestamp"] = datetime.utcnow().isoformat()
            self.state["candidates"].append(discovery)
            
            # Keep top 10 by cost
            self.state["candidates"].sort(key=lambda x: x.get("cost", float("inf")))
            self.state["candidates"] = self.state["candidates"][:10]
            
            self.state["iteration"] += 1
            self.state["last_updated"] = datetime.utcnow().isoformat()
    
    def write_exploration_log(self, agent_id: str, approach: str, result: str):
        """Agent logs an approach it tried (to avoid duplication)."""
        with self.lock:
            self.state["explored_regions"].append({
                "agent": agent_id,
                "approach": approach,
                "result": result,
                "timestamp": datetime.utcnow().isoformat()
            })

class ScratchpadAgent:
    """An agent that uses the shared scratchpad."""
    
    def __init__(self, agent_id: str, scratchpad: SharedScratchpad):
        self.agent_id = agent_id
        self.scratchpad = scratchpad
        self.model = "claude-3-5-sonnet-20241022"
    
    def explore(self, graph: dict):
        """Agent explores the graph and updates the scratchpad."""
        state = self.scratchpad.read_state()
        
        # Read what others have explored to avoid duplication
        explored_approaches = [e["approach"] for e in state["explored_regions"]]
        best_cost = min([c["cost"] for c in state["candidates"]], default=float("inf"))
        
        prompt = f"""You are agent {self.agent_id} in a collaborative search. 
        
Explored approaches so far: {explored_approaches}
Best cost found by swarm: {best_cost}
Your job: Find a new path from 0 to 7 using a different approach than others.

Graph: {json.dumps(graph)}

Output JSON: {{"path": [...], "cost": number, "approach": "brief description"}}"""
        
        response = client.messages.create(
            model=self.model,
            max_tokens=600,
            messages=[{"role": "user", "content": prompt}]
        )
        
        try:
            result = json.loads(response.content[0].text)
            self.scratchpad.write_discovery(self.agent_id, {"path": result["path"], "cost": result["cost"]})
            self.scratchpad.write_exploration_log(self.agent_id, result["approach"], "completed")
            return result
        except json.JSONDecodeError:
            return None

def main():
    graph = {
        0: {1: 10, 3: 5},
        1: {6: 8},
        3: {5: 3},
        5: {7: 4},
        6: {7: 2},
        7: {}
    }
    
    scratchpad = SharedScratchpad()
    
    # Create agents and run them
    agents = [ScratchpadAgent(f"agent_{i}", scratchpad) for i in range(3)]
    
    print("Starting parallel exploration with shared scratchpad\n")
    
    for iteration in range(3):
        print(f"=== Iteration {iteration + 1} ===")
        
        # All agents explore in parallel (simulated with threads)
        threads = []
        for agent in agents:
            t = threading.Thread(target=agent.explore, args=(graph,))
            threads.append(t)
            t.start()
        
        for t in threads:
            t.join()
        
        state = scratchpad.read_state()
        print(f"Best solution: {state['candidates'][0] if state['candidates'] else 'None'}")
        print(f"Total explored: {len(state['explored_regions'])} approaches\n")
    
    final_state = scratchpad.read_state()
    print("Final scratchpad state:")
    print(json.dumps(final_state, indent=2))

if __name__ == "__main__":
    main()

Conflict Resolution Strategies​

Last-write-wins​

The most recent write wins. Simple but can lose data.

Merge-based​

Combine multiple writes intelligently. For lists (candidates), merge by keeping the best N. For scalars, choose based on a rule.

Lock-based​

Agents must acquire a lock before writing. Prevents conflicts but can cause deadlock if not careful.

Vector clocks​

Assign a logical clock to each agent's writes. Use vector clocks to detect concurrent writes and decide merge order.

Here is a simple merge strategy for solutions:

def merge_candidates(existing: list[dict], new_candidate: dict) -> list[dict]:
    """Merge a new candidate with existing ones."""
    candidates = existing + [new_candidate]
    # Keep only top 10 by cost
    candidates.sort(key=lambda x: x.get("cost", float("inf")))
    return candidates[:10]

Scratchpad vs. Message-Passing​

AspectScratchpadMessage-Passing
LatencyLow (direct access)Medium (queue overhead)
ConsistencyCentralized (easier to reason about)Distributed (harder)
ScalabilityLimited by single serverScales across machines
DiscoverabilityEasy (agents see all context)Hard (agents only see direct messages)
DebuggingEasy (view full state history)Hard (distributed logs)

Scratchpad Best Practices​

  1. Make writes atomic: Use locks or transactions. Partial writes corrupt state.
  2. Include timestamps on all entries: Essential for debugging and understanding causality.
  3. Limit scratchpad size: Archive or prune old entries to prevent unbounded growth.
  4. Add access control: Agents should only write to sections they own (e.g., agent_1 writes to findings["agent_1"], not others' sections).
  5. Periodically dump state: Save scratchpad snapshots for auditing and recovery.

Key Takeaways​

  • Shared scratchpads are centralized state documents that all agents read and write to, enabling asynchronous coordination.
  • Choose storage backend based on needs: in-memory for short tasks, database for high concurrency, git for auditability.
  • Implement atomic writes, versioning, and conflict resolution to keep state consistent.
  • Use scratchpads for swarms, complex orchestrations, and scenarios where agents need to see full context.
  • Combine scratchpads with message-passing for optimal performance: scratchpad for shared context, messages for direct agent-to-agent work.

Frequently Asked Questions​

Is a shared scratchpad a bottleneck?​

For read-heavy workloads (many agents reading, few writing), a scratchpad is efficient. For write-heavy workloads (many agents constantly updating), contention can cause slowdowns. Use database write batching or partition the scratchpad (e.g., one section per agent) to mitigate.

Can I use a database as a scratchpad?​

Yes. Databases provide transactions, concurrency control, and persistence. Use them for high-scale systems or when durability is critical. Trade-off: slightly higher latency than in-memory scratchpads.

What if the scratchpad grows very large?​

Archive old entries to a separate table. Keep recent entries (current iteration + last few) in the active scratchpad. Periodically compact and replay if needed for recovery.

How do I handle agent failures with scratchpads?​

The scratchpad is persistent, so when an agent restarts, it reads the current state and continues. No work is lost. Use timestamps to detect stalled agents (no updates for > timeout) and reassign their work.

Can I use a scratchpad for long-running systems (days/weeks)?​

Yes, but add retention policies. Archive entries older than N days; keep a recent sliding window. This prevents unbounded growth and keeps access fast.

Further Reading​