Recursive Thought Expansion: Dynamic Reasoning Depth
Recursive Thought Expansion (RTE) enables LLMs to dynamically control reasoning depth—allocating shallow, fast analysis to simple problems and deep, detailed reasoning to complex ones. Unlike fixed-structure techniques like Chain of Thought or Tree of Thoughts, RTE allows the model itself to decide when and where to decompose a problem further, creating adaptive hierarchical reasoning that matches problem complexity.
Key Takeaways
- RTE dynamically zooms in on complex problem parts by recursively breaking them into simpler sub-thoughts, creating a tree-like reasoning structure with variable branch depth.
- Each recursion level represents one abstraction layer: high-level problem → decomposed sub-problems → further sub-problems until all components reach "simple enough to execute."
- Implement RTE with three components: a main prompt (generate high-level thoughts), an evaluator prompt (judge complexity), and recursive calls (feed complex thoughts back to the main prompt).
- Best use cases: hierarchical project planning (multi-year roadmaps), in-depth research (exploratory reports with nested topics), curriculum generation, and any problem naturally decomposable into trees.
- The result: plans and analyses with appropriate detail at each level—no wasted computation on trivial steps, no shallow treatment of complex ones.
What Is Recursive Thought Expansion?
Recursive Thought Expansion is a prompting strategy where the LLM decomposes a complex thought into smaller sub-thoughts, then recursively applies the same decomposition to those sub-thoughts until reaching atomic units. The key difference from CoT or ToT is adaptivity: the model decides when to stop decomposing based on thought complexity, not on a fixed depth.
Consider organizing a 3-day tech conference. A standard Chain of Thought produces a flat list:
1. Choose a venue
2. Find speakers
3. Sell tickets
4. Create schedule
An RTE-powered system produces a tree:
Main Plan
├── Logistics & Venue (simple - stop here)
├── Content & Speaker Curation (complex - expand)
│ ├── Define tracks (simple - stop)
│ ├── Form program committee (complex - expand further)
│ │ ├── Identify experts (simple)
│ │ ├── Send invitations (simple)
│ │ ├── Set review criteria (simple)
│ │ └── Hold kickoff meeting (simple)
│ └── Create schedule (simple)
└── Marketing & Tickets (simple - stop)
The tree has variable depth: logistics remains a single bullet, but "speaker curation" expands to three levels because each level revealed further complexity. This adaptive depth is RTE's core advantage.
How Does RTE Work? The Core Algorithm
RTE follows a recursive loop with four stages:
Stage 1: Generate High-Level Decomposition
Prompt the model to break the problem into major components without worrying about detail:
"Break down the problem of [PROBLEM] into 4-6 major high-level components.
For each, output one line."
Example output for "Design a global e-commerce supply chain":
1. Supplier sourcing and negotiation
2. Inventory management and warehousing
3. Logistics and last-mile delivery
4. Returns and reverse logistics
5. Reporting and analytics
Stage 2: Evaluate Complexity
For each component, run an evaluator prompt to rate complexity on a scale (simple / moderate / complex):
"Rate the complexity of this task: [TASK]. Respond with one word: simple, moderate, or complex."
Example:
Supplier sourcing → complex
Inventory management → complex
Last-mile delivery → complex
Reporting → simple
Stage 3: Recursively Expand Complex Items
For any task rated "complex," run the main decomposition prompt again on that task specifically:
"Further break down this task: [COMPLEX_TASK]. Output 4-6 sub-components."
Example for "Supplier sourcing and negotiation":
1. Identify supplier categories by product type
2. Create evaluation criteria (lead time, cost, quality, compliance)
3. Issue RFQs and collect responses
4. Compare bids and create shortlist
5. Negotiate terms and sign contracts
6. Set up supplier communication channels
Then re-evaluate these sub-components. Any rated "complex" recursively expand again. Continue until all components are "simple."
Stage 4: Generate Final Nested Plan
Once the tree is fully expanded, compile all levels into a nested outline that represents the complete reasoning process.
A Concrete Example: Recursive Planning
Let's walk through RTE step-by-step for "Write a comprehensive course on machine learning."
Level 1 Decomposition:
Prompt: "Break down creating a machine learning course into major sections."
Output:
1. Fundamentals and prerequisites
2. Supervised learning algorithms
3. Unsupervised learning techniques
4. Deep learning and neural networks
5. Model evaluation and selection
6. Deployment and productionization
Level 1 Evaluation:
Fundamentals → moderate (expand)
Supervised learning → complex (expand)
Unsupervised learning → complex (expand)
Deep learning → complex (expand)
Model evaluation → moderate (expand)
Deployment → simple (stop)
Level 2 Expansion (Supervised Learning):
Prompt: "Break down 'Supervised learning algorithms' into sub-topics for a course."
Output:
1. Linear regression and extensions
2. Logistic regression and classification
3. Decision trees and ensembles (Random Forest, Gradient Boosting)
4. Support Vector Machines
5. Naive Bayes classifiers
6. Neural network basics
Level 2 Evaluation:
Linear regression → simple (stop)
Decision trees and ensembles → complex (expand further)
Support Vector Machines → simple (stop)
Naive Bayes → simple (stop)
Neural networks → complex (expand further)
Level 3 Expansion (Decision Trees):
Prompt: "Break down 'Decision Trees and Ensembles' into specific lessons."
Output:
1. Decision tree fundamentals (splits, entropy, information gain)
2. Building a decision tree from scratch (math + Python code)
3. Random Forest overview and hyperparameters
4. Gradient Boosting (GBM, XGBoost, LightGBM)
5. Comparison: when to use which ensemble method
Final Nested Outline:
Machine Learning Course
├── Fundamentals & Prerequisites (module 1)
│ ├── Python & data manipulation
│ └── Linear algebra essentials
├── Supervised Learning (module 2)
│ ├── Linear regression
│ ├── Logistic regression
│ ├── Decision Trees & Ensembles (module 3)
│ │ ├── Decision tree fundamentals
│ │ ├── Building trees (math + code)
│ │ ├── Random Forest
│ │ ├── Gradient Boosting
│ │ └── When to use which
│ ├── Support Vector Machines
│ └── Neural Networks Intro (module 4)
│ ├── Perceptrons & activation functions
│ └── Backpropagation basics
├── Unsupervised Learning (module 5)
└── Deployment (module 6)
Notice how depth varies: "Deployment" remains a leaf (not decomposed), while "Decision Trees" expands to five sub-lessons. This matching of depth to complexity is RTE's core strength.
Implementing RTE Programmatically
A production RTE system consists of three functions: decompose, evaluate, and recurse.
import anthropic
client = anthropic.Anthropic()
def decompose(task: str, depth: int = 0) -> list[str]:
"""Generate high-level decomposition of a task."""
prompt = f"""Break down this task into 4-6 major components or subtasks:
Task: {task}
For each component, output one short line (5-10 words).
Do NOT rank or evaluate them—just list them."""
response = client.messages.create(
model="claude-opus-4-1",
max_tokens=500,
messages=[{"role": "user", "content": prompt}]
)
lines = response.content[0].text.strip().split('\n')
components = [line.split('. ', 1)[-1].strip() for line in lines if line.strip()]
return components
def evaluate_complexity(task: str) -> str:
"""Evaluate complexity of a task: simple, moderate, or complex."""
prompt = f"""Rate the complexity of accomplishing this task: "{task}"
Respond with exactly one word: simple, moderate, or complex.
Criteria:
- simple: Single straightforward action, 1-2 hours
- moderate: Multi-step task, 1-2 days of work
- complex: Requires significant planning, multiple decisions, or domain expertise"""
response = client.messages.create(
model="claude-opus-4-1",
max_tokens=10,
messages=[{"role": "user", "content": prompt}]
)
rating = response.content[0].text.strip().lower()
return rating if rating in ['simple', 'moderate', 'complex'] else 'moderate'
def rte_expand(task: str, depth: int = 0, max_depth: int = 4) -> dict:
"""Recursively expand a task using RTE."""
if depth > max_depth:
return {'task': task, 'expanded': False, 'reason': 'max depth reached'}
# Generate decomposition
components = decompose(task, depth)
# Evaluate and recurse on complex items
expanded_components = {}
for component in components:
complexity = evaluate_complexity(component)
if complexity == 'complex' and depth < max_depth:
# Recursively expand
expanded_components[component] = rte_expand(component, depth + 1, max_depth)
else:
# Keep as leaf
expanded_components[component] = {
'task': component,
'complexity': complexity,
'expanded': False
}
return {
'task': task,
'complexity_breakdown': expanded_components,
'depth': depth,
'expanded': True
}
def print_rte_tree(node: dict, indent: int = 0) -> None:
"""Pretty-print an RTE tree."""
task = node.get('task', '')
complexity = node.get('complexity', 'N/A')
print(' ' * indent + f"- {task} ({complexity})")
if node.get('expanded'):
for component, sub_node in node.get('complexity_breakdown', {}).items():
print_rte_tree(sub_node, indent + 1)
# Example usage
result = rte_expand("Design and launch a SaaS product", max_depth=3)
print_rte_tree(result)
Run this on "Design and launch a SaaS product" and you'll see a tree that automatically goes deep on "product design" but stays shallow on "setup company incorporation."
When to Use RTE (vs. CoT or ToT)
| Technique | Best For | Depth | Complexity |
|---|---|---|---|
| Chain of Thought (CoT) | Sequential reasoning, step-by-step problems | Fixed, shallow | Low to moderate |
| Tree of Thoughts (ToT) | Exploring multiple solution branches in parallel | Fixed, moderate | Moderate to high |
| Recursive Thought Expansion (RTE) | Hierarchical planning, variable-depth reasoning | Variable, adaptive | High (with variable depth) |
Choose RTE when:
- The problem has natural hierarchical structure (projects, reports, curricula).
- Different components require vastly different levels of detail.
- You need a plan, not a direct answer (decision-making contexts).
- You want the model to "decide" how deep to go, not you.
Choose CoT when:
- You need the model to explain its reasoning step-by-step.
- The problem is linear and sequential.
- You need speed (CoT is faster than RTE).
Choose ToT when:
- You're exploring multiple solution branches.
- You need the model to compare and rank options.
- Parallelism helps (you can afford the extra tokens/calls).
Common Pitfalls and Solutions
Pitfall 1: Depth Explosion
If your evaluator is too lenient (everything is "complex"), the tree explodes exponentially.
Solution: Define complexity operationally. "Complex" means "requires expertise, multiple decisions, or significant time." Set a max depth (typically 3-4 levels is enough).
Pitfall 2: Evaluator Inconsistency
The same task might be rated "simple" by the model once and "complex" another time.
Solution: Use the same evaluator prompt and model throughout. Consider fixing the evaluation criteria in the prompt or using a temperature of 0.
Pitfall 3: Loss of Context
Deep recursion can lose sight of the original problem.
Solution: Include the full problem statement in recursive prompts, not just the sub-task. Example: "Within the context of [ORIGINAL_PROBLEM], break down this sub-task: [SUB_TASK].".
Frequently Asked Questions
How deep should an RTE tree go?
Most practical problems resolve in 2-4 levels. Level 1 is the main decomposition, levels 2-3 add detail, and level 4+ is rarely needed. If you reach depth 5 without completion, either increase your "simple" threshold or set a hard max depth to avoid token waste.
Can I mix RTE with other techniques?
Absolutely. Use RTE to plan (hierarchical breakdown), then use CoT at the leaf level (step-by-step execution). Or use RTE to explore decision branches, then Tree of Thoughts at key nodes. Hybrid approaches are common in practice.
How many tokens does RTE cost compared to CoT?
RTE costs 2-5x more tokens than CoT because each recursion level requires a separate API call and re-prompting. For simpler problems, CoT is more efficient. Reserve RTE for complex, hierarchical problems where the token investment pays off in better planning.
How do I automate the tree visualization?
Generate JSON output from your RTE function and use a tree-rendering library like graphviz (Python) or mermaid (in Markdown). The example code above returns a nested dict; you can serialize it to JSON and render with client-side tools.
Can I use RTE without an evaluator prompt?
Yes. You can manually inspect the first-level decomposition, manually identify which components need expansion, and manually run subsequent prompts. This is slower but works for one-off planning tasks. Automation (with evaluator) is only necessary if you're building a reusable system.