Skip to main content

Reflexion: Self-Reflecting Language Agents Guide

Experience is the best teacher. The Reflexion framework enables AI agents to reflect on their past actions, generate their own feedback, and improve performance over time without direct human intervention. By combining self-reflection with verbal reinforcement stored in memory, Reflexion creates learning agents that become more robust and reliable with each task attempt.

Key Takeaways​

  • Reflexion adds a self-reflection phase to the ReAct loop, enabling agents to analyze failed trajectories and generate learning insights
  • Verbal reinforcement is instructive feedback generated by the agent about its own mistakes, stored in memory for future tasks
  • Memory integration allows agents to leverage past lessons in new attempts, enabling autonomous improvement without human feedback
  • The three-phase cycle (Execution, Evaluation, Self-Reflection) creates agents that learn from failure and become more robust over time

The Core Idea: Learn from a "Memory" of Past Mistakes​

The Reflexion framework builds upon the ReAct loop by adding a "post-mortem" phase where agents learn from their own failures. The framework operates in four distinct stages. First, execution: an agent using the ReAct framework attempts to complete a task, producing a trajectory of (Thought, Action, Observation) steps. Second, evaluation: a separate process assesses the outcome—was the task successful? Was it efficient? The evaluator provides a simple score (success or failure). Third, self-reflection: if the outcome was a failure, the Reflexion process kicks in. The agent is prompted with its own trajectory and asked to reflect on why it failed, pinpointing specific mistakes in reasoning or action. Fourth, verbal reinforcement and memory: the self-reflection output is concise instructive text (e.g., "I failed because I tried to search for two things at once. In the future, I should search for one piece of information at a time"). This verbal reinforcement is stored in the agent's memory—typically a text buffer or vector database. In the next episode, when the agent attempts the same or similar task, this memory is included in its initial prompt, effectively reminding it of learned lessons.

How Do You Implement the Reflexion Cycle in Practice?​

The Reflexion framework operates as a continuous improvement loop with three key components: the agent (performs actions), the evaluator (scores outcomes), and the reflector (generates lessons). Here's the step-by-step process.

Step 1: Agent Execution (ReAct Loop) The agent attempts a task using the ReAct framework—thinking, taking actions, and observing results. For example, the task might be "Find the birth dates of Elon Musk and the current CEO of Microsoft." The agent produces: Thought: "I need to find two birth dates. I can probably do this in one search." Action: search("birth dates of Elon Musk and Microsoft CEO") Observation: [Results are confusing and mixed.]

Step 2: Outcome Evaluation An evaluator determines if the task succeeded or failed. Success means the agent provided correct, complete information. Failure means missing or incorrect information, or abandonment of the task.

Step 3: Self-Reflection (If Failed) If the task failed, the agent is prompted: "Analyze your trajectory and explain why you failed. Your explanation should be concise instructive advice for your future self." The agent generates: "I failed because my search query was too complex and ambiguous. I tried to find two distinct pieces of information at once. In the future, when I need to find multiple pieces of information, I should break the problem down and perform a separate search for each piece of information."

Step 4: Memory Storage and Reuse This verbal reinforcement is stored in the agent's memory. In the next episode for the same task, the agent receives: "Your memory contains the following lesson: 'When I need to find multiple pieces of information, I should break the problem down and perform a separate search for each piece of information.' Question: Find the birth dates of both Elon Musk and the current CEO of Microsoft." Now informed by memory, the agent succeeds.

A Practical Example: A Failed Web Search Task​

Initial Task: Find the birth dates of both Elon Musk and the current CEO of Microsoft.

Episode 1: The Failure

The agent (using ReAct) produces this trajectory:

  • Thought: I need to find two birth dates. I can probably do this in one search.
  • Action: search("birth dates of Elon Musk and Microsoft CEO")
  • Observation: Search results are confusing and mixed, as the current Microsoft CEO is Satya Nadella, and the query is ambiguous.
  • Thought: I am confused by the results. I cannot extract the information. I have failed.

Evaluator verdict: Failure.

Self-Reflection Phase:

The Reflector receives:

You are a self-reflecting agent. You just failed at a task. Here is your trajectory:
[Insert the Thought, Action, Observation log from above]

Analyze your trajectory and explain why you failed. Your explanation should be a concise, instructive piece of advice for your future self.

Reflector output (Verbal Reinforcement):

"I failed because my search query was too complex and ambiguous. I tried to find two distinct pieces of information at once. In the future, when I need to find multiple pieces of information, I should break the problem down and perform a separate search for each piece of information."

Episode 2: Successful Retry with Memory

The agent is given the same task, but now its initial prompt includes its learned lesson:

Your memory contains the following lesson:
"When I need to find multiple pieces of information, I should break the problem down and perform a separate search for each piece of information."

Question: Find the birth dates of both Elon Musk and the current CEO of Microsoft.

Now the agent succeeds:

  • Thought: My memory tells me to search for one thing at a time. First, I will find Elon Musk's birth date.
  • Action: search("Elon Musk birth date")
  • Observation: Elon Musk was born on June 28, 1971.
  • Thought: I have Elon Musk's birth date. Now I need the birth date of the current Microsoft CEO. First I need to know who that is.
  • Action: search("current CEO of Microsoft")
  • Observation: The current CEO of Microsoft is Satya Nadella.
  • Thought: Now I know the CEO is Satya Nadella. I will search for his birth date.
  • Action: search("Satya Nadella birth date")
  • Observation: Satya Nadella was born on August 19, 1967.
  • Final Answer: Elon Musk was born on June 28, 1971, and Satya Nadella, the current CEO of Microsoft, was born on August 19, 1967.

The agent succeeded because it learned from its past mistakes and adjusted its strategy accordingly.

Why Is Reflexion So Powerful?​

Autonomous improvement is the first major advantage. Reflexion provides a mechanism for agents to get better over time without direct human intervention. Each failure becomes a learning opportunity automatically captured and reused. More human-like learning is the second advantage. The process of trial, error, and reflection is much closer to how humans learn than traditional supervised machine learning. Agents develop intuition and learn from experience rather than from labeled datasets. Improved robustness is the third. An agent that can learn from its failures is much more likely to be robust and reliable in the face of novel or unexpected situations. Each encountered failure strengthens the agent's defenses against similar future problems.

What Are Common Challenges When Implementing Reflexion?​

Challenge 1: Generating Quality Verbal Reinforcement Not all reflections are equally useful. Poor self-reflection can reinforce incorrect lessons. Solution: Use a strong LLM for the reflector role, and require specific, actionable output (avoid vague statements like "I need to try harder").

Challenge 2: Managing Memory Growth As agents complete more tasks, memory can grow unwieldy or become contradictory. Solution: Implement memory consolidation—periodically review and merge similar lessons, or use a vector database to retrieve only relevant past lessons for each new task.

Challenge 3: Distinguishing Between Systemic Failures and Task-Specific Failures Some failures reveal systematic weaknesses in the agent's approach; others are specific to the task at hand. A reflection that applies only to one specific task may not generalize. Solution: Use similarity metrics to determine when past lessons apply to new tasks, and allow lessons to be task-specific or domain-general based on their content.

Frequently Asked Questions​

How is Reflexion different from few-shot prompting with examples?​

Few-shot prompting provides static examples of how to complete tasks. Reflexion generates dynamic, personalized lessons from the agent's own failures. Reflexion adapts to specific failure modes the agent encounters, while few-shot examples are generic. Over many attempts, Reflexion creates a unique memory tailored to the agent's specific weaknesses.

Can you combine Reflexion with other frameworks like Chain-of-Thought?​

Absolutely. Reflexion is compatible with most prompting frameworks. Combining Reflexion with Chain-of-Thought produces agents that think step-by-step AND learn from their mistakes. Similarly, ReAct + Reflexion (the original framework) combines action-taking with reflection. You can also layer Reflexion on top of Tree-of-Thought or other reasoning frameworks.

How many reflection cycles does an agent typically need before improvement is visible?​

Improvement often appears within 3-5 failure cycles for straightforward tasks. However, complex tasks may require 10-20 iterations. The key is maintaining consistent evaluation metrics so you can measure whether lessons are actually being applied and improving performance.

Should the same agent generate its own reflections or use a separate "reflector" LLM?​

Both approaches work, but they have trade-offs. A single agent generating its own reflections is simpler and cheaper (one LLM instead of two). A separate reflector LLM can sometimes generate higher-quality reflections because it's prompted specifically for critiquing rather than executing. For production systems, a dedicated reflector often produces better results despite the added cost.

How do you prevent an agent's memory from becoming corrupted with incorrect lessons?​

Use a validation step. After generating verbal reinforcement, have the reflector verify that the proposed lesson makes sense and is properly articulated. Additionally, periodically test whether applying stored lessons actually improves performance—lessons that don't improve outcomes should be reviewed or discarded.

Best Practices for Reflexion Implementation​

Define clear success criteria before running episodes. The evaluator needs to know unambiguously whether a task succeeded or failed. Ambiguous success criteria lead to poor reflections.

Use specific, actionable reflections. Guide the reflector to generate advice that can actually be applied, not vague statements like "I should do better." Ask: "What specific mistake did you make? What will you do differently next time?"

Store lessons with metadata. Include timestamps, task types, and failure reasons with each verbal reinforcement. This enables selective memory retrieval—an agent working on task B might not need lessons learned from task A's failures.

Periodically audit memory quality. Review stored lessons to ensure they're accurate and useful. Remove outdated or incorrect lessons that could mislead future task attempts.

Test generalization. Verify that lessons learned on one task or domain transfer helpfully to similar tasks. Some lessons are domain-specific; others generalize broadly.

Further Reading​

What's Next?​

Both ReAct and Reflexion rely on the LLM's general reasoning abilities. But what if the problem is best solved not with words, but with code? In the next article, we'll explore Program-Aided Language Models (PAL), a technique that leverages an LLM's incredible coding abilities to solve problems that are difficult to tackle with natural language alone. Agents that can generate, execute, and reflect on code achieve a new level of problem-solving capability.