Skip to main content

Chain-of-Thought (CoT) Prompting: The Foundation

The most profound leap in prompt engineering came from a simple realization: asking an LLM to show its work doesn't just give you the answer, it helps the model find the right answer in the first place.

Introduction​

Welcome to the world of advanced prompting. In the previous chapter, we mastered the fundamentals of crafting clear and effective prompts. Now, we will explore techniques that can elicit complex reasoning and problem-solving behaviors from LLMs. The cornerstone of this new paradigm is Chain-of-Thought (CoT) prompting.

CoT prompting is a technique that encourages an LLM to break down a complex problem into a series of intermediate, sequential steps before arriving at a final answer. It's the difference between asking a student for the answer to a math problem and asking them to show their work. The very act of generating the intermediate steps improves the model's reasoning process, leading to dramatically better performance on tasks that require logic, arithmetic, or commonsense reasoning.

The "Aha!" Moment: Why CoT Works​

Standard prompting (where you just ask for the answer) requires the model to perform all of its reasoning "silently" and produce the final answer in one go. This is a difficult and error-prone process.

CoT works by externalizing the reasoning process. It mimics the way humans often think through a problem: step-by-step. By prompting the model to generate this chain of thought, we provide it with a structured "scratchpad" that has several key benefits:

  • Reduces Cognitive Load: The model can focus on one logical step at a time, rather than trying to solve the entire problem at once.
  • Enables Self-Correction: By laying out its reasoning, the model can sometimes spot its own errors and correct them in a subsequent step.
  • Provides Transparency: It allows us, the prompt engineers, to look inside the "black box" of the model's reasoning. We can see how it arrived at an answer, which is crucial for debugging and building trust.

Zero-Shot CoT: The Simplest Form​

The easiest way to elicit a chain of thought is to simply ask for it. This is known as Zero-Shot CoT, because you are not providing any examples of a reasoning process. You simply append a magical phrase to your prompt: "Let's think step by step."

Standard Prompt:

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
A:

(This can sometimes produce the wrong answer, like 11, by simply adding 5 + 2 + 3).

Zero-Shot CoT Prompt:

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?

A: Let's think step by step.

By adding that simple phrase, you prompt the model to first generate a reasoning chain and then the final answer.

Example CoT Output:

Let's think step by step.
1. Roger starts with 5 tennis balls.
2. He buys 2 cans of tennis balls.
3. Each can has 3 tennis balls, so 2 cans have 2 * 3 = 6 tennis balls.
4. The total number of tennis balls is the initial number plus the new balls: 5 + 6 = 11.
The answer is 11.

(Wait, I made a mistake in my example! Let me correct that... 5 + 6 is indeed 11. Let's try a different example where the error is more common.)

Let's try again with a better example.

Standard Prompt:

Q: A group of 5 friends is going to the movies. They each buy a ticket for $12 and a popcorn for $8. They also share 2 large sodas that cost $6 each. What was the total cost?
A:

(A simple model might just add the numbers: 5 + 12 + 8 + 2 + 6 = 33, which is wrong.)

Zero-Shot CoT Prompt:

Q: A group of 5 friends is going to the movies. They each buy a ticket for $12 and a popcorn for $8. They also share 2 large sodas that cost $6 each. What was the total cost?

A: Let's think step by step.

Example CoT Output:

Let's think step by step.
1.  There are 5 friends.
2.  Each friend buys a ticket for $12 and a popcorn for $8. The cost per friend is $12 + $8 = $20.
3.  The total cost for all 5 friends' tickets and popcorn is 5 * $20 = $100.
4.  They also buy 2 large sodas at $6 each. The total cost for sodas is 2 * $6 = $12.
5.  The total cost is the cost of tickets/popcorn plus the cost of sodas: $100 + $12 = $112.
The answer is 112.

As you can see, breaking the problem down makes it much less likely for the model to make a simple arithmetic error.

Few-Shot CoT: Providing Examples of Reasoning​

For more complex or novel problems, Zero-Shot CoT may not be enough. The model might not know how to break the problem down. In these cases, we can use Few-Shot CoT, where we provide one or more examples of a complete reasoning process in our prompt.

Few-Shot CoT Prompt:

Q: A juggler can juggle 16 balls. Half of the balls are golf balls, and half of the golf balls are blue. How many blue golf balls are there?
A: Let's think step by step.
1. There are 16 balls in total.
2. Half of the balls are golf balls, so there are 16 / 2 = 8 golf balls.
3. Half of the golf balls are blue, so there are 8 / 2 = 4 blue golf balls.
The answer is 4.

Q: A group of 5 friends is going to the movies. They each buy a ticket for $12 and a popcorn for $8. They also share 2 large sodas that cost $6 each. What was the total cost?
A: Let's think step by step.

By providing a high-quality example of a reasoning chain, you are teaching the model the pattern of reasoning you expect it to follow. This is an incredibly powerful way to adapt the model to new and complex tasks.

When to Use CoT Prompting​

CoT is most effective for tasks that require:

  • Multi-step arithmetic: Word problems, financial calculations, etc.
  • Commonsense reasoning: Puzzles, spatial reasoning, or questions about cause and effect.
  • Symbolic reasoning: Problems that involve manipulating symbols or following a set of rules.

It is generally not necessary for simpler tasks like summarization, translation, or creative writing, where the reasoning is more implicit.

Key Takeaways​

  • Chain-of-Thought (CoT) prompting unlocks the reasoning capabilities of LLMs.
  • It works by prompting the model to break down a problem into a series of intermediate steps.
  • Zero-Shot CoT is the simplest form, achieved by adding "Let's think step by step" to your prompt.
  • Few-Shot CoT is more powerful and involves providing one or more examples of a complete reasoning process.
  • Use CoT for tasks that require logic, arithmetic, or commonsense reasoning.

What's Next?​

Chain-of-Thought is the foundation of advanced prompting. But what if a single chain of thought is not enough? What if there are multiple ways to solve a problem, and you want to find the most reliable one? In the next article, we will explore the concept of Self-Consistency, a technique that builds upon CoT by generating multiple reasoning paths and choosing the most consistent answer.

By mastering Chain-of-Thought, you are no longer just asking for answers; you are teaching the model how to think.