Self-Refine and Critic

In plain words

You wrote your project report at 2 am. Instead of submitting the first draft, you read it, mark what is wrong, and rewrite those parts. An agent can do the same with its own answer: generate, critique, rewrite, repeat. The catch is that a model is good at spotting its own bad formatting and bad at spotting its own wrong facts, so the critique should come from a real tool wherever possible.

How it flows

A tiny example

history = []
answer = generate(task)
for i in range(3):
    problems = run_tool_check(answer)   # or self_critique(answer)
    history.append((answer, problems))
    if not problems:
        break
    answer = refine(task, history)      # sees everything tried so far
print(answer)

Notice that refine is handed the whole history, not just the latest critique, so it cannot repeat a mistake it already made.

What you will learn

• How an agent can look at its own answer, find the mistakes, and fix them.
• The three steps of a Self-Refine loop: generate, feedback, refine.
• Why a model checking itself is weak on facts, and how CRITIC fixes that with real tools.
• When to stop the loop, so you do not burn time and money forever.

The problem, simply

Think about your final year project report. You write the first draft at 2 am. It is almost right. One section is too long, one number is wrong, one figure caption is missing.

Now, what do you do? You read it again yourself. You mark the problems. Then you rewrite those parts. Maybe two rounds, then you submit.

An agent has exactly the same situation. It produces an answer that is 80 percent correct. A line of code has a small syntax error. A summary is longer than asked. A plan misses one edge case.

The obvious idea: let the agent read its own answer, write down what is wrong, and try again. That is the whole lesson. But there is a catch, and the catch is where the interesting engineering lives.

The idea

Three prompts, one model

Self-Refine is a technique from a 2023 research paper. It uses one model in three different roles:

1. Generate — produce a first answer for the task.
2. Feedback — look at that answer and write a critique. What is wrong?
3. Refine — rewrite the answer using the task, the old answer, and the critique.

Then feedback again on the new answer, refine again, and so on. You stop when the feedback step says "no issues left", or when your budget of passes is over.

The paper reported a solid average improvement across seven different tasks, including maths, code and dialogue. No extra training. No extra data. Just the same model, called three ways.

IMP

Important: the refine step must see the full history — every earlier answer and every earlier critique. If you drop the history, the model happily makes the same mistake again, and the whole thing stops helping.

A worked example

Suppose Priya asks the agent: "Write 3 short bullets about UPI, each under 60 characters."

First draft, bullet one is 95 characters long, and bullet three says UPI was invented in 1998.

Feedback step, the model checks its own work. It notices the length problem easily — length is something you can see. It does not notice the 1998 problem, because that wrong date came out of the same model and still looks perfectly reasonable to it.

Refine step, the model shortens bullet one. Feedback runs again, says "looks fine". Loop stops. Priya gets a neat, well formatted, factually wrong answer.

IMPRemember: a model is decent at judging style and format, and unreliable at judging its own facts.

CRITIC: give the critic real tools

CRITIC is another 2023 idea, and it fixes exactly that hole. Instead of asking the model to critique itself, you route the check through something outside the model:

• A search engine, for factual claims.
• A code interpreter, to actually run the code.
• A calculator, for arithmetic.
• Domain checkers — unit tests, a type checker, a linter.

The tool result comes back, and the critique is now grounded in something real, not in the model's own opinion. The refine step then works on that grounded critique.

In Priya's case, a date lookup catches the 1998 bullet immediately. The loop runs one more pass and fixes it.

One honest caveat: if your task has no external checker — say, writing a poem or picking a nicer heading — CRITIC becomes plain Self-Refine. Do not pay for a fake verifier that always returns "ok".

Same pattern, different names

You will hear this pattern called evaluator-optimizer. The evaluator scores the output and writes the critique, the optimizer rewrites it, and you loop until the evaluator is satisfied. It is the same loop with two role names.

In some agent toolkits it appears as an output guardrail — a validator that runs on the agent's final output. If the guardrail trips, the output is rejected and the agent retries. A guardrail that can call tools is CRITIC-shaped. A guardrail that is just a pure function is Self-Refine-shaped.

When to stop

Three stop conditions, and you want a combination, not one alone:

• The external verifier passes. Best one, when you have it.
• The model says "no issues" — cheap, but it lies sometimes, so require at least two passes.
• You hit the maximum number of passes. Always keep this as the last line of defence.

Build it

# A toy improvement loop. No API calls anywhere.
# Task: write 3 bullets about UPI, each under 60 characters.

# What the "model" produces on pass 1, 2 and 3 after seeing the history.
DRAFTS = [
    ["UPI was launched by NPCI and it is used for instant bank to bank transfer in India",
     "UPI works on a phone",
     "UPI was invented in the year 1998"],
    ["UPI moves money between banks instantly",
     "UPI works with a phone number or a UPI id",
     "UPI was invented in the year 1998"],
    ["UPI moves money between banks instantly",
     "UPI works with a phone number or a UPI id",
     "UPI was launched in 2016"],
]

KNOWN_WRONG = ["invented in the year 1998"]

def refine(pass_no, history):
    """Pretend model. A real one would read history and rewrite."""
    assert history is not None          # history must reach the refiner
    return DRAFTS[min(pass_no, len(DRAFTS) - 1)]

def self_feedback(bullets):
    """The model grading itself. It can see style, not truth."""
    return ["bullet %d is too long" % (i + 1)
            for i, b in enumerate(bullets) if len(b) > 60]

def external_check(bullets):
    """CRITIC step: a checker that lives outside the model."""
    problems = self_feedback(bullets)
    for i, b in enumerate(bullets):
        for bad in KNOWN_WRONG:
            if bad in b.lower():
                problems.append("bullet %d fails the fact check" % (i + 1))
    return problems

def loop(name, checker, max_passes=4):
    print("\n--- %s ---" % name)
    history = []
    bullets = []
    for i in range(max_passes):
        bullets = refine(i, history)
        problems = checker(bullets)
        history.append((bullets, problems))     # refiner sees everything tried
        print("pass %d: %d problem(s) %s" % (i + 1, len(problems), problems))
        if not problems:
            print("stop: checker is happy")
            return bullets
    print("stop: passes exhausted")
    return bullets

final_a = loop("Self-Refine (model checks itself)", self_feedback)
final_b = loop("CRITIC (outside checker)", external_check)
print("\nSelf-Refine last bullet :", final_a[-1])
print("CRITIC last bullet      :", final_b[-1])

Run it with python3 file.py. Look at two things. First, the self-checking loop stops after pass 2 and is happy — but its last bullet still says 1998. Second, the CRITIC loop keeps going one more pass, because the outside checker refuses to pass a wrong fact, and it ends with the correct bullet.

Where you will see this

• Coding agents like Claude Code and Cursor: they write code, run the tests, read the failures, and patch. The test runner is the external verifier.
• GitHub Copilot style review suggestions, where a second pass critiques the first draft of a patch.
• Customer support bots that draft a reply, run it through a policy checker, and rewrite before sending.
• Content and summary tools that check length, tone and banned words before showing you the output.
• Any pipeline where a linter or type checker sits between the model's output and production.

Common mistakes

• Rubber-stamp loops. Same model, same prompt style, critiquing its own work. It settles into "looks good to me" and you get zero improvement while paying full cost. Make the evaluator prompt structurally different, or use a separate smaller model as critic.
• Dropping the history. If refine only sees the latest critique, it forgets what it already tried and cycles between the same two wrong answers.
• Trusting self-critique on facts. Style and format, yes. Dates, numbers, API names, no. Ground those with a tool.
• Refining forever. Every pass costs latency and tokens. Budget one to three passes, then hand it to a human.
• A stub verifier. If the task has no real checker, do not build a fake one just to look CRITIC-shaped. You add delay and learn nothing.

If they ask in an interview

Q: What are the three steps of a Self-Refine loop, and why does the refine step need history?

A: Generate an answer, produce feedback on it, then refine using that feedback. The refine step gets all earlier answers and critiques, so it does not repeat a mistake it already made. Without history, quality drops noticeably.

Q: What does CRITIC change compared to Self-Refine?

A: It replaces self-critique with a verification step routed through external tools — search, a code interpreter, a calculator, unit tests. The critique is then grounded in a real result instead of the model's own opinion, which matters most for factual and numeric claims.

Q: How would you stop such a loop in production?

A: Combine conditions rather than relying on one. Stop when the external verifier passes, or when the model reports no issues and at least two passes are done, or when a hard maximum iteration count is hit.

Try these

1. Run the code with max_passes=1. Does the CRITIC loop still help? Write down why or why not.
2. Make the external checker noisy: with some probability it reports a fake problem. Watch how the loop behaves. This is closer to real guardrail stacks than the clean version.
3. Add a second checker that runs len(bullets) == 3 and feed it a draft with 4 bullets. See how the two checkers combine into one critique list.
4. Rewrite the loop so a "big" generator and a deliberately different "small" critic are separate functions with different rules. Does splitting them catch more problems than one function doing both?

Words, simply

Quick recap

• Generate, critique, rewrite, repeat — that is the whole loop, and the rewrite must see the history.
• A model is fine at judging its own style and terrible at judging its own facts, so ground the critique in a tool wherever you can.
• Always combine stop conditions and cap the number of passes; one to three is usually enough.