Recipe

How to Stop Your AI From Making Things Up

A 15-minute routine that makes your AI catch its own mistakes before you ever see them — plan first, second set of eyes, check the source.

The scary part of AI was never that it is dumb. It is that it can hand you a wrong number with a straight face, and you will nod. The fix is a routine, not blind trust. Make it plan before it works. Bring in a fresh set of eyes to tear the plan apart. Point it at your real numbers and tell it to look, not guess. Make it check itself at every step. And give it permission to say three words most people never allow it: “I do not know.” Fifteen minutes of habit that catches the mistake before it reaches you.

Total time

15 minutes

Difficulty

Easy

Makes

Numbers you can actually trust

Tool

Claude or ChatGPT

Ingredients

The task you are worried about getting wrong
The source it should check against (a report, a connected data source, or last period's version)

The prompt

Before you touch this, write me a plan — do not start yet. Then launch a subagent to critique the plan with fresh eyes and tell me where it will fail. Use the data I gave you as the source of truth; if you are not sure of a number, look it up, do not guess. At each step, stop and QC yourself against the source. I would rather you say “I do not know” than make something up.

The method

Here is the fear, and it is the right one to have. The AI does a clean, confident job on a report, and buried in the middle is one number it invented. Not a crazy number — a plausible one, in the same font as all the real ones, riding along to the total. You reread it twice and miss it both times, because it looks exactly like the truth. You do not fix that by trusting the machine more or less. You fix it with a routine that makes the AI catch its own mistakes before they reach you.

Make it plan first. "Write me a plan — do not start yet." Most wrong numbers begin as a wrong approach, and the plan is where you catch it while it is free to fix.
Bring in a second set of eyes. "Launch a subagent to critique this plan and tell me where it will fail." A reviewer that has not been talked into your assumptions finds the holes.
Ground it in the source. "Use these numbers as the source of truth. If you are not sure, look it up — do not guess." An AI reading your real data has far less room to invent it.
Make it QC itself at every step. "Stop and QC yourself against the source before you move on." Mid-task, not just at the end.

A few habits that catch the confident errors. Give it an out — "I would rather you say ‘I do not know’ than guess" is the single most useful line you can add. Make every number show its papers ("tell me which file and row each figure came from"). Ask for the same number two ways and see if they match. Check one number at random like an auditor and trace it yourself. And do not lead the witness: ask "is this right, and how do you know?" instead of "this looks high, right?" — an eager AI will just agree with a leading question.

The deepest fix is not more self-checking; it is never letting the AI work from memory. A number read live from the source barely has room to go wrong — which is the whole point of a real data connection. Make these checks permanent by putting them in your instructions file, and when a long session starts to drift, hand off to a fresh one before it costs you. More on the Learn page.

On the plate

The good draft

What good looks like: “Store 12 (Mission & 3rd) did $18,400 last week — that is from the Weekly Sales report, row 12. I could not find a figure for Store 7; its row was blank, so I left it out rather than estimate. Two numbers looked off, so I totaled revenue by store and by day and they matched.”

The bad draft

What a confident guess looks like: “All 8 stores are performing well, with total revenue around $150,000 for the week.” No sources, no store names, a suspiciously round total — every number here could be invented and you would never know.

Notes from the kitchen

Why does AI make up numbers in the first place?

Because it is built to produce the most plausible next words, not to look things up. When it has the real figure, it uses it. When it does not, it fills the gap with something that fits the pattern — a number that looks right in context. That is a hallucination: not a lie, a confident guess. So the fix is to never leave it a gap to fill: give it the source, and give it permission to say it does not know.

What does "launch a subagent" mean — do I need to be technical?

No. In Claude you just ask, in plain English, for a subagent to review the work with fresh eyes, and it spins up a reviewer that has not seen your chat. Nothing to install. If your tool does not have subagents, open a new chat, paste in the plan or result, and ask it to critique it as a skeptic who knows nothing about it.

Isn't asking the AI to check itself like the fox guarding the henhouse?

Partly, which is why self-review alone is not the whole recipe. The two moves that break the loop are the fresh set of eyes (a reviewer that has not been talked into the same assumptions) and grounding it in your source data. Self-QC catches the careless errors; those two catch the confident ones.

Does this guarantee it won't hallucinate?

No, and do not trust anyone who says otherwise. It cuts hallucinations hard and catches most of them earlier, before a wrong number carries through. The honest bar is not perfection, it is fewer mistakes than a rushed human, caught sooner. You still give the final answer a human look when it matters.

How do I make these checks automatic?

Put them in your instructions file. Write "always plan first, always check the source, QC yourself before showing me" into your CLAUDE.md or Project instructions and the AI runs these checks by default, every session, instead of waiting for you to ask.

Make it plan first. "Write me a plan — do not start yet." Most wrong numbers begin as a wrong approach, and the plan is where you catch it while it is free to fix.
Bring in a second set of eyes. "Launch a subagent to critique this plan and tell me where it will fail." A reviewer that has not been talked into your assumptions finds the holes.
Ground it in the source. "Use these numbers as the source of truth. If you are not sure, look it up — do not guess." An AI reading your real data has far less room to invent it.
Make it QC itself at every step. "Stop and QC yourself against the source before you move on." Mid-task, not just at the end.

About the author

By the Expo Kitchen. The Expo Kitchen is where Expo tests AI recipes for restaurant operators — plain-English, do-this-on-Saturday guides, written by operators, for operators. Browse more recipes and guides.