Stop AI from Making Stuff Up — How to Avoid AI “Confidently Spouting Nonsense”
Ever had this experience: you ask AI a question, it responds sounding super professional and authoritative. Then you fact-check it, and realize everything it said is wrong?
I have. Pretty awkward.
This is the AI “hallucination” problem — it’s not trying to trick you, it genuinely believes what it’s saying is correct.
The prompt技巧 (prompt techniques) I’m sharing today are specifically for dealing with this problem. Use them well, and AI accuracy can improve significantly.
It’s like having a friend who’s super enthusiastic and wants to help answer everything you ask. Sometimes they really know the answer, but sometimes they’re afraid of disappointing you, so they “fill in the gaps” based on scattered information.
Claude is the same way. Its goal is to help you, so sometimes it gets “overly enthusiastic.”
Where’s the Problem
Claude sometimes “makes up” facts because it wants to help you too much.
You ask: “How much did the world’s heaviest hippo weigh?”
It might not know the exact answer, but doesn’t want to let you down, so it “pieces together” some data that looks reasonable from fragments in its training data.
And it says it with such confidence, sounding so self-righteous, you wouldn’t suspect a thing.
These situations are most prone to problems:
- Asking obscure knowledge
- Processing super long documents then asking about specific details
- Questions that have misleading information themselves
Give AI a Chance to Say “No”
The simplest approach: explicitly tell Claude “just say you don’t know if you don’t know.”
Ask it directly what Beyoncé’s 8th album is, and it might confidently tell you a wrong answer because it doesn’t want to disappoint you.
But if you add: “If you’re not sure or the information isn’t accurate, please just say you don’t know.”
Its response changes: “I’m not sure if Beyoncé has an 8th studio album. As far as I know, she has released 7 studio albums.”
See, just added that one sentence.
Similar prompt技巧 include:
Please only answer if you're certain the answer is correct. If uncertain, please state clearly.
If you're not confident in your answer, tell me you don't know.
Don't guess or make up information. If not mentioned in the document, state directly.
Give Claude a “way out,” and it won’t stubbornly give you wrong answers. This trick is simplest and noticeably improves AI accuracy.
Make AI Find Evidence First, Then Answer
This works especially well when processing long documents.
Say you give Claude a dozens-page company prospectus and ask: “How many subscribers did this company have on May 31, 2020?”
Ask directly, and it might get led astray by “distractor information” in the document, thinking similar numbers are the answer.
Make Claude “find evidence” first:
Please read the document below, then answer the question.
[Document content...]
Question: How many subscribers did this company have on May 31, 2020?
Please answer according to these steps:
1. First list all original text quotes related to subscriber count in <Relevant Quotes> tags
2. Then give final answer based on these quotes
3. If no clear answer in quotes, state that document lacks relevant information
Claude will obediently find evidence first, rather than guessing based on impressions.
It might respond like this:
<Relevant Quotes>
- "As of December 31, 2019, we had 85,000 subscribers"
- "As of May 31, 2020, our subscribers grew 49x"
</Relevant Quotes>
Answer: Based on document information, as of May 31, 2020, subscribers were approximately 85,000 × 49 = 4.165 million.
The answers it gives will be based on quotes it finds, not relying on “feeling.” This is the most practical of prompt技巧, significantly improving AI accuracy.
Lower “Temperature,” Make AI Less “Creative”
Claude has a parameter called “temperature,” ranging from 0 to 1.
At temperature 0, answers are most stable, most conservative, almost identical each time.
At temperature 1, answers are more creative, more varied.
Higher temperature means more “creative,” but also more likely to “let itself go,” greater chance of hallucinations.
When you need particularly accurate answers, you can ask for low temperature:
Please answer this question in a conservative, accurate way, prioritize correctness over creativity.
Although you can’t directly control the temperature parameter, you can guide Claude to adopt a more cautious attitude through prompt技巧.
Look at Some Examples
When asking obscure knowledge
Directly ask: “Tell me what the heaviest hippo in 1905 weighed.” Claude might make up a number.
Add “If you’re not sure or don’t have exact data in this area, please just say you don’t know,” and it’ll honestly admit not knowing.
It might answer: “I don’t have exact data on the heaviest hippo in 1905, can’t accurately answer this question.”
When processing long documents
Directly ask revenue data, Claude might get confused by various numbers in the document.
Have it first list all original text data related to revenue in
When there’s distracting info in the document
I’ve encountered this situation before, a document with a bunch of similar numbers, but none were the data point for the time I wanted.
Make Claude first confirm which information directly corresponds to the target date, only give answers based on directly relevant information, and it won’t get misled by surface information.
Combining Tricks Works Best
These techniques can be combined:
Please read the document below, then answer the question.
[Document content...]
Question: [Your question]
Please answer according to these requirements:
1. First list original text quotes supporting your answer in <Evidence> tags
2. Ensure each conclusion has clear evidence support
3. If not enough info in document, clearly say you don't know
4. Don't guess or make up data
Both gives Claude space to say “no,” and requires it find evidence first — double protection.
Pitfalls to Avoid
Don’t just ask for results without process. Directly asking for answers is prone to problems.
Don’t fear “I don’t know.” When Claude says it doesn’t know, it’s actually a good thing, better than confidently giving you wrong info.
Don’t use leading questions. “Is this company’s revenue pretty high?” This type of question easily leads Claude to agree with you. Directly ask “What is this company’s revenue? Please answer based on data,” this is more objective.
Key Points
AI “hallucinating” is normal, not a bug.
Give Claude the right to say “I don’t know,” and it won’t make things up.
When processing long documents, make Claude find evidence before drawing conclusions.
When high AI accuracy is needed, guide Claude to be more conservative, not too “creative.”
Combining multiple prompt技巧 works best.
When Claude says “I don’t know,” it’s actually helping you avoid being misled by wrong information.
Next Steps
Avoiding AI hallucinations is just part of Prompt工程. Coming up: few-shot prompting, multi-step reasoning, output format control — these prompt技巧.
Combine these techniques, and Claude can really become your capable AI assistant.
Ever been burned by AI “confidently spouting nonsense”? What happened? Share your experience in the comments.
Found this article helpful?
- Like: If helpful, give it a like to let more people see it
- Share: Share with friends or colleagues who might need it
- Follow: Follow 梦兽编程 so you don’t miss more practical technical articles
- Comment: Any questions or thoughts? Welcome to discuss in the comments
Your support is my biggest motivation for creating!