AI Hallucination Mitigation

An LLM "hallucinates" when it generates plausible-sounding content that is false. This happens for structural reasons — the model is optimised for likelihood, not truth — and no prompt or technique eliminates it completely. What you can do is drive the rate from uncomfortable (often 5–15% on factual tasks) to tolerable (< 1%), while exposing residual errors to your users and systems.

Ranked roughly by effort-to-payoff.

Retrieval grounding: the largest single lever

For any task where the answer exists in some corpus — documentation, customer records, policies — retrieval-augmented generation cuts hallucination rate by 50–90%. The model cites from the retrieved context instead of confabulating.

Three requirements for RAG to actually reduce hallucination (not just add noise):

  1. The prompt instructs citation from sources only. "Answer using only the provided sources. If the sources don't contain the answer, reply 'I don't know.'" Without this, the model reads the sources as hints and freely adds extra "knowledge."
  2. Retrieval recall is high enough. If the right chunk isn't in context, the model answers from priors — i.e. hallucinates. Aim for recall@10 > 90% on your eval; if you're below, fix retrieval before blaming generation.
  3. Source labels are explicit and parseable. [Source: doc-423] per chunk. The model cites using these labels, and you can verify post-hoc that cited sources actually support the claim.

This is RagImplementationPatterns territory in depth.

Constrained generation for structured outputs

If the output is a JSON schema, a specific DSL, or a closed set of choices, constrain generation to the valid grammar. Tools: outlines, lm-format-enforcer, OpenAI's response_format, Anthropic's tool use, grammar-constrained generation in llama.cpp.

Benefits:

Do this wherever the output is machine-consumed. You save retry logic, you eliminate a whole class of parsing errors, and you force the model to commit to one of the valid answers rather than invent a new one.

Calibrated abstention

Teach the model to say "I don't know" as a first-class output. Three moves:

Without the eval, the model learns to always answer because that's what the training distribution rewards. Adding unanswerable cases trains the calibration that matters.

Measure calibration: if the model says "I don't know" 5% of the time, of the remaining 95% confident answers, how often is it right? You want both numbers to shift — more abstention, higher confidence-conditional accuracy.

Self-verification passes

Have the model check its own output against the sources:

Draft answer: {answer}
Sources: {sources}

For each claim in the draft, identify which source supports it. 
If any claim is not supported, flag it.

Then either:

Caveat: self-verification has its own hallucination rate. Works better when the verification uses a separate model call (so it's not just continuing its own generation) and when sources are short enough to fully include.

Citation enforcement

Post-hoc verification of the citations the model produced:

  1. Model generates answer with inline citations: "The refund policy allows 30 days [source: doc-12]."
  2. Your code extracts [source: doc-12], fetches that source, and checks that the claim is supported.
  3. Unsupported claims → flag, re-prompt, or remove.

This is where you catch fabricated citations — the classic "the model cited a paper that doesn't exist" case. For a production system answering user queries, citation verification is a must-have, not a nice-to-have.

Uncertainty signals

Ask the model for confidence and use it:

Use uncertainty to route: high confidence → deliver; low confidence → escalate to human, ask a clarifying question, or abstain.

Domain-specific fine-tuning — with caveats

Fine-tuning on domain data can reduce hallucination by teaching the model the domain's vocabulary, common structures, and admission patterns. But it also teaches the model new "facts" that become memorised-and-potentially-wrong, especially as the real-world facts drift.

Guidance:

See LLMFineTuning for the recipe.

What doesn't work as well as claimed

Measurement

You need a task-specific factuality eval. Generic benchmarks (TruthfulQA, HaluEval) measure the average; your traffic isn't the average.

Build 100–500 questions where you know the right answer (from docs, database, or labelled corpus). For each model output:

RAGAS, TruLens, or a homegrown LLM-judge over these three axes works. Run weekly at minimum. See LlmEvaluationMetrics for the metric catalogue.

Residual hallucination is a product problem

Even with everything above, residual hallucination rate won't hit zero. At some point the engineering reaches diminishing returns and the question becomes: how does the product handle wrong outputs?

Further reading