evidenceQuote (hallucination gate)
evidenceQuote is a required field on every LGTM review finding: the LLM must quote the exact offending code verbatim. Findings whose quote can't be grep'd back into the diff or the surrounding file are dropped as hallucinations before the user sees them.
The failure mode evidenceQuote fixes
LLMs confabulate. Ask a model to review a diff and it will sometimes produce findings about variables that don't exist, functions with the wrong name, or entire code paths that aren't in the file. These findings sound authoritative — they cite line numbers, name specific patterns, propose fixes — but they're referencing hallucinated code.
Users lose faith in an AI reviewer that fabricates evidence faster than for almost any other reason. A false positive that quotes real code is a discussion; a false positive that quotes code that doesn't exist is a bug in the tool.
The evidenceQuote contract
LGTM's structured-output schema requires every finding to include an evidenceQuote field: a verbatim substring from the code being reviewed. Not paraphrased, not summarised — the exact bytes the LLM is pointing at.
After each agent produces its findings, a deterministic validator (no LLM) checks each evidenceQuote against the diff and the fetched file content. If the quote can be found, the finding proceeds. If it can't — the LLM invented the code — the finding is dropped and logged. The user never sees it.
Straight-line matching handles trivial cases. Normalised matching (collapse whitespace, ignore trailing commas) handles quotes the LLM lightly reformatted. Fuzzy matching is intentionally not used — a fuzzy match on hallucinated code is worse than dropping the finding.
Why this beats "just trust the LLM's line numbers"
Line numbers are a much weaker anchor than a verbatim quote. An LLM can produce a plausible-looking line number for code that isn't there ("L42" when the file has 30 lines). A quote either matches or it doesn't — there's no ambiguity to hide behind.
The evidenceQuote also gives the synthesizer something concrete to reason about when dedup'ing multi-agent findings on the same code. Two agents both quoting the same substring becomes a consensus signal; two agents quoting different substrings that both match the code becomes two distinct findings.
What this doesn't catch
evidenceQuote is a hallucination filter, not a truth filter. A finding can quote real code and still be a false positive — the code exists, but the flagged issue doesn't apply. That's what the adversarial verifier is for.
Together, evidenceQuote (does the code exist?) + adversarial verifier (does the described problem exist in that code?) + convention-file loader (does the finding respect the repo's own rules?) form LGTM's three-layer precision floor.
How LGTM eliminates hallucinated findings
Verbatim quote required · deterministic validator · no fuzzy match
Go to the product pageFAQs
How is this different from evidence-anchoring in RAG systems?
Same idea, different setting. RAG systems anchor outputs to retrieved passages; evidenceQuote anchors code-review findings to the source code being reviewed. Both defeat confabulation by requiring the output to be grounded in text the system knows exists.
What if the LLM's quote has cosmetic differences (trailing space, quote style)?
The validator normalises whitespace and trailing commas before matching. A quote that differs only in cosmetic ways still matches. A quote that differs in identifiers, operators, or literal values doesn't match — those aren't cosmetic, they're semantic.
How many findings does evidenceQuote actually drop?
Varies by model and prompt maturity. On the July 2026 pipeline, the drop rate is in the low single digits per PR on the frontier models (gpt-5.4-pro, Claude Opus) and higher on mini/flash tiers. The gate is most valuable as a floor: even a highly hallucinatory small model can't produce findings that reference nonexistent code, which lets us route the docs and readability agents to the cheaper tier without regretting it.
Related across LGTM
Related terms
Adversarial verifier (LLM review)
An adversarial verifier is a second LLM pass whose job is to refute every finding produced by the primary review agents. Findings that survive refutation are kept; findings that don't are dropped before they reach the user.
AI code review
AI code review uses large language models (and increasingly multi-agent pipelines) to review pull requests for bugs, security issues, performance regressions, readability, and style — automatically, on every PR, in 30-90 seconds.
LLM code review pipeline
An LLM code review pipeline is the end-to-end system that ingests a GitHub PR webhook, fetches the diff, gathers repo context, runs one or more LLM agents in parallel, synthesizes their outputs, and posts the result back to GitHub as a review with inline comments.