Chapter 20: Hallucination Prevention, Detection, and Mitigation

Preamble#

Hallucination is the cardinal failure mode of generative language models and, by extension, the single greatest threat to the reliability of agentic AI systems. When a model produces output that is fluent, confident, and structurally coherent — yet factually incorrect, logically unsound, or unsupported by any evidence in its context — the downstream consequences propagate through every dependent agent, tool invocation, and committed artifact. In agentic settings, hallucination is not merely an inconvenience; it is a systemic integrity failure that can corrupt memory stores, trigger cascading incorrect tool calls, poison retrieval indices, and permanently degrade organizational knowledge bases.

This chapter formalizes hallucination as a measurable, classifiable, and mechanically addressable engineering problem. We define a rigorous taxonomy, identify root causes through distributional and information-theoretic analysis, architect prevention mechanisms as first-class system components, specify detection pipelines with quantifiable precision, design mitigation protocols with bounded recovery semantics, establish production-grade metrics, and construct adversarial testing infrastructure. The central principle: hallucination control is not a prompt engineering concern — it is an architectural invariant enforced at every layer of the agentic execution stack.

20.1 Taxonomy of Hallucinations: Factual, Logical, Contextual, Confabulatory, and Structural#

A precise taxonomy is the prerequisite for targeted prevention and detection. Hallucinations are not monolithic; they arise from distinct failure mechanisms and require distinct countermeasures.

20.1.1 Formal Definition#

Let $\mathcal{C}$ denote the context provided to the model (instructions, retrieved evidence, memory, tool outputs), $\mathcal{K}$ denote the ground-truth knowledge relevant to the task, and $\mathcal{O}$ denote the model's generated output. A hallucination is any atomic claim $c \in \mathcal{O}$ satisfying:

That is, a claim is hallucinated if it is either (a) not entailed by the provided context (intrinsic hallucination) or (b) inconsistent with ground truth (extrinsic hallucination), or both.

We further decompose:

Intrinsic hallucinations are detectable from context alone. Extrinsic hallucinations require external verification.

20.1.2 Taxonomy Categories#

20.1.3 Formal Category Definitions#

Factual Hallucination: Claim $c$ asserts a proposition $p$ where $p$ is verifiable against an authoritative knowledge base $\mathcal{K}_{\text{auth}}$:

Logical Hallucination: The output contains a reasoning chain $r_1 \rightarrow r_2 \rightarrow \cdots \rightarrow r_n$ where at least one inference step is invalid:

Contextual Hallucination: A claim $c$ references or paraphrases context $\mathcal{C}$ but distorts, omits, or inverts the meaning:

where $\text{faithful}(c, \mathcal{C})$ denotes semantic entailment from $\mathcal{C}$ to $c$.

Confabulatory Hallucination: The model generates a specific, detailed claim that is not present in $\mathcal{C}$, not derivable from $\mathcal{C}$, and presents it with high confidence:

Confabulations are the most dangerous hallucination type because they appear most credible to downstream consumers.

Structural Hallucination: The output $\mathcal{O}$ violates a structural constraint $\mathcal{S}$ (schema, type, format):

20.1.4 Severity Classification#

Define hallucination severity $\text{sev}(c)$ as a function of downstream impact:

where:

• $\text{propagation\_risk}$: probability that the hallucinated claim feeds into downstream agent decisions, memory writes, or tool invocations
• $\text{detectability}^{-1}$: inverse of how easily the hallucination can be caught (confabulations score highest)
• $\text{consequence}$: domain-specific impact (financial loss, safety risk, data corruption)

20.2 Root Cause Analysis: Training Data Gaps, Distributional Shift, Context Window Overflow, Retrieval Failure#

20.2.1 Causal Framework#

Hallucinations arise from identifiable failure mechanisms in the generation pipeline. Understanding root causes enables targeted prevention rather than symptom-level patching.

Causal Model: Let $H$ denote the event "hallucination occurs." The probability of hallucination is a function of contributing causes:

assuming approximate conditional independence of causes. The principal causes are:

20.2.2 Training Data Gaps (Parametric Knowledge Gaps)#

The model's parametric knowledge $\mathcal{K}_{\theta}$ is a lossy compression of the training corpus $\mathcal{D}_{\text{train}}$. For queries outside $\mathcal{D}_{\text{train}}$'s distribution or for long-tail facts with low frequency, the model lacks reliable parametric grounding:

The second factor is near 1.0 for most models — models default to generation rather than abstention, producing fluent confabulations in knowledge-sparse regions.

Information-Theoretic Perspective: The model's uncertainty about the correct answer for query $q$ can be quantified by the predictive entropy:

High predictive entropy signals that the model is uncertain, and generation in high-entropy regions is disproportionately likely to hallucinate. The hallucination risk per token:

20.2.3 Distributional Shift#

When the task input distribution at inference time $\mathcal{D}_{\text{inf}}$ diverges from the training distribution $\mathcal{D}_{\text{train}}$, the model operates out-of-distribution (OOD):

OOD inputs trigger interpolation or extrapolation from learned patterns, producing outputs that are syntactically valid but semantically unreliable.

Temporal Shift: Knowledge evolves after training cutoff. Facts that were true at training time may be outdated at inference time:

For facts with high temporal volatility (stock prices, API versions, geopolitical state), staleness directly correlates with hallucination probability.

20.2.4 Context Window Overflow#

When the total context $|\mathcal{C}|$ approaches or exceeds the model's effective processing capacity, attention degradation causes the model to lose track of critical evidence:

Empirically, attention fidelity follows a U-shaped curve (the "lost in the middle" phenomenon): evidence at the beginning and end of the context receives disproportionate attention, while evidence in the middle is partially or fully ignored.

Formal Attention Degradation Model: For evidence item $e_i$ at position $p_i$ in context of length $L$:

where $\tau_{\text{start}}, \tau_{\text{end}}$ are decay constants and $\alpha_{\text{base}}$ is the baseline attention floor. When $\text{effective\_attention}(e_i) < \theta_{\text{attn}}$, evidence $e_i$ is effectively invisible to the model, and claims requiring $e_i$ will be generated from parametric memory (hallucination-prone) rather than from context.

20.2.5 Retrieval Failure#

In retrieval-augmented systems, hallucination occurs when:

1. Retrieval miss: Relevant evidence exists but is not retrieved ($\text{recall} < 1$)
2. Retrieval noise: Retrieved evidence is irrelevant or misleading ($\text{precision} < 1$)
3. Retrieval staleness: Retrieved evidence is outdated
4. Retrieval poisoning: Retrieved evidence itself contains errors

Retrieval-Hallucination Coupling: The model may treat irrelevant retrieved evidence as authoritative, producing hallucinations that are worse than no-retrieval baseline — the model confabulates a connection between the query and the irrelevant evidence. This is the retrieval poisoning failure mode.

20.2.6 Root Cause Attribution Matrix#

20.3 Prevention by Design#

Prevention is architecturally superior to detection. A system that prevents hallucination by construction is more reliable, more efficient, and more auditable than one that generates freely and filters post hoc.

20.3.1 Retrieval-Grounded Generation: Constraining Output to Evidence-Supported Claims#

Principle: Every factual claim in the model's output must be traceable to a specific evidence item in the retrieved context. The model is instructed — and mechanically constrained — to generate only claims that are entailed by provided evidence.

Formal Constraint: Let $\mathcal{O} = \{c_1, c_2, \ldots, c_m\}$ be the set of atomic claims in the output. The groundedness constraint requires:

where $\mathcal{E}$ is the evidence set and $\theta_{\text{ground}} \in [0, 1]$ is the minimum entailment confidence.

Grounding Score: The overall groundedness of an output:

Architectural Implementation:

1. Evidence-first context construction: Place retrieved evidence prominently in the context window (first segment after system prompt), not buried in the middle.
2. Explicit grounding instruction: The compiled prompt includes a hard constraint: "Every factual claim must reference a specific evidence item by identifier. Claims without supporting evidence must be explicitly flagged as uncertain."
3. Citation slot enforcement: The output schema requires a source_ref field on every claim object. Claims with null source_ref are automatically quarantined.

Pseudo-Algorithm: Retrieval-Grounded Generation Pipeline

ALGORITHM RetrievalGroundedGeneration(query, evidence_set, output_schema)
────────────────────────────────────────────────────────────────────────
INPUT:  query Q, evidence set E, output_schema S
OUTPUT: grounded_output O, groundedness_score G
 
1.  // Phase 1: Context compilation with evidence prioritization
2.  ranked_evidence ← RankByRelevance(E, Q)
3.  context ← CompileContext(
        role_policy = GROUNDING_POLICY,
        evidence = ranked_evidence,
        output_schema = S,
        constraints = ["Every claim must cite evidence by ID",
                       "Flag uncertain claims explicitly",
                       "Do not infer beyond evidence"]
    )
4.  ASSERT TokenCount(context) ≤ TOKEN_BUDGET
5.  
6.  // Phase 2: Constrained generation
7.  raw_output ← LLM.Generate(context, schema=S, temperature=LOW)
8.  O ← Parse(raw_output, S)
 
9.  // Phase 3: Groundedness verification
10. claims ← ExtractAtomicClaims(O)
11. G ← 0.0
12. ungrounded ← ∅
 
13. FOR EACH claim c IN claims DO
14.     best_support ← MAX over e IN E of Entailment(e, c)
15.     IF best_support < θ_ground THEN
16.         ungrounded ← ungrounded ∪ {c}
17.     END IF
18.     G ← G + best_support
19. END FOR
 
20. G ← G / |claims|
 
21. // Phase 4: Handle ungrounded claims
22. IF |ungrounded| > 0 THEN
23.     O ← RemoveOrFlagClaims(O, ungrounded, strategy=FLAG_AS_UNCERTAIN)
24.     IF |ungrounded| / |claims| > UNGROUNDED_THRESHOLD THEN
25.         TRIGGER regeneration with stricter evidence constraints
26.     END IF
27. END IF
 
28. RETURN (O, G)

20.3.2 Structured Output Enforcement: JSON Schema, Type Constraints, and Enum Restrictions#

Principle: Structural hallucinations are the most mechanically preventable category. By constraining the model's output to a strict schema, entire classes of invalid output are eliminated at the generation level.

Enforcement Layers:

Constrained Decoding Formalization: At each generation step $t$, the model produces a distribution over vocabulary $\mathcal{V}$:

Constrained decoding applies a mask $M_t \subseteq \mathcal{V}$ derived from the grammar/schema state:

This guarantees that the output is structurally valid by construction, with zero post-hoc structural hallucination.

Schema-Hardened Output Contract:

ClaimOutput {
  claim_id: UUID (required),
  statement: string (required, minLength=1, maxLength=500),
  claim_type: enum [FACTUAL, INFERENTIAL, PROCEDURAL, UNCERTAIN] (required),
  source_refs: array of EvidenceID (required, minItems=0),
  confidence: number (required, minimum=0.0, maximum=1.0),
  caveats: array of string (optional),
}

20.3.3 Chain-of-Verification: Decompose → Generate → Verify → Filter Pipelines#

Principle: Instead of generating a complete output and then checking it, interleave generation with verification at the sub-claim level. The Chain-of-Verification (CoVe) pattern decomposes the task, generates candidate outputs, constructs verification questions for each sub-claim, answers those questions independently, and filters the output based on verification results.

Formal Pipeline:

Stages:

1. Decompose: Extract atomic claims $\{c_1, \ldots, c_m\}$ from draft output $\mathcal{O}_{\text{draft}}$
2. Formulate: For each claim $c_j$, generate a verification question $q_j$ that, if answered correctly, would confirm or refute $c_j$
3. Verify: Answer each $q_j$ independently (using retrieval, tool calls, or a separate model call with isolated context)
4. Compare: Check whether the verification answer $a_j$ is consistent with the original claim $c_j$
5. Filter: Retain consistent claims, flag or remove inconsistent ones

Pseudo-Algorithm: Chain-of-Verification

ALGORITHM ChainOfVerification(query, context, evidence)
──────────────────────────────────────────────────────
INPUT:  query Q, context C, evidence E
OUTPUT: verified_output O_verified, verification_report V
 
1.  // Stage 1: Draft generation
2.  O_draft ← LLM.Generate(CompileContext(Q, C, E), temperature=STANDARD)
 
3.  // Stage 2: Claim decomposition
4.  claims ← DecomposeIntoClaims(O_draft)
5.  // Each claim: {id, text, type, position_in_output}
 
6.  // Stage 3: Verification question formulation
7.  verification_plan ← ∅
8.  FOR EACH claim c IN claims DO
9.      IF c.type IN {FACTUAL, INFERENTIAL} THEN
10.         vq ← FormulateVerificationQuestion(c)
11.         verification_plan ← verification_plan ∪ {(c, vq)}
12.     END IF
13. END FOR
 
14. // Stage 4: Independent verification (isolated context)
15. verification_results ← ∅
16. FOR EACH (c, vq) IN verification_plan DO
17.     // CRITICAL: Use isolated context — no access to O_draft
18.     v_context ← CompileContext(query=vq, evidence=E, exclude=O_draft)
19.     v_answer ← LLM.Generate(v_context, temperature=LOW)
20.     
21.     consistency ← CheckConsistency(c.text, v_answer)
22.     verification_results ← verification_results ∪ {(c, v_answer, consistency)}
23. END FOR
 
24. // Stage 5: Filtering
25. retained_claims ← ∅
26. flagged_claims ← ∅
27. FOR EACH (c, va, cons) IN verification_results DO
28.     IF cons ≥ θ_consistency THEN
29.         retained_claims ← retained_claims ∪ {c}
30.     ELSE
31.         flagged_claims ← flagged_claims ∪ {(c, va, cons)}
32.     END IF
33. END FOR
 
34. // Stage 6: Reconstruct output from retained claims
35. O_verified ← ReconstructOutput(O_draft, retained_claims, flagged_claims)
36. V ← VerificationReport(verification_results, pass_rate, flagged_claims)
 
37. RETURN (O_verified, V)

Computational Cost: CoVe incurs approximately $|\text{factual\_claims}| + 1$ LLM calls per output. Token cost:

This is typically $2\times$ to $5\times$ the cost of unverified generation. The cost is justified when hallucination has high downstream impact ($\text{sev}(c) > 0.5$).

Optimization: For cost-sensitive deployments, apply CoVe selectively:

• Only verify claims with low confidence or high severity
• Batch verification questions into a single LLM call with structured output
• Cache verification results for repeated or similar claims

20.3.4 Abstention Policies: "I Don't Know" Triggers, Confidence-Gated Responses#

Principle: A system that can reliably abstain when uncertain is safer than one that always generates an answer. Abstention is a first-class output type, not a failure mode.

Formal Abstention Policy: Define the abstention decision function:

where $\hat{P}(H)$ is the estimated hallucination probability and $\theta_H, \theta_G$ are configurable thresholds.

Abstention Triggers (conditions under which the system must abstain or escalate):

Confidence Estimation: The system estimates confidence through multiple signals:

where $G$ is groundedness, $\hat{\mathcal{H}}$ is normalized entropy, $\text{SC}$ is self-consistency score, and $\text{retrieval\_quality}$ measures evidence sufficiency.

Pseudo-Algorithm: Confidence-Gated Response

ALGORITHM ConfidenceGatedResponse(query, context, evidence)
──────────────────────────────────────────────────────────
INPUT:  query Q, context C, evidence E
OUTPUT: response R (RESPOND | ABSTAIN | PARTIAL_WITH_CAVEATS)
 
1.  // Phase 1: Pre-generation abstention checks
2.  IF |FilterRelevant(E, Q)| = 0 AND RequiresFactualAnswer(Q) THEN
3.      RETURN ABSTAIN(reason="no_relevant_evidence")
4.  END IF
5.  IF DomainMatch(Q) < θ_domain THEN
6.      RETURN ABSTAIN(reason="out_of_domain")
7.  END IF
 
8.  // Phase 2: Generate candidate output
9.  O ← LLM.Generate(CompileContext(Q, C, E), temperature=LOW)
 
10. // Phase 3: Post-generation confidence assessment
11. claims ← ExtractAtomicClaims(O)
12. claim_confidences ← ∅
 
13. FOR EACH claim c IN claims DO
14.     g_c ← MAX over e IN E of Entailment(e, c)
15.     claim_confidences ← claim_confidences ∪ {(c, g_c)}
16. END FOR
 
17. overall_conf ← ComputeOverallConfidence(claim_confidences, E, Q)
 
18. // Phase 4: Decision
19. IF overall_conf ≥ θ_high THEN
20.     RETURN RESPOND(O, confidence=overall_conf)
21. ELSE IF overall_conf ≥ θ_low THEN
22.     // Partial response with caveats on low-confidence claims
23.     caveated_output ← AddCaveats(O, claim_confidences, θ_claim)
24.     RETURN PARTIAL_WITH_CAVEATS(caveated_output, confidence=overall_conf)
25. ELSE
26.     RETURN ABSTAIN(
27.         reason="low_confidence",
28.         details=LowConfidenceClaims(claim_confidences),
29.         suggested_actions=["provide_more_context", "try_different_query"]
30.     )
31. END IF

Calibration Requirement: Abstention thresholds must be calibrated empirically. An overly aggressive abstention policy reduces utility; an overly permissive one allows hallucinations through. Calibration uses a held-out evaluation set:

where:

• $\text{FalseAbstentionRate}(\theta)$: fraction of correct answers that are suppressed
• $\text{HallucinationLeakRate}(\theta)$: fraction of hallucinated answers that pass through

20.4 Detection Mechanisms#

When prevention is insufficient — due to novel query types, edge cases, or inherently uncertain domains — detection mechanisms identify hallucinations before they propagate downstream.

20.4.1 Cross-Reference Verification Against Retrieved Evidence#

Mechanism: Decompose the output into atomic claims and verify each claim against the retrieved evidence using natural language inference (NLI).

Entailment Classification: For each claim-evidence pair $(c_j, e_i)$, classify the relationship:

Claim-Level Verdict:

Priority: CONTRADICTED takes precedence over ENTAILS (a single contradiction outweighs multiple supports, triggering investigation).

Pseudo-Algorithm: Cross-Reference Verification

ALGORITHM CrossReferenceVerification(output, evidence)
─────────────────────────────────────────────────────
INPUT:  output O, evidence set E
OUTPUT: verification_report VR
 
1.  claims ← ExtractAtomicClaims(O)
2.  VR ← VerificationReport()
 
3.  FOR EACH claim c IN claims DO
4.      entailments ← ∅
5.      contradictions ← ∅
6.      neutrals ← ∅
7.      
8.      FOR EACH evidence e IN E DO
9.          nli_result ← NLI_Model.Classify(premise=e.text, hypothesis=c.text)
10.         nli_score ← NLI_Model.Score(premise=e.text, hypothesis=c.text)
11.         
12.         IF nli_result = ENTAILS AND nli_score ≥ θ_entail THEN
13.             entailments ← entailments ∪ {(e, nli_score)}
14.         ELSE IF nli_result = CONTRADICTS AND nli_score ≥ θ_contradict THEN
15.             contradictions ← contradictions ∪ {(e, nli_score)}
16.         ELSE
17.             neutrals ← neutrals ∪ {(e, nli_score)}
18.         END IF
19.     END FOR
20.     
21.     IF |contradictions| > 0 THEN
22.         verdict ← CONTRADICTED
23.         supporting_evidence ← contradictions
24.     ELSE IF |entailments| > 0 THEN
25.         verdict ← SUPPORTED
26.         supporting_evidence ← entailments
27.     ELSE
28.         verdict ← UNSUPPORTED
29.         supporting_evidence ← ∅
30.     END IF
31.     
32.     VR.AddClaimResult(c, verdict, supporting_evidence)
33. END FOR
 
34. VR.ComputeSummary()
35. RETURN VR

20.4.2 Self-Consistency Checking: Multiple Generations, Temperature Sampling, Majority Vote#

Principle: If a model truly "knows" the answer, it should produce consistent answers across multiple independent generation attempts. Inconsistency signals uncertainty, which correlates with hallucination risk.

Formal Definition: Generate $k$ independent outputs $\{\mathcal{O}_1, \ldots, \mathcal{O}_k\}$ for the same query $q$ with context $\mathcal{C}$, using non-zero temperature to induce variation:

Self-Consistency Score: Measure pairwise agreement across generations:

where $\text{agree}(\mathcal{O}_i, \mathcal{O}_j) \in [0, 1]$ measures semantic equivalence.

For structured outputs, agreement can be computed per field:

Majority Vote Selection: For each atomic claim or field, select the value that appears most frequently:

Hallucination Signal: Claims where no majority exists ($\max_v \text{count}(v) < \lceil k/2 \rceil$) are high-risk and should be flagged.

Pseudo-Algorithm: Self-Consistency Check

ALGORITHM SelfConsistencyCheck(query, context, k, temperature)
─────────────────────────────────────────────────────────────
INPUT:  query Q, context C, num_samples k, temperature T
OUTPUT: consensus_output O*, consistency_report CR
 
1.  outputs ← ∅
2.  FOR i FROM 1 TO k DO
3.      O_i ← LLM.Generate(CompileContext(Q, C), temperature=T, seed=random())
4.      outputs ← outputs ∪ {O_i}
5.  END FOR
 
6.  // Decompose each output into atomic claims
7.  all_claim_sets ← [ExtractAtomicClaims(O_i) FOR O_i IN outputs]
 
8.  // Cluster semantically equivalent claims across outputs
9.  claim_clusters ← ClusterBySemantic(all_claim_sets, similarity_threshold=0.85)
 
10. // Compute per-claim consistency
11. claim_verdicts ← ∅
12. FOR EACH cluster IN claim_clusters DO
13.     frequency ← |cluster.members| / k
14.     IF frequency ≥ MAJORITY_THRESHOLD THEN
15.         verdict ← CONSISTENT
16.         representative ← SelectMostCommonFormulation(cluster)
17.     ELSE IF frequency ≥ MINORITY_THRESHOLD THEN
18.         verdict ← UNCERTAIN
19.         representative ← SelectMostCommonFormulation(cluster)
20.     ELSE
21.         verdict ← INCONSISTENT
22.         representative ← NULL
23.     END IF
24.     claim_verdicts ← claim_verdicts ∪ {(cluster, verdict, frequency, representative)}
25. END FOR
 
26. // Assemble consensus output from CONSISTENT claims
27. O* ← AssembleFromClusters(
28.     include=[cv FOR cv IN claim_verdicts WHERE cv.verdict = CONSISTENT],
29.     flag=[cv FOR cv IN claim_verdicts WHERE cv.verdict = UNCERTAIN],
30.     exclude=[cv FOR cv IN claim_verdicts WHERE cv.verdict = INCONSISTENT]
31. )
 
32. CR ← ConsistencyReport(claim_verdicts, overall_SC=Mean(frequencies))
33. RETURN (O*, CR)

Cost-Consistency Trade-off: The token cost scales linearly with $k$:

Typical values: $k \in \{3, 5, 7\}$. Higher $k$ improves detection reliability but increases cost. An adaptive strategy:

20.4.3 Entailment-Based Fact Checking: NLI Models for Claim-Evidence Alignment#

Mechanism: Deploy a dedicated Natural Language Inference (NLI) model as a verification oracle. The NLI model is architecturally distinct from the generation model, providing an independent verification signal.

NLI Model Specification:

where $\text{label} \in \{\text{ENTAILMENT}, \text{CONTRADICTION}, \text{NEUTRAL}\}$ and $\text{confidence} \in [0, 1]$.

Claim-Level Faithfulness Score: For claim $c_j$ against evidence set $\mathcal{E}$:

Output-Level Faithfulness: Aggregate across all claims:

Contradiction Detection: A claim is definitively hallucinated if:

This is a stronger signal than mere lack of support (NEUTRAL), because it identifies active conflicts between the output and the evidence.

Multi-Granularity Entailment: Check entailment at multiple granularities:

Pseudo-Algorithm: Multi-Granularity Entailment Check

ALGORITHM MultiGranularityEntailment(output, evidence, granularities)
────────────────────────────────────────────────────────────────────
INPUT:  output O, evidence E, granularities G = [SENTENCE, PASSAGE, DOCUMENT]
OUTPUT: entailment_report ER
 
1.  ER ← EntailmentReport()
 
2.  FOR EACH granularity g IN G DO
3.      IF g = SENTENCE THEN
4.          claims ← SentenceTokenize(O)
5.          premises ← SentenceTokenize(CONCAT(E))
6.      ELSE IF g = PASSAGE THEN
7.          claims ← ParagraphSegment(O)
8.          premises ← [e.text FOR e IN E]
9.      ELSE  // DOCUMENT
10.         claims ← [O]
11.         premises ← [CONCAT(E)]
12.     END IF
13.     
14.     FOR EACH claim c IN claims DO
15.         scores ← ∅
16.         FOR EACH premise p IN premises DO
17.             (label, conf) ← NLI_Model(premise=p, hypothesis=c)
18.             scores ← scores ∪ {(p, label, conf)}
19.         END FOR
20.         
21.         best_entail ← MAX conf WHERE label=ENTAILMENT IN scores
22.         worst_contra ← MAX conf WHERE label=CONTRADICTION IN scores
23.         
24.         ER.Add(granularity=g, claim=c, 
25.                entailment_score=best_entail,
26.                contradiction_score=worst_contra,
27.                verdict=ComputeVerdict(best_entail, worst_contra))
28.     END FOR
29. END FOR
 
30. ER.ComputeAggregates()
31. RETURN ER

20.4.4 External Knowledge Base Verification: Real-Time Fact Checking Against Authoritative Sources#

Mechanism: For factual claims that cannot be verified against the retrieved context (either because the context is insufficient or the claim is about general knowledge), query authoritative external knowledge bases in real time.

Knowledge Base Hierarchy (ordered by authority):

Verification Decision: For each unverified claim $c$, determine whether external verification is warranted:

where $\text{verifiable}(c)$ indicates the claim is about an objectively verifiable fact (not an opinion or inference).

Pseudo-Algorithm: External Verification Pipeline

ALGORITHM ExternalVerification(claim, source_hierarchy, latency_budget)
────────────────────────────────────────────────────────────────────
INPUT:  claim C, source_hierarchy S[], latency_budget L
OUTPUT: external_verdict EV
 
1.  // Formulate verification query from claim
2.  vq ← FormulateVerificationQuery(C)
3.  
4.  // Query sources in priority order with latency control
5.  elapsed ← 0
6.  FOR EACH source s IN S (ordered by priority) DO
7.      IF elapsed + EstimatedLatency(s) > L THEN
8.          BREAK  // Budget exhausted
9.      END IF
10.     
11.     results ← QuerySource(s, vq, deadline=L - elapsed)
12.     elapsed ← elapsed + ActualLatency(results)
13.     
14.     IF results.found THEN
15.         // Compare claim against retrieved authoritative information
16.         agreement ← CompareClaim(C, results.data)
17.         IF agreement.confidence ≥ θ_external THEN
18.             EV ← ExternalVerdict(
19.                 status = IF agreement.consistent THEN VERIFIED ELSE REFUTED,
20.                 source = s,
21.                 evidence = results.data,
22.                 confidence = agreement.confidence
23.             )
24.             RETURN EV
25.         END IF
26.     END IF
27. END FOR
 
28. // No authoritative source could verify or refute
29. RETURN ExternalVerdict(status=UNVERIFIABLE, reason="no_authoritative_source")

20.5 Mitigation Strategies#

When hallucinations are detected — whether by prevention mechanisms, detection pipelines, or downstream verification — the system must mitigate their impact without discarding the entire output.

20.5.1 Targeted Regeneration with Corrective Context Injection#

Principle: Rather than regenerating the entire output (wasteful and potentially introduces new hallucinations), surgically replace only the hallucinated claims by injecting the detection results as corrective context.

Formal Approach: Given output $\mathcal{O}$ with detected hallucinations $H_{\text{detected}} = \{c_{h_1}, c_{h_2}, \ldots\}$, construct a corrective context:

where $\text{FeedbackSignals}$ includes:

• The specific claims identified as hallucinated
• The evidence that contradicts each claim (if available)
• The verification verdict for each claim
• Explicit instructions to correct only the flagged claims

Pseudo-Algorithm: Targeted Regeneration

ALGORITHM TargetedRegeneration(output, hallucinated_claims, evidence, max_attempts)
──────────────────────────────────────────────────────────────────────────────────
INPUT:  output O, hallucinated_claims H[], evidence E, max_attempts M
OUTPUT: corrected_output O', correction_report CR
 
1.  O' ← O
2.  attempt ← 0
 
3.  WHILE |H| > 0 AND attempt < M DO
4.      attempt ← attempt + 1
5.      
6.      // Construct corrective context
7.      corrections_needed ← ∅
8.      FOR EACH claim c IN H DO
9.          contradicting_evidence ← FindContradictions(c, E)
10.         corrections_needed ← corrections_needed ∪ {
11.             claim_text: c.text,
12.             issue: c.verdict,
13.             contradicting_evidence: contradicting_evidence,
14.             instruction: "Replace this claim with a factually correct, 
15.                           evidence-supported alternative or remove it"
16.         }
17.     END FOR
18.     
19.     // Targeted regeneration prompt
20.     regen_context ← CompileContext(
21.         role = CORRECTION_POLICY,
22.         original_output = O',
23.         corrections_needed = corrections_needed,
24.         evidence = E,
25.         instruction = "Correct ONLY the flagged claims. Preserve all other content."
26.     )
27.     
28.     O' ← LLM.Generate(regen_context, temperature=LOW)
29.     
30.     // Re-verify corrected output
31.     new_claims ← ExtractAtomicClaims(O')
32.     new_verification ← CrossReferenceVerification(O', E)
33.     H ← [c FOR c IN new_verification.claims WHERE c.verdict IN {CONTRADICTED, UNSUPPORTED}]
34.     
35.     // Track convergence
36.     IF |H| ≥ |previous_H| THEN
37.         // Not converging — break and escalate
38.         BREAK
39.     END IF
40.     previous_H ← H
41. END WHILE
 
42. CR ← CorrectionReport(attempts=attempt, remaining_issues=H, changes_made=Diff(O, O'))
43. IF |H| > 0 THEN CR.escalation_required ← TRUE END IF
44. RETURN (O', CR)

Convergence Guarantee: The algorithm terminates in at most $M$ attempts. If hallucinations are not resolved within $M$ attempts, the system escalates to human review rather than looping indefinitely.

20.5.2 Citation Enforcement: Every Claim Linked to Source, No Anonymous Assertions#

Principle: Mandatory citation enforcement transforms hallucination from a detection problem into a visibility problem. If every claim must cite its source, unsupported claims become immediately visible — both to automated verification and to human reviewers.

Citation Schema:

CitedClaim {
  claim_text: string (required),
  citations: array of Citation (required, minItems=1),
  claim_confidence: number (required, minimum=0, maximum=1),
}
 
Citation {
  source_id: EvidenceID (required),
  source_text: string (required),    // The specific passage cited
  relationship: enum [SUPPORTS, DERIVED_FROM, BASED_ON] (required),
  page_or_location: string (optional),
}

Enforcement Levels:

Production agentic systems must operate at Level 3 or 4.

Attribution Verification:

Pseudo-Algorithm: Citation Enforcement and Verification

ALGORITHM CitationEnforcement(output, evidence_index)
───────────────────────────────────────────────────
INPUT:  output O (with citation schema), evidence_index EI
OUTPUT: verified_output O', citation_report CR
 
1.  claims ← ExtractCitedClaims(O)
2.  CR ← CitationReport()
 
3.  FOR EACH claim c IN claims DO
4.      // Check citation existence
5.      IF c.citations = ∅ THEN
6.          CR.AddViolation(c, "missing_citation")
7.          CONTINUE
8.      END IF
9.      
10.     citation_valid ← FALSE
11.     FOR EACH cit IN c.citations DO
12.         // Verify source exists
13.         IF NOT EI.Exists(cit.source_id) THEN
14.             CR.AddViolation(c, "nonexistent_source", cit)
15.             CONTINUE
16.         END IF
17.         
18.         // Verify cited text matches actual source
19.         actual_text ← EI.GetText(cit.source_id, cit.page_or_location)
20.         text_match ← SimilarityScore(cit.source_text, actual_text)
21.         IF text_match < θ_text_match THEN
22.             CR.AddViolation(c, "misquoted_source", cit, actual_text)
23.             CONTINUE
24.         END IF
25.         
26.         // Verify entailment
27.         entailment ← NLI_Model(premise=actual_text, hypothesis=c.claim_text)
28.         IF entailment.label = ENTAILS AND entailment.confidence ≥ θ_entail THEN
29.             citation_valid ← TRUE
30.             CR.AddVerified(c, cit, entailment.confidence)
31.         ELSE
32.             CR.AddViolation(c, "non_entailing_citation", cit, entailment)
33.         END IF
34.     END FOR
35.     
36.     IF NOT citation_valid THEN
37.         CR.FlagClaim(c, "no_valid_citation")
38.     END IF
39. END FOR
 
40. // Remove or flag claims without valid citations
41. O' ← FilterOutput(O, CR, strategy=FLAG_INVALID)
42. RETURN (O', CR)

20.5.3 Human Review Escalation for High-Stakes or Low-Confidence Outputs#

Principle: For outputs with high downstream impact or persistent low confidence, the system must escalate to human review rather than committing potentially hallucinated content. Escalation is not a failure — it is the system operating within its designed reliability envelope.

Escalation Decision Function:

This multiplicative formulation ensures that high-stakes outputs require proportionally higher confidence to avoid escalation, while low-stakes outputs can proceed with lower confidence.

Escalation Tiers:

Escalation Package: The escalation must provide the reviewer with:

EscalationPackage {
  output: FullOutput,
  flagged_claims: FlaggedClaim[],        // With specific issues
  verification_report: VerificationReport,
  evidence_used: Evidence[],
  confidence_breakdown: ConfidenceBreakdown,
  suggested_corrections: Suggestion[],    // Model's best-effort fixes
  task_context: TaskContext,             // Why this output was generated
  deadline: Timestamp,                   // When the review is needed
}

20.6 Hallucination Metrics: Faithfulness Score, Attribution Precision, and Factual Accuracy Rate#

20.6.1 Core Metrics Framework#

A production hallucination monitoring system requires formally defined, continuously computable metrics. These metrics serve as quality gates, regression alerts, and optimization targets.

Metric Taxonomy:

20.6.2 Faithfulness Score#

Definition: The degree to which every claim in the output is entailed by the provided context.

where $N = |\mathcal{O}|$ is the number of atomic claims, and $P(\text{ENTAILS})$ is computed by an NLI model.

Properties:

• Range: $[0, 1]$
• Computable without external knowledge base (uses only provided context)
• Does not detect extrinsic hallucinations where the context itself is wrong
• Sensitive to claim extraction quality

Weighted Faithfulness (incorporating claim severity):

where $w_j = \text{sev}(c_j)$ weights more critical claims higher.

20.6.3 Attribution Precision#

Definition: The fraction of cited claims whose citations are valid (source exists, cited text matches, and entailment holds).

Decomposition into sub-metrics:

Overall: $\text{AP} = \text{AP}_{\text{existence}} \cdot \text{AP}_{\text{accuracy}} \cdot \text{AP}_{\text{entailment}}$

20.6.4 Factual Accuracy Rate#

Definition: The fraction of verifiable factual claims that are correct according to an authoritative reference.

where $\mathcal{O}_{\text{factual}} \subseteq \mathcal{O}$ is the subset of claims that are objectively verifiable.

Limitation: FAR requires access to a ground-truth knowledge base $\mathcal{K}$, which is not always available at runtime. FAR is therefore primarily an evaluation metric (computed on benchmarks) rather than a runtime metric.

20.6.5 Composite Hallucination Score#

Combine individual metrics into a single composite score for dashboard reporting and alerting:

where $w_F + w_A + w_G + w_S = 1$, and $F, \text{AP}, G, \text{SC}$ are faithfulness, attribution precision, groundedness, and self-consistency respectively. A score of 0 indicates no detected hallucination; a score approaching 1 indicates severe hallucination.

Quality Gate: The system defines a maximum acceptable hallucination score:

Outputs exceeding $H_{\max}$ are rejected and routed to mitigation.

20.6.6 Metrics Computation Pipeline#

ALGORITHM ComputeHallucinationMetrics(output, context, evidence, config)
───────────────────────────────────────────────────────────────────────
INPUT:  output O, context C, evidence E, config (weights, thresholds)
OUTPUT: HallucinationMetrics HM
 
1.  // Extract claims
2.  claims ← ExtractAtomicClaims(O)
3.  factual_claims ← FilterFactualClaims(claims)
4.  cited_claims ← FilterCitedClaims(claims)
 
5.  // Faithfulness
6.  faith_scores ← ∅
7.  FOR EACH c IN claims DO
8.      f_c ← MAX over e IN C of NLI_Entailment(e, c)
9.      faith_scores ← faith_scores ∪ {(c, f_c)}
10. END FOR
11. F ← Mean([f FOR (_, f) IN faith_scores])
 
12. // Attribution Precision
13. valid_citations ← 0; total_cited ← |cited_claims|
14. FOR EACH c IN cited_claims DO
15.     IF VerifyCitation(c, E) THEN valid_citations ← valid_citations + 1 END IF
16. END FOR
17. AP ← IF total_cited > 0 THEN valid_citations / total_cited ELSE 1.0
 
18. // Groundedness
19. grounded ← |{c ∈ claims : MAX NLI(E, c) ≥ θ_ground}|
20. G ← grounded / |claims|
 
21. // Self-Consistency (if multiple samples available)
22. SC ← IF config.self_consistency_samples > 1 THEN
23.         ComputeSelfConsistency(O, config.samples)
24.      ELSE 1.0  // Assume consistent if not checked
 
25. // Composite
26. HS ← 1 - (w_F·F + w_A·AP + w_G·G + w_S·SC)
 
27. HM ← HallucinationMetrics {
28.     faithfulness = F,
29.     attribution_precision = AP,
30.     groundedness = G,
31.     self_consistency = SC,
32.     composite_score = HS,
33.     per_claim_scores = faith_scores,
34.     quality_gate_pass = (HS ≤ H_max)
35. }
36. RETURN HM

20.7 Continuous Hallucination Monitoring in Production: Drift Detection and Regression Alerting#

20.7.1 Monitoring Architecture#

Production hallucination monitoring requires continuous, automated evaluation of a representative sample of system outputs. The monitoring system operates as an independent service that consumes output traces and produces hallucination assessments without blocking the primary execution path.

Monitoring Pipeline:

Sampling Strategy: Evaluating every output is cost-prohibitive (each evaluation involves NLI model calls and potentially external verification). Use stratified sampling:

where $p_{\text{medium}} \approx 0.1$ and $p_{\text{low}} \approx 0.01$ are configurable.

20.7.2 Drift Detection#

Hallucination rates may change over time due to model updates, data distribution shifts, retrieval index degradation, or prompt drift. The monitoring system must detect statistically significant increases in hallucination rate.

Statistical Process Control: Track the hallucination score as a time series $\{H_t\}$ and detect shifts using CUSUM (Cumulative Sum) control charts:

where $\mu_0$ is the baseline mean hallucination score, $\delta$ is the allowable slack, and an alarm triggers when:

for threshold $h$. The parameters $\delta$ and $h$ control the trade-off between detection sensitivity and false alarm rate.

Average Run Length (ARL): The expected number of samples before detecting a true shift of magnitude $\Delta$:

The monitoring system is configured to achieve $\text{ARL}_1 \leq T_{\text{detect}}$ for the minimum actionable shift $\Delta_{\min}$, while maintaining $\text{ARL}_0 \geq T_{\text{false}}$ under no-shift conditions.

Pseudo-Algorithm: Continuous Hallucination Monitor

ALGORITHM ContinuousHallucinationMonitor(output_stream, config)
─────────────────────────────────────────────────────────────
INPUT:  output_stream S (continuous), config (baseline, thresholds)
OUTPUT: continuous alerts
 
1.  μ_0 ← config.baseline_hallucination_rate
2.  δ ← config.cusum_slack
3.  h ← config.cusum_threshold
4.  S_plus ← 0    // Upper CUSUM statistic (detect increase)
5.  S_minus ← 0   // Lower CUSUM statistic (detect decrease, for completeness)
6.  window ← RollingWindow(size=config.window_size)
 
7.  FOR EACH output O IN S DO
8.      // Sampling decision
9.      IF Random() > SampleRate(O.stakes) THEN CONTINUE END IF
10.     
11.     // Compute hallucination metrics
12.     HM ← ComputeHallucinationMetrics(O, O.context, O.evidence, config)
13.     
14.     // Record to time series
15.     RecordMetric("hallucination.faithfulness", HM.faithfulness, timestamp=Now())
16.     RecordMetric("hallucination.attribution", HM.attribution_precision, timestamp=Now())
17.     RecordMetric("hallucination.composite", HM.composite_score, timestamp=Now())
18.     
19.     // Update rolling window
20.     window.Add(HM.composite_score)
21.     
22.     // CUSUM update
23.     S_plus ← MAX(0, S_plus + (HM.composite_score - μ_0 - δ))
24.     
25.     // Alert check
26.     IF S_plus > h THEN
27.         EMIT Alert(
28.             type = "hallucination_rate_increase",
29.             current_rate = window.Mean(),
30.             baseline_rate = μ_0,
31.             cusum_statistic = S_plus,
32.             recent_examples = window.WorstK(5),
33.             recommended_actions = [
34.                 "Inspect retrieval quality",
35.                 "Check for prompt drift",
36.                 "Verify model version",
37.                 "Review evidence index freshness"
38.             ]
39.         )
40.         // Reset after alert (or maintain for sustained alerts)
41.         S_plus ← 0
42.     END IF
43.     
44.     // Per-output quality gate
45.     IF NOT HM.quality_gate_pass THEN
46.         EMIT OutputAlert(O.task_id, HM, severity=ComputeSeverity(HM, O.stakes))
47.     END IF
48. END FOR

20.7.3 Regression Alerting#

Beyond drift detection (gradual change), the system must detect regressions — sudden, discrete increases in hallucination rate caused by code deployments, model swaps, or configuration changes.

Change-Point Detection: Associate hallucination rate changes with system change events:

where $\bar{H}_{[a,b]}$ is the mean hallucination score over interval $[a, b]$ and $w$ is the evaluation window.

Change Event Correlation: The monitoring system maintains a log of system change events (model deployments, prompt updates, retrieval index rebuilds, tool schema changes) and automatically correlates hallucination rate changes with the nearest preceding change event.

Automated Bisection: When a regression is detected, the system can automatically bisect recent changes to identify the causal change:

ALGORITHM RegressionBisection(change_log, regression_time, eval_set)
─────────────────────────────────────────────────────────────────
INPUT:  change_log CL, regression_time T_r, eval_set ES
OUTPUT: causal_change CC
 
1.  // Identify candidate changes
2.  candidates ← [c IN CL WHERE c.timestamp IN [T_r - Δ, T_r]]
3.  
4.  IF |candidates| = 1 THEN RETURN candidates[0] END IF
5.  
6.  // Binary search through change sequence
7.  lo ← 0; hi ← |candidates| - 1
8.  WHILE lo < hi DO
9.      mid ← (lo + hi) / 2
10.     
11.     // Deploy system at state after candidates[mid]
12.     system_mid ← DeployAtState(candidates[mid].resulting_state)
13.     score_mid ← EvaluateHallucination(system_mid, ES)
14.     
15.     IF score_mid > θ_regression THEN
16.         hi ← mid      // Regression present at this point
17.     ELSE
18.         lo ← mid + 1  // Regression not yet introduced
19.     END IF
20. END WHILE
21. 
22. CC ← candidates[lo]
23. RETURN CC

20.7.4 Metric Dashboards#

The production monitoring dashboard exposes:

20.8 Adversarial Hallucination Testing: Red Team Prompts, Edge Cases, and Boundary Probing#

20.8.1 Adversarial Testing Philosophy#

A system that passes only benign test cases provides insufficient hallucination guarantees. Adversarial testing systematically probes the boundaries of the system's reliability by crafting inputs designed to maximize hallucination probability. This is the hallucination equivalent of security penetration testing.

Objective: Identify the system's hallucination frontier — the boundary in input space beyond which the system cannot maintain its faithfulness guarantees.

Formal Objective: Find inputs $q^*$ that maximize hallucination score subject to being plausible user queries:

20.8.2 Attack Taxonomy#

20.8.3 Red Team Prompt Generation#

Systematic Generation: Rather than relying on human creativity alone, generate adversarial prompts programmatically:

Pseudo-Algorithm: Adversarial Prompt Generation

ALGORITHM GenerateAdversarialPrompts(attack_taxonomy, domain, evidence_index)
────────────────────────────────────────────────────────────────────────────
INPUT:  attack_taxonomy AT, domain D, evidence_index EI
OUTPUT: adversarial_prompt_set APS
 
1.  APS ← ∅
 
2.  // Category 1: Knowledge boundary probing
3.  FOR EACH entity e IN SampleEntities(EI, tier=LONG_TAIL, n=50) DO
4.      q ← TemplateQuery("Describe the detailed history of {entity}", e)
5.      APS ← APS ∪ {AdversarialPrompt(q, category=KNOWLEDGE_BOUNDARY, 
6.                     expected_behavior=ABSTAIN_OR_CAVEAT)}
7.  END FOR
 
8.  // Category 2: Entity substitution
9.  FOR EACH (known_entity, obscure_variant) IN EntitySubstitutionPairs(D, n=30) DO
10.     q_known ← "What is {known_entity}'s primary function?"
11.     q_adversarial ← "What is {obscure_variant}'s primary function?"
12.     APS ← APS ∪ {AdversarialPrompt(q_adversarial, category=ENTITY_SUBSTITUTION,
13.                    baseline_query=q_known,
14.                    expected_behavior=IF_EXISTS_ANSWER_ELSE_ABSTAIN)}
15. END FOR
 
16. // Category 3: Context poisoning
17. FOR EACH sample IN SampleQueries(D, n=20) DO
18.     correct_evidence ← Retrieve(EI, sample.query)
19.     poisoned_evidence ← InjectContradiction(correct_evidence)
20.     APS ← APS ∪ {AdversarialPrompt(sample.query, 
21.                    context=poisoned_evidence,
22.                    category=CONTEXT_POISONING,
23.                    expected_behavior=DETECT_CONTRADICTION)}
24. END FOR
 
25. // Category 4: Constraint contradiction
26. FOR EACH constraint_set IN GenerateContradictoryConstraints(D, n=15) DO
27.     q ← FormulateQuery(constraint_set)
28.     APS ← APS ∪ {AdversarialPrompt(q, category=CONSTRAINT_CONTRADICTION,
29.                    expected_behavior=REPORT_CONTRADICTION)}
30. END FOR
 
31. // Category 5: Scale stress (lost in the middle)
32. FOR EACH sample IN SampleQueries(D, n=10) DO
33.     critical_evidence ← Retrieve(EI, sample.query, top_k=1)
34.     padding ← GenerateIrrelevantContent(token_count=LARGE)
35.     buried_context ← padding[:len/2] + critical_evidence + padding[len/2:]
36.     APS ← APS ∪ {AdversarialPrompt(sample.query,
37.                    context=buried_context,
38.                    category=SCALE_STRESS,
39.                    expected_behavior=FIND_AND_USE_EVIDENCE)}
40. END FOR
 
41. // Category 6: Compositional reasoning chains
42. FOR chain_length IN [3, 5, 7, 10] DO
43.     FOR i FROM 1 TO 5 DO
44.         chain ← GenerateReasoningChain(D, length=chain_length)
45.         q ← FormulateChainQuery(chain)
46.         APS ← APS ∪ {AdversarialPrompt(q, category=COMPOSITIONAL,
47.                        chain_length=chain_length,
48.                        expected_behavior=CORRECT_CHAIN_RESULT,
49.                        ground_truth=chain.final_answer)}
50.     END FOR
51. END FOR
 
52. RETURN APS

20.8.4 Evaluation Framework for Adversarial Tests#

Scoring Dimensions per Adversarial Prompt:

Adversarial Robustness Score: For the full adversarial test suite $\mathcal{T}_{\text{adv}} = \{(q_i, \text{expected}_i)\}$:

where $\text{behavior\_match} \in [0, 1]$ measures how closely the system's actual behavior matches the expected behavior.

Category-Level Analysis: Report ARS per attack category to identify specific vulnerabilities:

20.8.5 Continuous Adversarial Testing Pipeline#

Adversarial tests must execute continuously, not as one-time assessments:

ALGORITHM ContinuousAdversarialPipeline(system, schedule)
────────────────────────────────────────────────────────
INPUT:  system S, schedule (frequency, scope)
OUTPUT: continuous adversarial assessment reports
 
1.  // Static adversarial suite (curated, version-controlled)
2.  static_suite ← LoadAdversarialSuite(version=CURRENT)
 
3.  // Dynamic adversarial generation (evolves with system changes)
4.  EVERY schedule.frequency DO
5.      // Generate new adversarial prompts targeting recent changes
6.      recent_changes ← GetRecentSystemChanges()
7.      dynamic_suite ← GenerateTargetedAdversarial(recent_changes)
8.      
9.      // Combine suites
10.     full_suite ← static_suite ∪ dynamic_suite
11.     
12.     // Execute
13.     results ← ∅
14.     FOR EACH (prompt, expected) IN full_suite DO
15.         actual_output ← S.Execute(prompt)
16.         hallucination_metrics ← ComputeHallucinationMetrics(actual_output, prompt)
17.         behavior_score ← EvaluateBehavior(actual_output, expected)
18.         results ← results ∪ {(prompt, actual_output, hallucination_metrics, behavior_score)}
19.     END FOR
20.     
21.     // Compute scores
22.     ARS ← ComputeARS(results)
23.     ARS_by_category ← ComputeCategoryARS(results)
24.     
25.     // Regression detection
26.     IF ARS < ARS_baseline - Δ_regression THEN
27.         EMIT RegressionAlert(
28.             current_ARS = ARS,
29.             baseline_ARS = ARS_baseline,
30.             degraded_categories = [c FOR c IN ARS_by_category WHERE c.score < c.baseline - Δ],
31.             failing_prompts = [r FOR r IN results WHERE r.behavior_score < θ_fail]
32.         )
33.     END IF
34.     
35.     // Promote new adversarial discoveries to static suite
36.     new_failures ← [r FOR r IN results 
37.                     WHERE r.behavior_score < θ_fail 
38.                     AND r.prompt IN dynamic_suite]
39.     IF |new_failures| > 0 THEN
40.         ProposeAddToStaticSuite(new_failures)  // Human review before addition
41.     END IF
42.     
43.     // Report
44.     PublishReport(ARS, ARS_by_category, results, timestamp=Now())
45. END EVERY

20.8.6 Hallucination Error Budget#

Analogous to SRE error budgets, define a hallucination error budget that quantifies the acceptable hallucination rate over a time window:

where $|\mathcal{O}_T|$ is the number of outputs in period $T$ and $H_{\max}$ is the maximum acceptable per-output hallucination score.

When $\text{HEB}(T) \leq 0$, the hallucination budget is exhausted, triggering:

1. Deployment freeze: No changes that could increase hallucination risk
2. Increased sampling rate: Monitor more outputs to get tighter estimates
3. Root cause investigation: Mandatory analysis of worst-scoring outputs
4. Threshold tightening: Reduce $\theta_H$ in abstention policies to filter more aggressively
5. Escalation increase: Route more outputs to human review

The hallucination error budget creates an organizational feedback mechanism that directly couples agent quality to deployment velocity.

Summary: Hallucination Control as an Architectural Invariant#

The fundamental architectural invariant:

No output may be committed to downstream systems, memory stores, or external consumers without passing through the hallucination quality gate. This invariant is enforced mechanically by the orchestration runtime, not by prompt instructions.

Key Equations Reference:

This chapter establishes hallucination control as a multi-layered, mechanically enforced architectural discipline — not a prompt engineering afterthought. The taxonomy, root cause analysis, prevention pipelines, detection mechanisms, mitigation protocols, formal metrics, continuous monitoring infrastructure, and adversarial testing framework together form a complete system for ensuring that agentic AI outputs meet the faithfulness, accuracy, and attribution standards required for production deployment at enterprise scale.