Why Multi-Agent Consensus Collapses into Hallucination

A common architectural pattern in autonomous agent research assumes that if a single large language model hallucinates, an ensemble of debating agents will catch the error. The architecture pairs a generator with one or more critics, verifiers, or peer debaters. Over multiple rounds of structured dialogue, the agents critique intermediate reasoning steps and converge on a verified consensus.

In production environments, this assumption regularly fails. When debate loops run without deterministic, non-LLM grounding oracles, iterative critique does not reliably filter out errors. Instead, the ensemble frequently converges on plausible, mutually reinforced falsehoods—a failure state we term consensus collapse.

Ungrounded Multi-Agent Loop:
[Agent A: Proposes Premise P] ──► [Agent B: Critiques P]
         ▲                                   │
         │                                   ▼
[Agent A: Yields or Elaborates P'] ◄── [Agent B: Confirms Compromise]
         │
         ▼
[Consensus Reached on Shared Falsehood (Collapse)]

---

The Mechanics of Consensus Collapse

Consensus collapse is not an edge-case software bug; it is a structural consequence of how autoregressive language models process feedback in conversational context.

Three distinct mechanisms drive this degradation:

1. In-Context Sycophancy and Social Pressure

LLMs trained via Reinforcement Learning from Human Feedback (RLHF) exhibit a documented bias toward conversational agreeableness. When Agent B challenges Agent A with a confidently phrased critique—even one based on flawed reasoning—Agent A's probability distribution shifts toward conceding or accommodating the critique rather than defending its valid initial premise.

Once an erroneous statement enters the shared context window, both models condition all future token generation on its existence. The error transforms from an isolated hypothesis into an established conversational premise.

2. The Persuasion-Accuracy Asymmetry

In an ungrounded natural language debate, correctness does not correlate monotonically with persuasive rhetoric. A generated argument that is linguistically fluent, detailed, and structurally authoritative often overrides a concise, correct observation. If a verifier agent lacks access to ground-truth execution, it evaluates claims on stylistic plausibility and token co-occurrence rather than factual validity.

3. Autoregressive Error Compounding

In our prior analysis of stateful multi-agent loops, we showed that multi-turn agent interactions carry cumulative latency and state baggage. In consensus loops, they also carry cumulative epistemic debt.

Let the probability that an individual agent produces an undetected subtle error at step $t$ be $\epsilon$. If the critique protocol does not possess independent external verification, the probability that the error persists across $k$ debate rounds is not $\epsilon^k$ (independent trials), but rather approaches:

$$P(\text{Persistence}) \approx \epsilon \cdot \prod_{i=1}^{k-1} (1 - \delta_i)$$

where $\delta_i$ represents the verifier's marginal detection sensitivity. When $\delta_i$ drops as the conversation context fills with rationalizations, the likelihood of self-correction decays to zero.

---

Empirical Comparison: Ungrounded Consensus vs. Grounded Verification

To observe how consensus behaves under ambiguity, consider an ensemble tasked with debugging a complex distributed lock lease timeout.

Architecture Pattern	Rounds	Final Outcome	Failure Mode
Ungrounded Peer Debate (2 Models, 3 Turns)	3	Consensus on incorrect lock expiration offset	Consensus Collapse: Critic introduced a false assumption; Generator adopted it.
Majority Voting (5 Independent Paths)	1	3/5 Correct, 2/5 Incorrect	High Token Cost: Preserved accuracy because trajectories remained strictly isolated.
Adversarial Red Team (Ungrounded Critic)	4	Oscillating disagreement; timeout halt	Livelock: Models alternated mutually exclusive claims without resolution.
Deterministic Grounded Verifier (Sandboxed Runner)	2	Verified correct patch	Fast Convergence: Critic backed by reproducible test harness rejected false hypotheses instantly.

As shown in the table, increasing the number of ungrounded debate rounds did not improve accuracy; it merely increased token expenditure while hardening the models' commitment to the hallucinated premise. This reflects the broader finding that most agent failures never throw an error: the system exits cleanly with a 200 OK status code and a unanimous vote, yet the output is categorically wrong.

---

Why Independent Sampling Outperforms Sequential Debate

When teams implement multi-agent consensus, their implicit mental model is often the Condorcet Jury Theorem: if each independent voter has a probability $p > 0.5$ of being correct, the probability of a correct majority decision approaches 1 as the ensemble size increases.

However, Condorcet's theorem requires strict voter independence. Sequential debate explicitly destroys independence:

Independent Ensemble (Condorcet Valid):
Prompt ──┬──► Model 1 ──► Candidate Output 1 ──┐
         ├──► Model 2 ──► Candidate Output 2 ──┼──► Majority Vote
         └──► Model 3 ──► Candidate Output 3 ──┘

Sequential Debate (Independence Destroyed):
Prompt ──► Model 1 (Output 1) ──► Model 2 (Conditioned on Output 1) ──► Collapsed Agreement

Because Model 2 ingests Model 1's tokens, the probability distribution of Model 2 is conditioned on Model 1's failure modes. If Model 1 hallucinates a non-existent API parameter, Model 2 does not evaluate the API in a vacuum; it evaluates Model 1's framing, immediately narrowing its search space to rationalizing the phantom parameter.

---

The Grounding Rule: Anchoring the Verifier

To build multi-agent systems that resist consensus collapse, architects must enforce a strict separation between generation and verification:

1. No LLM-Only Feedback Loops: An LLM agent must never be the sole judge of another LLM agent's factual accuracy in production pipelines.
2. Deterministic Grounding Oracles: Every verification step must evaluate claims against an external source of truth:
   • Compilers and static type analyzers (e.g., TypeScript compiler, AST linters).
   • Ephemeral execution sandboxes (e.g., Docker containers running unit test suites).
   • Cryptographic signatures and schema validators (e.g., Zod, JSON Schema).
   • Read-only database assertions and vector similarity invariants.
3. Trace Legibility: Ensembles must expose their intermediate debate trace to human auditors. When an agent concedes an assertion, the system log must record whether that concession was triggered by a compiler failure or purely conversational pressure, addressing the core legibility problem in autonomous agents.

When verification is bound to deterministic environments rather than conversational agreement, multi-agent systems retain their ability to explore creative solution spaces without drifting into shared, ungrounded fiction.