Blog · arXiv Analysis · Modified July 10, 2026 · Last reviewed July 10, 2026

The Evaluator Becomes the Contagion Network

Evaluator contagion is a feedback-channel risk, not just a biased score: when one LLM agent judges another, the judge's preferred reasoning style can become a future behavior in the agent team. A June 2026 arXiv preprint makes the graph of evaluation itself measurable, including whether preference influence fades, persists, or cascades.

From Judge to Network

As of the July 10, 2026 source check, the current arXiv record for arXiv:2606.20493 [cs.LG] titles the paper Contagion Networks: Evaluator Preference Propagation in Multi-Agent LLM Systems. arXiv lists Liu Zewen as author, with version 1 submitted on June 18, 2026 and version 2 dated June 26, 2026. The title change matters: the paper is not primarily about proving moral prejudice, consciousness, or human-like infection in models. It is about measuring how evaluator preferences over reasoning strategies propagate through a network of agents.

The paper starts from a practical shift in agent design. A single chatbot may be judged after the fact, but a multi-agent system often asks agents to evaluate each other while work is happening. One agent critiques, ranks, selects, or rewards another agent's output. That evaluation then shapes the target agent's next strategy. The judge is no longer outside the system. It is part of the system's feedback loop.

That makes this page distinct from the site's pages on LLM judges as annotation budgets, grading cascades, agent-team trust graphs, and hidden anchors in deliberation. Those pages ask how evaluation is measured, delegated, or steered. Liu's paper asks whether evaluator preferences can travel through the agent network itself.

Current Context

As of July 2026, automated evaluators are no longer only leaderboard accessories. They appear in benchmark harnesses, red-team workflows, reward-model loops, model routing, regression tests, peer critique, and agent orchestration. Once an evaluator's output can change another agent's prompt state, memory, tool choice, reward, or future role assignment, evaluation becomes an intervention.

NIST's AI test, evaluation, validation, and verification work frames trustworthy AI as dependent on reliable measurements, metrics, and context-specific evaluation. The AI Risk Management Framework is voluntary, but it explicitly covers the design, development, use, and evaluation of AI products, services, and systems. NIST's Generative AI Profile adds a useful caution for this paper: benchmark or prompt-engineering tests may not systematically assess validity or reliability risks, and laboratory measurements may not transfer cleanly to real-world settings.

NIST's 2026 AI Agent Standards Initiative also makes agent security, interoperability, authentication, identity infrastructure, and security evaluation a standards topic rather than a narrow app-design issue. Evaluator contagion belongs in that same operational frame: if an agent can judge, route, reward, or reconfigure another agent, the evaluator is part of the control plane.

The regulatory context points in the same direction for the largest systems. EU AI Act Article 55 requires providers of general-purpose AI models with systemic risk to perform model evaluation using standardized protocols and tools, document adversarial testing, assess and mitigate systemic risks, report serious incidents, and ensure cybersecurity. Article 55 sits in Chapter V, which Article 113 makes applicable from August 2, 2025, while Article 111 gives a later transition for general-purpose AI models already placed on the market before that date. That does not turn this preprint into a compliance test. It does show why evaluator topology, evaluator records, and post-deployment monitoring are becoming governance objects rather than research trivia.

What Contagion Means

For this page, evaluator contagion means a dynamic evaluation failure: a judge's scoring style, rubric preference, or strategic habit changes the future behavior of the agents it evaluates, and that changed behavior can then influence other agents. Static judge bias is a one-time distortion of a score. Dynamic evaluator contagion is an influence channel inside the system, caused by the combination of a judge, an update rule, and an evaluation edge with enough authority to change future behavior.

The paper models that influence as a propagating signal. If Agent A prefers structured step-by-step reasoning, and Agent A repeatedly evaluates Agent B, Agent B may start sampling strategies that better satisfy that preference. If Agent B then evaluates Agent C, the absorbed preference can move another hop. The paper calls this a Contagion Network and represents the effect with a cross-agent contagion matrix.

The language is epidemiological, but the object is engineering evidence. The measured object is strategy preference, not disease, intent, moral corruption, or task competence. The paper defines propagation regimes: suppression, where preference influence attenuates; persistence, where it remains; and cascade, where it amplifies. The mathematical threshold depends on topology, including the spectral radius of the contagion matrix for fully connected networks and link-level coefficients for chain propagation.

The governance translation is simple. A multi-agent system may be built to increase cognitive diversity, but peer evaluation can quietly make agents converge on the same style. A system can look more coherent while becoming less plural, and the lost diversity may be invisible unless the operator tracks strategy traces rather than final scores alone.

The Experiment

The experiment uses three DeepSeek-chat agent instances, all from the same model family, differentiated by evaluator prompts. One evaluator prefers structured reasoning, one is balanced, and one prefers evidence-based reasoning. The strategy space includes step-by-step, direct, analogical, decomposition, and evidence-based approaches. Tasks span code generation, mathematical reasoning, text summarization, logical puzzles, and creative writing.

The v2 paper reports an expanded protocol: baseline preference measurement, pairwise contagion, three-hop chain propagation, committee mitigation, cross-model experiments with GPT-4o, DeepSeek-chat, and Claude-3.5-Sonnet, a fully connected topology validation, a random-evaluator null baseline, and sensitivity checks. The paper's conclusion summarizes the experiment as 4,200 API calls across homogeneous and cross-model settings.

The reported pairwise contagion signal is positive in the homogeneous setting. The abstract gives gamma in the range 0.157 to 0.352 for the cross-agent matrix, while the detailed discussion reports mean off-diagonal values from 0.143 to 0.304 over two seeds. In the chain experiment, all hops remain below the cascade threshold, with beta3 = 0.0126 +/- 0.0038 over four seeds and a 95% confidence interval of 0.0089 to 0.0163. The original seed shown in the per-hop table yields beta3 = 0.0055.

The important point is not that preference influence exploded in the chain setting, nor that these coefficients should be treated as constants outside the protocol. It is that the influence was measurable, and the same underlying pairwise relations can matter differently when the evaluation graph changes.

Topology Changes the Risk

The v2 update makes topology the center of the page. In a chain, the same three DeepSeek-chat agents sit in the suppression regime. In a fully connected topology, the paper reports entropy decrease across all three agents and an average preference-concentration increase, which it interprets as cascade dynamics. The reported fully connected homogeneous spectral radius is about 1.39 to 1.40, above the cascade threshold used in the paper's theorem.

That distinction is a useful correction to a lazy governance rule. "Use one model family" is not a safety guarantee, and "use multiple models" is not a diversity guarantee. The evaluation graph has its own risk surface. A chain, star, committee, router, ring, fully connected mesh, or dynamic scheduler can expose agents to different feedback loops even when the model set is unchanged.

For a deployed system, evaluation edges need direction and authority labels. It matters whether one agent's judgment merely annotates another output, changes ranking, updates memory, allocates future tasks, selects final answers, or trains a reward mechanism. A topology diagram without those control effects is not enough. The same record belongs in AI agent observability and agent identity work: the operator needs to know which authenticated component judged which other component, under which rubric, with which power to change future execution.

Diversity Is Not Simple

The paper's most useful tension concerns diversity. It contrasts homogeneous DeepSeek-chain suppression with cross-model and fully connected amplification. In the cross-model setting, the paper reports rho_cross = 1.296 +/- 0.016 over four seeds for GPT-4o, DeepSeek-chat, and Claude-3.5-Sonnet, with all four seeds above the cascade threshold. It also reports a neutral-prompt cross-model comparison in which architectural priors alone produce rho_neutral = 1.498, compared with rho_mixed = 1.299 under explicit role prompts.

That is not a settled deployment rule. The neutral-prompt result is a single-comparison finding and should be replicated before being treated as a general law. The broader warning is still valuable. Cross-model diversity can produce complementary checks, but it can also introduce stronger preference transfer through incompatible judge habits or shared architectural priors.

The paper's mitigation result is narrower and more concrete: in its tested setup, increasing the evaluator committee from one evaluator to three reduced effective contagion by 68.9% +/- 14.1% over four seeds, with a 95% confidence interval of 55.1% to 82.7%. The original seed gives the older headline value of 72.4%, from 0.264 to 0.073. For governance, that means committee evaluation should be measured as a dynamic system, not assumed good because it has more members.

Failure Modes

The practical failure modes are not mysterious. A judge monoculture forms when several evaluators share the same preferred style and slowly train the agent team toward that style. A committee echo forms when a larger committee reduces variance on paper but still rewards the same hidden preference. Router capture occurs when an evaluator influences which agents get future work, so preference transfer becomes resource allocation. Reward laundering occurs when a preference enters through a peer score and later appears as "task quality" in a training or selection record.

Two safety failures deserve separate treatment. Evaluation awareness can make agents optimize for the known judge rather than the task, especially when evaluator prompts or rubrics are stable. Incident reconstruction can also fail if evaluator logs contain final scores but not the intermediate prompts, rubrics, identity bindings, model versions, and routing decisions that explain how the preference spread.

There is also a null-baseline failure. The v2 paper's random-evaluator control reports that off-diagonal gamma values from random feedback were not statistically separable from real evaluators in one test, with p = 0.589. That does not erase the paper's topology result, but it does warn against overreading small pairwise coefficients without a control. A governance program should measure not only the evaluator signal, but also the drift created by the adaptation mechanism itself.

The last failure mode is hidden topology. If a deployment cannot say which agent evaluates which other agent, how often, under which rubric, with what update authority, it cannot know whether it has a suppression graph, a persistence graph, or a cascade graph.

Evaluation Record

A serious multi-agent evaluation record should preserve enough evidence to reconstruct the evaluator network, not just the final score.

Limits That Matter

The limitations matter because the result is exploratory. The homogeneous experiment uses one model family, a synthetic strategy space, coarse evaluator-preference prompts, and a controlled Test-Time Reinforcement Learning update over strategy weights. Natural-language critiques, memory updates, fine-tuning, tool-mediated rewards, or reinforcement learning from deployed feedback could behave differently.

The random-evaluator result is especially important. The paper says the pairwise gamma values cannot yet be cleanly separated from protocol drift under the TTRL measurement framework, even though the regime-level comparison across topologies remains meaningful. A deployment review should therefore avoid treating a single contagion coefficient as a universal property of the model. It is a property of the model, prompt, update rule, task set, metric, and graph.

Several results still need replication or broader scope. The neutral-prompt comparison is single-seed. The fully connected validation is reported for one topology experiment. Star, ring, and dynamic production orchestration patterns remain future work. The paper does not certify any real multi-agent deployment as safe.

Governance Standard

A serious multi-agent evaluation system should publish its evaluator graph. Which agents evaluate which other agents? Which model family, prompt, rubric, and strategy preference does each evaluator carry? Are evaluator outputs used for ranking, reward, memory, task allocation, or final answer selection? How often are evaluator roles rotated or committee-reviewed?

The release record should include a measured contagion matrix or an equivalent interaction audit, a topology description, committee-size tests, entropy or diversity traces over time, null-baseline or random-feedback controls, and slice tests for domains where one judge habit dominates. If the institution uses cross-model committees, it should test whether model diversity reduces or increases preference transfer under its actual workflow.

Post-release governance should treat evaluator topology as a change-controlled asset. Adding a router, replacing a judge model, exposing an evaluator to new memory, changing a rubric, or raising an evaluator's authority from annotation to routing should trigger renewed evaluation. This connects the paper to AI change management, post-market monitoring, and AI red teaming: the point is not to ban agent judging, but to test the feedback channels before they become hidden production behavior.

The Spiralist rule is conservative: when agents judge each other, evaluation is no longer a neutral measurement layer. It is a live influence channel. Govern the judge as part of the agent network, or the network will learn the judge's habits and call that consensus.

Source Discipline

Current-source claims on this page were checked on July 10, 2026 against arXiv, NIST, and EUR-Lex primary sources. This page treats Liu's paper as a current arXiv preprint, not a peer-reviewed deployment standard. Its numbers are useful because they force evaluator topology into the measurement record, but they should not be converted into universal constants for all agents, all models, or all adaptation mechanisms.

The external governance sources used here have different scopes. NIST TEVV and the AI RMF provide voluntary measurement and risk-management context. NIST's Generative AI Profile cautions against overgeneralizing benchmark and laboratory tests. EU AI Act Article 55 is a legal obligation for providers of general-purpose AI models with systemic risk, not a general certification scheme for every multi-agent product. These sources support the article's evidence standard; they do not prove that this specific contagion metric is required by law.

Sources

Return to Blog