Superficial Success vs. Internal Breakdown: An Empirical Study of Generalization in Adaptive Multi-Agent Systems

One-Line Summary

An empirical study of two representative adaptive multi-agent systems (AFlow, AgentDropout) across six domains shows that learned topologies (i) fail to transfer out-of-distribution (topological overfitting) and (ii) often retain reasonable accuracy while the underlying collaboration has already collapsed (illusory coordination)—with role-related and connection-related breakdowns accounting for ~59% of failures under domain shift.

Background & Motivation

Adaptive multi-agent systems (MAS) optimize both the set of agents A (roles) and their connections C (communication topology) from data, much like supervised topology search. The appeal is obvious: plug in a strong base LLM, learn a task-specific collaboration graph, and enjoy the gains. The paradox is that these systems are built from general-purpose LLMs yet are routinely tuned on a narrow slice of tasks—so it is unclear whether they behave as general-purpose systems at all.

This is more than an academic concern. Constructing an adaptive MAS is expensive (multiple LLM calls, repeated search, orchestration overhead), and deploying a separate MAS per task defeats the original motivation. If adaptive MAS only work in-domain, they are effectively narrow solvers in disguise.

Study Setup

Finding 1: Topological Overfitting

Single-domain MAS optimization produces topologies that are surprisingly brittle under distribution shift. AgentDropout trained on CaseHOLD (Legal) drops from 63.5% in-domain to an average of 55.78% across the five unseen domains, and AgentDropout trained on StrategyQA (Commonsense) cannot even produce valid outputs on most other domains (e.g., 0.6% / 0.5% / 0.1% / 15.7% on Legal / Detective / Science / Math) because topologies tuned for binary true/false answers fail to produce multiple-choice or numerical solutions.

AgentDropout on GPT-oss-20B — Train Domain (row) → Test Domain (col) Accuracy

A simple mitigation—mixing training data across all six domains while holding the total instance budget constant—recovers most of the in-domain baselines and significantly stabilizes the worst cases, hinting that generalization failures are driven by narrow training scope rather than an intrinsic limit of adaptive MAS.

Finding 2: Illusory Coordination

Even where accuracy looks acceptable, the qualitative and quantitative analyses reveal that collaboration is often not the reason. Applying MAST to 100 execution logs, role- and connection-related failures (categories 1–6) make up roughly 59% of all errors under domain transfer.

Failure Distribution under Domain Transfer (MAST)

Representative case studies show the pattern vividly: a Legal Text Extractor attempting Carnot-efficiency physics (role misalignment), a Validator ignoring prior agent outputs and re-solving from scratch (input neglect), and an Answer Synthesizer replying “True” to a multiple-choice question (task violation).

Role Alignment (R) and Connection Significance (O) under Domain Transfer — AgentDropout

Each value is normalized per row by the row-wise maximum, so the diagonal (in-domain) entries equal 1.00. Low normalized values expose illusory coordination even when raw accuracy is deceptively reasonable.

• Accuracy up, R collapsed: CaseHOLD → SciBench accuracy looks tolerable, but R falls to 22% of its in-domain value—the “success” does not come from role adherence.
• Accuracy down, R intact: COM2 → StrategyQA shows R at 82% of in-domain despite poor accuracy—confirming that accuracy and collaboration quality are largely dissociated.
• Influential but unhelpful messages: CaseHOLD-trained topologies produce strongly negative O under most transfers (down to −2.07 at SciBench), meaning messages drive outputs yet degrade them.
• Accuracy is a weak signal: Pearson correlations between accuracy and R / O are near-zero or mixed across all six benchmarks (|ρ| ≤ 0.12), formalizing that final-answer correctness is not a reliable proxy for coordination quality.
• Multi-domain training helps, but unevenly: It yields more stable positive O and higher R on most pairs, yet multi-hop transfer remains fragile.

Correlation and Ablation

A component-swap ablation isolates what exactly overfits. Role-OOD (keep in-domain connections, swap roles with OOD ones) drops accuracy by an average of −13.00 pp, while Connection-OOD (swap only connections) drops by only −1.24 pp—indicating that learned roles are substantially more task-specific than learned connections. A notable exception is MuSiQue (Multi-Hop), where Connection-OOD alone causes a 5.36 pp drop, showing that valid inter-agent links matter most when the task itself requires multi-hop integration.

Why It Matters

The paper reframes what it means for an adaptive MAS to “work.” A system can post competitive numbers on a new benchmark while internally collapsing into a single strong LLM carrying the load, with other agents producing irrelevant or actively misleading messages. Benchmarks that reward only final-answer accuracy miss this, and the field’s current optimization objectives actively encourage it.

• Diagnosis, not dismissal. Illusory coordination is defined as a diagnostic regime (high accuracy, low R and/or low O)—the claim is not that collaboration is always fake, but that it is invisible unless measured.
• Two portable metrics. R and O are lightweight—sentence-embedding cosines plus a per-message LLM-as-judge—and can be added to any adaptive MAS evaluation pipeline.
• Design implication. Because roles are the dominant source of overfitting, future adaptive MAS may benefit from role regularization, multi-domain role libraries, or joint role–connection objectives that explicitly reward cross-domain transfer.
• Evaluation implication. The community should move beyond single-domain accuracy and pair it with internal-dynamics diagnostics; reporting only aggregate correctness can reward systems that do not actually coordinate.

Links