One model answers alone, and you get one model's blind spots. Mixture-of-Agents runs a panel of LLMs, feeds all their answers into another panel, feeds those into a final aggregator — a few layers deep. The strange part: a stack of mid-tier open-source models out-reasons a single frontier model.
loading…
A query flows in. Each layer's proposers answer in parallel; their answers are glued together as references and handed to the next layer, which answers again — better, because it can see everyone's draft. A final aggregator writes the response.
Ask one committee a hard question and you get a noisy first draft. Now circulate that draft through another committee — and another — each one reading every prior answer and improving on it. That relay is Mixture-of-Agents.
A single LLM has fixed weaknesses baked in from training. Whatever it gets wrong, it gets wrong with confidence — there's nobody to catch it.
Independently trained models fail in partly independent ways. Pool several answers and a synthesizer can cancel the noise — this is the fusion idea.
Take the synthesized answer and feed it back as input to another panel. Each layer improves on the last. Iterating gives monotonic gains.
An LLM generates a better answer when shown other models' answers — even when those other answers came from weaker models. The aggregator isn't picking a winner; it's doing genuine synthesis, pulling good fragments from many drafts.
That single observation is what makes MoA more than a curiosity. If a model only ever copied the best answer it saw, a panel would be capped at its strongest member. Instead each layer can produce something none of its inputs contained — which is why a stack of mid-tier open models can out-reason one frontier model.
The whole rest of this lesson is mechanism: two roles, the layered formula, the one magic prompt, the numbers, and the one way it breaks.
MoA splits every model into one of two roles. The trick is that they want opposite things: proposers are valued for diversity, the aggregator for capability.
A proposer's job is not to be right. It's to offer a "nuanced and different perspective" — a reference the aggregator can mine. So a weaker model from a different lineage is genuinely useful: different training data = different, partly-independent errors.
The aggregator is the capability-sensitive seat — put your strongest model there. And the same model can play both roles in different layers; a good aggregator is usually a good proposer too.
The architecture is l layers × n agents. Drag the dials below and watch the stack redraw — more depth gives better answers but serializes more rounds, and the LLM-call count climbs. Press Run query to send a token down the layers.
Two pieces matter. ⊕ is not a sum — it means "glue the agents' outputs in as numbered references and tell the next model to synthesize them." And every layer re-attaches x₁, the real question, so the chain can never drift off-topic.
The published sweet spot is 3 layers × 6 proposers ("Together MoA"); the 2-layer "MoA-Lite" already matches GPT-4o. Quality here is an illustrative curve — real gains are monotonic-but-diminishing in depth.
A baseline that just selects the best proposed answer loses to one that writes a combined one. Click fragments from the proposals below and watch the aggregator weave a single answer better than any input — including from answers that were worse overall.
Notice the dashed proposal is weaker overall — yet it holds the best line about cost. Synthesis lets the aggregator take that one good fragment.
An LLM-ranker that picks one whole answer can never beat its strongest proposal. Good fragments inside weaker answers are thrown away.
Six different proposers beat six samples of one model. Different lineages = more independent errors to cancel.
Synthesis tends to inflate length — MoA slightly loses on conciseness even as it wins on correctness and factuality.
There is no special architecture, no fine-tuning, no new loss. The "fusion" happens entirely because of how the aggregator is asked. This text is identical in the paper, the reference repo, and OpenAlice's own hermes-agent implementation.
References are appended as a numbered list (each prepended with \n{i+1}. ) — and injected only when the list is non-empty, so layer 1 with no references is just a plain answer. The clauses "critically evaluate" and "do not replicate" are exactly what turn concatenation into synthesis.
Keep that "some may be incorrect" clause in mind — in §07 we'll see it is not a reliable defense against a deliberately deceptive reference.
Proposers run hot (e.g. 0.6–0.7) for variety; the aggregator runs cooler (~0.4) for focused synthesis.
A minimum-successful-references threshold means one failed proposer doesn't kill the query — the aggregator works with whoever answered.
Every layer re-sees the original user query alongside the references, so a long chain can't drift away from what was actually asked.
AlpacaEval 2.0, length-controlled win rate. An MoA built entirely from open-source models beat GPT-4 Omni by +7.6 points — and the single best proposer alone managed only 43.9%. The whole is far more than its parts. Hover the bars.
Bars animate in on scroll. These are LLM-judged win rates — length-controlled to remove the "longer = better" bias, but still "preferred by a judge," not ground-truth correctness. The +7.6 headline is real and replicated; read it as a preference signal.
Near-saturated, so gains are small but consistent. With GPT-4o as aggregator, MoA reaches 9.40.
Beats the best single proposer on robustness, correctness, factuality, completeness, insightfulness, metacognition — loses only on conciseness.
Open-weight token prices make MoA-Lite match GPT-4o's cost while beating its quality; some configs hit GPT-4-Turbo quality at ~2× lower cost.
| Configuration | Proposers | Layers | Aggregator | LC win rate |
|---|---|---|---|---|
| MoA w/ GPT-4o | 6 open-source | 3 | GPT-4o | 65.7% ±0.7 |
| MoA (all open) | 6 open-source | 3 | Qwen1.5-110B-Chat | 65.1% ±0.6 |
| MoA-Lite | same 6 | 2 | Qwen1.5-72B-Chat | 59.3% ±0.2 |
| GPT-4 Omni (May 2024) | — | — | single model | 57.5% |
| Qwen1.5-110B-Chat | — | — | best single proposer | 43.9% |
The same property that makes MoA strong — every layer re-reads every prior answer — is its weakness. A 2025 follow-up injected one carefully-instructed deceptive agent into a 6-agent / 3-layer stack. Flip the switch and watch the poison propagate up the layers.
The aggregator's instruction to "critically evaluate… some may be incorrect" does not reliably make it reject a confidently-wrong reference. The deceptive answer keeps getting re-presented at every layer, so the corruption propagates upward instead of being filtered out.
The measured drop: 49.2% → 37.9% — essentially all the MoA advantage, gone, from a single poisoned voice. Defenses (a larger/stronger aggregator, limiting how much each agent sees) help only partially.
Only ever stack models you trust. A MoA panel of untrusted, third-party agents isn't just a quality boost — it's an attack surface. This is the multi-agent twin of prompt-injection discipline.
Proposers and aggregators. The layered formula. The one synthesis prompt. The benchmark where open beats frontier, and the single deceptive voice that erases it all. MoA is just fusion, stacked — and a real lever, not an academic flex. You now know how to build a council that sharpens itself.
A council is only as trustworthy as its members. Now see how a single model is tuned to be helpful, honest, and worth seating on the panel.