𝕏olver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team

𝕏olver Scaffold. At each iteration, agents receive their past reasoning history and top ranked exemplars to generate new thoughts and responses, using tools (e.g., code) as needed. A judge model ranks outputs, and an intermediate memory maintains the best responses over time. Exemplars are initialized via episodic retrieval and continually updated with high-quality solutions from the memory. Iteration stops when convergence or max steps are reached, followed by final verification.

We analyze the effect of varying the number of agents and reasoning iterations on 𝕏olver's performance. In a controlled setup, we fix one variable (e.g., 3 agents or 2 iterations) and incrementally increase the other. Performance improves consistently on both AIME ’25 and LiveCodeBench with more agents or iterations. To probe deeper, we conduct a budget-controlled experiment on the AIME ’25 dataset, where the total reasoning budget (i.e., number of agents × number of iterations) is fixed. While iterative reasoning remains a crucial factor for 𝕏olver's performance, we find that increasing the number of agents—particularly beyond a minimum of three—yields additional, emergent improvements, leading to over a 4% performance gain.

We evaluate the impact of different retrieval strategies on 𝕏olver by comparing three settings: (1) External Retrieval, where the model retrieves the top-k (e.g., k = 5) most similar problems and their solutions from an external corpus; (2) Self-Retrieval, where the model recalls the top-k most similar problems and solutions from its own internal memory; and (3) No Retrieval, where neither external nor self-retrieval is used. Performance on both AIME '25 and LiveCodeBench follows the trend: External Retrieval > Self-Retrieval > No Retrieval.

We perform a fine-grained analysis of 𝕏olver's performance across both MATH-500 and LiveCodeBench.On MATH-500, 𝕏olver (both o3-mini-medium and QWQ-32B) consistently outperforms CheatSheet across nearly all seven subject categories, despite the latter relying on costly per-problem memory updates. The only exception is in Number Theory, where o3-mini-medium scores 99.2 compared to CheatSheet's 99.5. As for QWQ-32B, 𝕏olver achieves substantial accuracy gains over CheatSheet across all categories, with improvements of +9.0% in Prealgebra, +8.5% in Algebra, +11.0% in Number Theory, +8.5% in Counting and Probability, +8.8% in Geometry, +10.0% in Intermediate Algebra, and +7.5% in Precalculus. On LiveCodeBench, 𝕏olver demonstrates even more pronounced gains. The o3-mini-medium variant achieves 95.6%, 90.4%, and 85.8% accuracy on Easy, Medium, and Hard problems respectively, significantly outperforming CodeSim by +4.5%, +11.9%, and a striking +32.3% margin on hard examples. Even with a weaker QWQ-32B backbone, 𝕏olver (95.2%, 87.5%, 70.0%) surpasses all baselines and achieves similar gains.

We perform a detailed analysis of token usage, reporting input, reasoning, and output statistics for 𝕏olver (QWQ-32B) across all datasets. Our LLM token usage has computational complexity of O(mI), where m is the number of agents and I is the number of reasoning iterations. However, the runtime complexity remains O(I) since the dynamic agents operate in parallel. This is significantly more efficient than the self-consistency, which typically require 32–64 generations per example, as well as the baseline CheatSheet framework, which incurs a memory update complexity of O(n²)—quadratic in the test dataset size—due to usefulness estimation over all previous examples after solving each new example. As a multi-agent system, 𝕏olver allocates a majority of its tokens to context sharing and inter-agent communication, while approximately 25% are spent on actual reasoning steps. We also compare the total token usage of 𝕏olver with a single agent reasoning framework Search-o1 for token count. As expected, 𝕏olver incurs higher token costs—approximately 1.5× that of Search-o1—due to its collaborative and iterative multi-agent reasoning. However, this moderate increase represents a highly efficient trade-off given the substantial performance improvements observed.

We conduct an in-depth analysis combining qualitative runtime inspection with controlled experiments to understand the reasoning and problem-solving strategies behind 𝕏olver. We begin by manually studying Xolver’s agent interaction traces on AIME '25 and LiveCodeBench. These case studies reveal that at each iteration, dynamic agents attempt to improve upon earlier failures by leveraging Judge agent feedback and by aligning with top-ranked outputs stored in the shared memory. This process results in progressively refined outputs, increased agent alignment, and eventual convergence toward correct solutions. To verify this behavior systematically, we conduct a controlled experiment across both math and code tasks. We instantiate two dynamic agents with complementary strengths: a Coder agent and a Mathematician agent, each proficient in one domain but suboptimal in the other. We then measure their performance and agreement across iterations—defined as the percentage of problems in which both agents independently produce the same correct answer (for math) or code that passes the same test cases (for code). Both agents demonstrate consistent accuracy improvements over time, accompanied by a rising agreement rate.