Predictable Swarm Scaling

The task DAG任务 DAG 结构

Any complex exploration task can be represented as a directed acyclic graph of steps, a DAG. Each step is a node, and an edge runs from a prerequisite to the step it unlocks. No step can be its own prerequisite, so the edges never close a cycle. A step can start only once all of its prerequisites are complete, and deeper steps generally take longer. The DAGs in this post grow by trial and error, as in a search for a pretraining recipe. Each new step is a variant of a parent in the layer above, and parents that improved on what they built on are drawn more often. A step's value is its parent's value plus a random change. The animation below grows a small example.

复杂的探索任务可以抽象为一张由步骤构成的有向无环图，即 DAG。每个步骤是一个节点，边从前置步骤指向它解锁的步骤。没有步骤能成为自己的前置，所以边不会成环。一个步骤只有在所有前置步骤完成后才能开始，而且越深的步骤通常耗时越长。本文的 DAG 靠反复试错长出来，就像搜索预训练 recipe。每个新步骤是上一层某个父节点的变体，相对自己所依赖步骤改进大的父节点更常被抽中。一个步骤的分值等于父节点的分值加一个随机变化。下面的动画用一个小例子把这个过程演一遍。

Two scores for two kinds of exploration对应两类探索的两种分数

A finished step can be scored in two ways, one for each kind of exploration. In discovery, an agent learning a codebase or a protocol wants every fact. Each step is one fact worth one point, so the score is coverage, the share of steps finished. In optimization, an agent searching for a pretraining recipe keeps only its best result. Its best score is the highest value among finished steps, scaled so that the DAG's best recipe is worth 1. Coverage counts every branch, while the best score waits on the one chain that leads to the best recipe, deep in the DAG. Figures 2 to 10 plot both side by side, and the section on deployment asks how their different shapes change the right way to spend k agents.

一个完成的步骤有两种计分方式，各对应一类探索。发现型探索中，熟悉代码库或协议的智能体想要每一条事实。每个步骤就是一条事实、记一分，所以分数是覆盖度，即已完成步骤的比例。优化型探索中，搜索预训练 recipe 的智能体只保留最好的结果。它的最优分数是已完成步骤中的最高分值，按 DAG 的最优 recipe 记为 1 缩放。覆盖度数的是所有分支，最优分数等的却是通向最优 recipe 的那一条链，它在 DAG 的深处。图 2 到图 10 都并排画出这两种分数；部署一节讨论两者形状的差异如何改变 k 个智能体的最佳用法。

Single-agent baseline单智能体基准

When a single agent works through the task, each completed step unlocks new options downstream, so progress speeds up through the middle stages. Time is measured in units of t_mid, about how long one agent needs to finish half the steps. Across 16 runs on task DAGs averaging about 730 steps, both scores follow an S-shaped curve in log time. The best score rises more gently, since it moves only when a variant beats every step before it.

单个智能体独立探索时，每完成一个步骤都会解锁下游的新选项，使中段推进逐渐加速。时间以 t_mid 为单位，约为单智能体完成一半步骤所需的时间。在平均约 730 个步骤的任务 DAG 上跑 16 次，单智能体的两种分数在对数时间上都呈 S 形。最优分数升得更缓，因为只有当某个变体超过此前所有步骤时它才前进。

Standard swarms标准蜂群

A standard swarm caps the number of concurrent agents. Each agent follows its branch to the end, then stops and frees its slot. The coordinator immediately starts another agent on the next ready step.

标准蜂群设置了最大并发人数上限。每个智能体沿着当前分支一直推进，直到没有后续步骤可做时退出并释放名额。调度器随即派出新的智能体，接手其他已经就绪的步骤继续工作。

Refilling empty slots prevents agents from sitting idle. With 16 agents, the coverage speedup rises from 13x at half coverage to 22x at 90%, and the best score gains 6.6x and 15x at the same levels.

随时补齐空位避免了智能体闲置。16 个智能体时，覆盖度的提速从一半处的 13 倍升到 90% 处的 22 倍；最优分数在同样两个水平上提速 6.6 倍和 15 倍。

Both curves keep their S-shape and shift earlier as the swarm grows. At the halfway level the best score shifts less, since it waits on the deep chain that holds the best recipe, and extra agents cannot walk one chain faster.

两条曲线都保持 S 形，随着蜂群变大整体提前。在一半水平上，最优分数提前得少一些，因为它等的是通向最优 recipe 的那条深链，多加智能体也无法让一条链走得更快。

Recursive swarms递归蜂群

A recursive swarm runs without a central coordinator. A single root agent starts the run. Whenever an agent finishes a step that unlocks several new steps, it keeps one and forks child agents for the others, up to a cap on swarm size. Each child agent works through its own branch to the end.

递归蜂群不需要中央调度器。最初只有一个根智能体开始执行。每当一个步骤完成后解锁了多个新步骤，该智能体继续做其中一个，并为其他分支派生出新的子智能体，总人数受预设上限约束。每个子智能体独立运行，直到各自的分支结束。

Concurrency naturally follows the branching of the task. Starting from a single agent slows the ramp-up, so at a cap of 16 the speedup at half coverage is 6.7x. Because agents can fork whenever branches appear, the swarm sustains work better in later stages and reaches 12x at 90% coverage. The best score gains 4.6x and 13x at the same levels.

蜂群人数随着分支的展开自然增减。因为最开始只有单个智能体，前期的并发建立较慢，在 16 个智能体的上限下，覆盖度在一半处的提速为 6.7 倍。后续遇到新分支又能随时派生，因此后期能维持较好的产出，到 90% 覆盖度时提速达到 12 倍。最优分数在同样两个水平上提速 4.6 倍和 13 倍。

Comparing the two swarms两种蜂群的对比

We run both swarms on the same task DAGs at swarm sizes from 2 to 64. From 4 agents up, the standard swarm is faster on both scores. With 16 agents it reaches 90% coverage 22x sooner than one agent, against 12x for the recursive swarm, because it keeps every slot occupied until fewer steps are ready than agents. On the best score the gap shrinks to 15x against 13x, since both swarms wait on the same deep chain.

在同一批任务 DAG 上、2 到 64 的蜂群规模下对比两种蜂群。从 4 个智能体起，标准蜂群在两种分数上都更快。16 个智能体时，它到 90% 覆盖度比单智能体快 22 倍，递归蜂群为 12 倍，原因是在就绪步骤少于智能体之前，它一直让每个名额都有人干活。在最优分数上差距缩小到 15 倍对 13 倍，因为两种蜂群等的是同一条深链。

Step selection rules步骤选取规则

We compare five ways to choose the next step: sticking to the agent's own branch, picking at random, picking the shallowest step first, picking the shortest step first, and picking the critical-path step first. The critical-path step heads the longest chain of remaining work.

我们对比了挑选步骤的五种策略：优先沿自身分支继续做、随机挑选、优先选最浅的步骤、优先选耗时最短的步骤，以及优先选关键路径上的步骤，即下游剩余工作链最长的那个步骤。

On coverage, prioritizing shallow or short steps unlocks new branches fastest. With 2 agents, picking shallow steps first reaches half coverage 3.1x sooner than one agent, against 1.7x for agents that stick strictly to their own branches. Critical path first is the slowest rule on coverage. It heads for the deep layers where the best recipe sits and reaches half the best score soonest at every swarm size.

在覆盖度上，优先挑选浅层或耗时短的步骤能最快解锁后续分支。以 2 个智能体为例，优先做浅层步骤到一半覆盖度比单智能体快 3.1 倍，而坚持做自身分支为 1.7 倍。关键路径优先在覆盖度上最慢，却直奔最优 recipe 所在的深层，在每个蜂群规模上都最早到达一半最优分数。

In real-world exploration, however, an agent knows neither the full structure of the task nor the values of steps it has not tried. Rules like critical path first need to see the whole DAG in advance, which is impossible in an unfamiliar environment. In practice, an agent naturally follows its own branch to the end and only then asks for a new step. Because the own-branch rule assumes nothing about the environment, we use it for all remaining experiments.

但在真实的未知任务中，智能体在开始前既不知道后续所有步骤的完整结构，也不知道没试过的步骤的分值。像关键路径优先这类策略需要预知全局结构，在现实中并不可能做到。在实际探索时，智能体自然会先顺着自己手头的分支往下做，直到当前分支没有后续步骤了，再去领取新步骤。这种做法不需要对具体环境做任何特殊假设，因此后续所有实验均统一采用这一策略。

Scheduling overhead and communication cost调度开销与沟通成本

Working as a swarm introduces two sources of overhead. Scheduling overhead is the one-time cost of assigning a step and handing over context: 5% of a median step whenever an agent starts, is dispatched, or builds on another agent's work. Communication cost is the continuous effort active agents spend to synchronize progress: every step takes 2% longer for each other agent at work, so the share of time spent on communication rises with swarm size.

蜂群协作带来两种开销。调度开销是派发任务和交接上下文的一次性代价：每当智能体启动、被派发任务，或者在别的智能体的成果上继续，就多花中位步骤耗时的 5%。沟通成本是在岗智能体之间持续同步进展的开销：每多一个在岗的同伴，每一步就慢 2%，所以蜂群越大，花在沟通上的时间占比越高。

Across hundreds of handoffs, scheduling stays under 1% of agent-time at every swarm size. Communication accumulates with every step: in a standard swarm it takes 2% of agent-time at 2 agents, 10% at 8, and 37% at 64. Up to 8 agents a swarm keeps most of its cost-free speedup, and past 32 agents, doubling a standard swarm adds little speed on either score.

在数百次交接中，调度开销在每个规模下都不到总工时的 1%。沟通成本则随每一步持续累积：标准蜂群里，2 个智能体时占总工时的 2%，8 个时占 10%，64 个时占 37%。8 人以内的蜂群保留了大部分无开销时的提速；超过 32 人后，标准蜂群再翻倍，两种分数都几乎不再变快。

Total compute across swarm setups不同蜂群设定下的总算力消耗

When a task is divided among multiple agents, does total compute blow up?

把任务分给多个智能体做，总的算力消耗会不会大幅膨胀？

We measure total compute as agent-time, the active working time summed across all agents, and compare it with what one agent needs. Across all swarm setups, reaching 90% coverage takes between 0.64x and 1.35x the agent-time of one agent, and a standard swarm of 16 needs only 0.69x. To reach 90% of the best score, every swarm needs less than one agent, between 0.70x and 0.96x. One agent easily gets stuck on deep, expensive branches early on, while a swarm advances many branches at once and leaves those steps in progress.

我们把所有智能体的实际工作时间加总记为总工时，与单个智能体所需的总工时对比。在所有蜂群设定下，到 90% 覆盖度所需的总工时在单智能体的 0.64 到 1.35 倍之间，16 个智能体的标准蜂群只要 0.69 倍。到 90% 最优分数时，每种蜂群所需的总工时都少于单智能体，在 0.70 到 0.96 倍之间。单个智能体很容易在前期卡进又深又耗时的分支；蜂群多路并发，把这些步骤晾在进行中。

How task structure limits speedup任务结构对提速的影响

Swarm speedup depends on how wide the task is. Each task keeps its number of steps, about 740 on average, while its depth changes, so depth and width trade off. Many layers leave few steps per layer and form a narrow chain, while few layers leave many parallel steps in each. Cutting the number of layers from 136 to 4 widens the task from 5 steps per layer to 184.

蜂群能提速多少，取决于任务本身有多宽。每个任务的步骤数保持不变，平均约 740 步，只改变深度，所以深度与宽度此消彼长。层数多时每层步骤少，接近一条细长的链；层数少时每层有大量可并行的步骤。把层数从 136 层压缩到 4 层，任务宽度就从每层 5 步扩展到每层 184 步。

On the narrowest DAG, 5 steps per layer, a standard swarm of 16 reaches 90% coverage 3.9x sooner than one agent and 99% only 1.3x sooner. Most agents wait in line for the one open chain, and the deepest steps set the pace. The best score gains a little more, 4.9x at 90%.

在最窄的 DAG 上，每层 5 步，16 个智能体的标准蜂群到 90% 覆盖度比单智能体快 3.9 倍，到 99% 只快 1.3 倍。大多数智能体在排队等那条唯一打开的链，节奏由最深的步骤决定。最优分数的提速略高，90% 处为 4.9 倍。

On a wide DAG with 92 steps per layer, many branches run at once and the same swarm reaches 90% coverage 30x sooner, above its own size. One agent spends most of its time on deep steps that a swarm leaves in progress. The best score gains 12x there. On the widest DAG the recursive swarm falls behind, reaching 90% coverage only 1.1x sooner. Its root opens the whole first layer at once and forks children for as many steps as the cap allows. It keeps the rest and ends up doing most of the work. The width of the task and the layer profile behind it set the ceiling on speedup, whatever the number of agents.

相反，在每层 92 步的宽 DAG 上，大量分支同时推进，同样的蜂群到 90% 覆盖度快了 30 倍，超过了它自己的规模。单智能体的大部分时间花在蜂群可以晾在进行中的深层步骤上。最优分数在这里提速 12 倍。在最宽的 DAG 上递归蜂群掉了队，到 90% 覆盖度只快 1.1 倍。根智能体一次打开整个第一层，按上限为尽可能多的步骤派生子智能体。剩下的都留给自己，于是大部分工作落在它身上。任务的宽度以及背后的层级剖面决定提速的上限，与智能体数量无关。

Elo per token: continue a session or start new onesElo per token：接着跑还是重开

When Agents Slow Down² rates an agent's best solution so far at each token budget against its other budgets, called self-Elo, and against other systems, called joint Elo. It fits a Bradley–Terry model within each task and reads the result as Elo against log budget. Its reference is repeated sampling with attempts of a fixed length: a tenfold budget buys tenfold attempts, and their best gains 400 Elo per decade on any task. A session whose self-Elo climbs more slowly than that learns less from its own history than a restart would. The last budget at which its self-Elo still climbs faster than 400 per decade is its scaling inflection point. The paper finds that agents fall below the reference after a few context compactions, and that splitting the budget into parallel sessions then wins.

When Agents Slow Down² 把智能体在每个 token 预算下迄今的最优解，与它在其他预算下的最优解比较，得到 self-Elo；与其他系统的最优解比较，得到 joint Elo。它在每个任务内拟合 Bradley–Terry 模型，把结果读成 Elo 对 log 预算的曲线。参考线是固定长度的重复采样：预算翻十倍，尝试次数也翻十倍，其中的最优值在任何任务上都稳得 400 Elo。self-Elo 爬得比这条线慢的会话，从自己的历史里学到的还不如重开一次。一个会话的 self-Elo 仍以每十倍超过 400 的速度上升的最后一个预算，就是它的 scaling inflection point。论文发现智能体在几次上下文压缩之后就掉到这条线以下，此时把预算拆成并行会话更划算。

Figure 11 repeats the construction on these DAGs over the paper's span of three decades up to the budget. Agent-time takes the place of tokens, and the best score is the quality of a solution. Four systems spend the same budget. The first, pass@1, is one long session. The second, pass@4, is the paper's split: the best of four independent sessions on a quarter of the budget each. The other two are group@4, one standard swarm of four, and pass@2 group@2, the better of two swarms of two on half the budget each. As in the paper, the left panel rates pass@1 and group@4, the two single-session systems, in self-Elo against the reference. The right panel compares all four in joint Elo, since a best of several sessions has a narrower spread that self-Elo would reward. Like the paper's agents, pass@1 starts on the reference, bends below it, and passes its scaling inflection point early. Past the bend pass@4 overtakes it and leads at the budget, because a session's final best score depends far more on which DAG it grew than on its last stretch of work. group@4 tracks pass@1, since its agents share one DAG, and pass@2 group@2, with two DAGs, falls between group@4 and pass@4. The agents here forget nothing, so the bend comes from the DAG. Late steps land in thin deep layers with few variants to draw from, while a fresh session re-rolls the wide shallow ones.

图 11 在这些 DAG 上重做这套构造，预算范围取论文的跨度，即预算之前的三个十倍。智能体时间代替 token，最优分数衡量解的质量。四种系统花同一份预算。第一种 pass@1 是一个长会话。第二种 pass@4 是论文的拆分：四个独立会话各用四分之一预算、取其中的最优。另外两种是 group@4，即一个 4 人标准蜂群，以及 pass@2 group@2，即两个 2 人蜂群各用一半预算、取较好的那个。与论文一样，左图用 self-Elo 对照参考线评估两个单会话系统 pass@1 和 group@4。右图用 joint Elo 比较全部四种，因为多个会话取最优的离散度更小，self-Elo 会因此偏向它。和论文里的智能体一样，pass@1 起点贴着参考线，随后弯到线下，很早就越过 scaling inflection point。过了这个弯，pass@4 反超它并在预算处领先，因为一个会话最终的最优分数主要取决于它长出了哪张 DAG，最后一段工作带来的提升小得多。group@4 跟着 pass@1 走，因为它的智能体共享一张 DAG；pass@2 group@2 有两张 DAG，介于 group@4 和 pass@4 之间。这里的智能体什么都不会忘，所以弯下去的原因在 DAG。后期的步骤落在稀疏的深层，可抽的变体少，而新会话重新在宽阔的浅层里抽签。