Larger Tokenizers Suffer More Multi-Epoch Regret分词器越大，multi-epoch regret 越高

Byte-level BPE builds a vocabulary by merging frequent pairs字节级 BPE 靠合并高频对建出词表

Keeping only the K lowest-ranked tokens of the 128K vocabulary (for K from 1K to 64K) is the same as stopping BPE at K tokens. We call any token outside the first 8K a rare token. Take " memor" (#16,420) as our running example. The 128K vocabulary gives it its own input and output rows. The 8K vocabulary splits it into " mem" and "or", and the byte vocabulary reads six bytes. Next, we measure what those dedicated rows do across multiple epochs.

只留 128K 词表里编号最小的 K 个 token（K 从 1K 到 64K），就等于让 BPE 在合出 K 个 token 时停手。我们把前 8K 之外的 token 叫作少见 token。全文拿 " memor"（#16,420）举例：128K 给它单独分了输入行和输出行；8K 词表把它拆成 " mem" 和 "or"；字节词表则直接读六个字节。下面我们就来测一测，多训几个 epoch 时，这些专属行会带来什么影响。

Matched 1-epoch and 8-epoch runs compare 256-byte to 128K vocabularies成对跑 1-epoch 和 8-epoch，对比 256 字节到 128K 词表

Larger tokenizers suffer more multi-epoch regret at 8 epochs训 8 个 epoch，分词器越大，multi-epoch regret 越高

This study grew out of an earlier experiment. The hashed 2-gram and 3-gram input tables of the Over-Tokenized Transformer (Huang et al., 2025) lower the loss up to 520m and overfit when the data repeats. At 1.2B, which trains for 2.4 epochs, the gain becomes a loss. We therefore asked whether the vocabulary of a plain tokenizer has the same problem.

这项研究源于我们更早的一个实验。Over-Tokenized Transformer（Huang 等人，2025）的哈希 2-gram 和 3-gram 输入表在 520m 及以下能降低损失，但数据一重复就过拟合。到了 1.2B，数据要训 2.4 个 epoch，收益就变成了损失。所以我们想知道，普通分词器的词表是不是也有同样的问题。

Read more · Experiment page延伸阅读 · 实验主页 Sampled Softmax with Over-Tokenized Transformer The earlier experiment: a sampled softmax from 130m to 1.2B, and the over-tokenized input tables that overfit repeated data.更早的实验：130m 到 1.2B 上的采样 softmax，以及在重复数据上过拟合的 over-tokenized 输入表。

Figure 1 shows that multi-epoch regret grows with vocabulary size at 300m. With fixed 4096-token windows (dashed line), regret rises from near zero on the 256-byte vocabulary to about 0.08 bits per byte at 128K. Vocabularies from 512 to 8K sit on a flat plateau. Above 8K, every doubling pushes regret higher and wipes out most of 128K's 1-epoch lead over 8K. Over 8 epochs, 128K lowers training loss only slightly more, yet every memorized bit hurts unseen text much more.

图 1 测了 300m 模型下不同词表的 multi-epoch regret。虚线是固定 4096 token 窗口的结果：regret 从 256 字节词表的几乎为零，涨到 128K 的约 0.08 比特/字节。512 到 8K 词表停在一个平缓的平台上。过了 8K，词表每翻一倍，regret 就往上涨一截，几乎吃掉了 128K 在 1 个 epoch 下对 8K 的优势。训 8 个 epoch 时，128K 的训练损失只多降了一点点，但它每多背下 1 比特，在未见文本上损失都多得多。

Even when every window holds the same text (solid line), multi-epoch regret still rises above 4K. With 4096-token windows, a smaller vocabulary packs less text per window and takes more steps to read the corpus: 54,247 steps for bytes versus 11,443 for 128K. Matching the text per window puts every run at about 11,444 steps. Then vocabularies from 256 bytes to 4K have a regret of 0.02 to 0.03 bits per byte, so bytes no longer sit near zero. Above 4K, regret still climbs with vocabulary size, from 0.031 at 8K to 0.081 at 128K. With equal text per window, 16K has 0.053 regret and 32K has 0.039, flipping their order from the 4096-token runs. Both runs in a setup use the same subset of documents, and changing the window length keeps only about 9 percent of those documents the same. That document shift may explain why 16K and 32K flip, and why byte regret rises from 0.0003 to 0.0175.

词表越小，4096 token 窗口装的文本就越少，读完同样多的文本要走更多步数：字节词表要走 54,247 步，128K 只要 11,443 步。图里的实线改成让每个窗口装同样的文本，每次运行都走约 11,444 步。这么对齐后，256 字节到 4K 词表的 multi-epoch regret 都在 0.02 到 0.03 比特/字节，字节词表不再接近零。4K 往上，regret 还是随词表变大而上升：8K 是 0.031，128K 是 0.081。装同样文本时，16K 的 regret 是 0.053，32K 是 0.039，顺序跟 4096 token 窗口时反了过来。同一套配置的两次运行重复的是同一批文档，换个窗口长度，重合的文档就只剩约 9%。所以 16K 和 32K 顺序颠倒，还有字节词表从 0.0003 涨到 0.0175，都可能来自换了一批被重复的文档。

In the right panel of Figure 1, we test an over-vocab setup on 128K. It adds hashed 2- and 3-gram input tables (25.7M rows in total that map adjacent token groups to vectors) and a second 2-gram output head. Its 1-epoch loss drops by about 0.02 bits per byte, but its multi-epoch regret jumps to 0.71, about nine times 128K's. The same panel shows a one-stage H-Net (Hwang, Wang and Gu, 2025). H-Net learns byte chunks directly from bytes without giving them lookup rows. It aims at the length of 128K tokens (4.74 bytes) and reaches about 4 bytes per chunk on the training text. Its regret is 0.031, under two fifths of 128K's 0.081. In a second run that reaches about 6.8 bytes per chunk, longer than 128K's tokens, the regret is 0.015, although the 1-epoch loss is higher (1.432 against 1.373). So compressing the sequence as far as 128K does not by itself create 128K's regret.

在图 1 右栏，我们给 128K 加上哈希 2-gram 和 3-gram 输入表（共 2570 万行，把相邻两三个 token 哈希到表里查向量），再加第二个预测 2-gram 的输出头，记作 over-vocab。它的 1-epoch 损失降了约 0.02 比特/字节，但 multi-epoch regret 暴涨到 0.71，大概是 128K 的 9 倍。同一栏还画了单阶段 H-Net（Hwang、Wang 和 Gu，2025）。它直接从字节学出块，但不给这些块建查表行。它的目标是 128K token 的长度（4.74 字节），在训练文本上实际约 4 字节一块。它的 regret 只有 0.031，不到 128K（0.081）的五分之二。另一次运行实际约 6.8 字节一块，比 128K 的 token 还长，regret 是 0.015，不过 1-epoch 损失更高（1.432，前者是 1.373）。所以把序列压得和 128K 一样短，本身并不会带来 128K 那样的 regret。

Figure 2 tests 130m and 520m models and finds the same vocabulary ordering as 300m. At 520m, multi-epoch regret wipes out 128K's 1-epoch lead over 8K, leaving them tied. Still, this overall average does not tell us where the regret lands: on common tokens, or on rare tokens like " memor" and unfamiliar domains.

图 2 测了 130m 和 520m 两个规模，词表的排序跟 300m 一样。在 520m 上，multi-epoch regret 抹平了 128K 在 1 个 epoch 下对 8K 的领先，最后两者打平。不过光看整体平均分，还看不出 regret 到底是落在常见 token 上，还是落在 " memor" 这类少见 token 和陌生领域上。

Multi-epoch regret lands on unfamiliar text and rarely seen tokensmulti-epoch regret 集中在陌生文本和很少出现的 token 上

Much of 128K's multi-epoch regret sits on tokens that the 8-epoch subset rarely holds. Figure 4 groups evaluation positions by how often the target token appears in the 8-epoch subset. Two effects stand out. First, 128K puts many targets into low-count bins, whereas 99.7 percent of byte targets appear over 10,000 times. Second, at the same count (from a few hundred upward), 128K has up to twice the regret per byte of 8K. Splitting tokens into ten frequency bins shows the same spike in 128K's rarest tenth, whether grouped by input token or predicted token. At 128K, tokens seen fewer than 1,000 times make up 9.5 percent of the text, yet hold 45 percent of the regret. The 8K rank cutoff itself matters less: tokens outside the first 8K are 37 percent of the text and 41 percent of the regret. Half of them appear fewer than 45 times, while " memor" appears 8,849 times. Next, we ask whether the byte vocabulary also degrades if we shrink the subset and run more epochs.

图 4 按目标 token 在 8-epoch 训练子集里的出现次数，把评测位置分组。这里能拆出两件事：第一，128K 把很多目标切进了低频档，而字节词表有 99.7% 的目标出现超过一万次；第二，就算出现次数一样（几百次往上），128K 每字节的 multi-epoch regret 最高也是 8K 的两倍。按训练词频分成十等份，不管是按输入 token 还是预测 token 分，128K 最少见的十分之一都会冒出同样的尖峰。在 128K 里，出现不到 1,000 次的 token 只占文本的 9.5%，却占了 45% 的 regret。前 8K 这条分界线本身反倒没那么关键：8K 之外的 token 占文本的 37%、占 regret 的 41%，其中一半出现不到 45 次，而 " memor" 其实出现了 8,849 次。看到这儿还有个问题：要是把训练子集缩得更小、训更多 epoch，字节词表自己会不会也跟着变差？

The 256-byte vocabulary holds through 8 epochs and degrades from 16256 字节词表扛得住 8 个 epoch，16 个起开始变差

Does the byte vocabulary ever overfit? Yes, just later. In Figure 5, we fix the step count and shrink the training subset to run more epochs. The 256-byte vocabulary stays flat through 8 epochs and degrades from 16 epochs onward. Still, a smaller vocabulary packs fewer bytes into a window and takes more steps on the corpus. Next, we check whether window length or step count explains the vocabulary gap.

字节词表也会过拟合吗？会，只是来得更晚。图 5 固定总步数，不断缩小训练子集：256 字节词表在 8 个 epoch 以内一直很稳，从 16 个 epoch 开始变差。不过小词表每个窗口装的字节少，跑完语料走的步数多。我们还得排查一下，前面看到的词表差距是不是窗口长度或者步数搞出来的。

Window length and step count do not explain the gap窗口长度和步数解释不了词表差距

Neither a shorter window nor taking more steps explains the multi-epoch regret gap. An 8K token has about a quarter fewer bytes than a 128K token, so an 8K window holds a quarter less text and takes a third more steps. Figure 6 tests both factors. On the left, giving 128K the window bytes or step count of 8K leaves 128K's regret at about 0.08 bits per byte. Also, where in the window the regret sits is flat across positions. On the right, stretching the byte window to match a 128K window's text and step count raises 256-byte regret to about 0.02 bits per byte, still under a quarter of 128K's regret. And as Figure 1 showed, at equal steps the byte vocabulary sits level with every vocabulary up to 4K. Even with matched windows and steps, though, the main network sees " memor" as one token instead of six bytes. We still need to check whether sequence compression causes the regret.

差距不是窗口更短或步数更多造成的。8K 的每个 token 比 128K 少约四分之一字节，所以 8K 的窗口少装四分之一文本，步数多出三分之一。图 6 排查了这两个因素。左栏把 128K 的窗口字节数或步数改成跟 8K 一样，128K 的 multi-epoch regret 还是停在约 0.08 比特/字节，regret 在窗口各个位置上的分布也是平的。右栏反过来，把字节窗口拉长，让它装的文本和走的步数都跟 128K 差不多；256 字节词表的 regret 涨到了约 0.02 比特/字节，还是不到 128K 的四分之一。在图 1 里，步数拉平后，字节词表跟 4K 以内的词表处在同一水平。不过就算对齐了窗口和步数，主干网络看到的 " memor" 还是一个 token，而不是六个字节。还得看看是不是序列压短本身惹的祸。

With smaller tables, 128K's multi-epoch regret nears that of smaller vocabularies把 128K 的表变小，multi-epoch regret 就接近小词表

Does multi-epoch regret come from the vocabulary itself or from the extra embedding parameters? Truncating a vocabulary shrinks both at once. Figure 7 shrinks only the parameters by giving 128K low-rank input and output tables (factoring each large table into two smaller matrices). This keeps 128K's tokens, windows, steps, and 8-epoch document subset intact. Tables of rank 96, 192 and 384 match the parameter counts of the 16K, 32K, and 64K tables. They cut regret to 0.039, 0.053, and 0.074, down from 0.081 for plain 128K. That drop is the firm part. The match to a specific vocabulary size is looser: with equal text per window, 16K and 32K have regrets of 0.053 and 0.039, so the gaps between each rank and its matching vocabulary lie within what a different subset of documents can move. Because the compressed sequences stay identical while regret drops by up to half, sequence compression alone does not set the regret. Also, rank 96 fits unique text 0.043 worse than 16K yet has about the same regret, pointing to parameter count rather than fit quality. Low-rank tables are no fix, though. After 8 epochs, ranks 96 (1.507) and 192 (1.478) end up worse in validation loss than plain 128K (1.467), and rank 384 finishes about even (1.471). Each rank has one run.

multi-epoch regret 来自词表变大本身，还是来自多出来的 embedding 参数？直接截断词表会把词表和参数一起变小，分不清是谁的作用。所以图 7 只砍参数：给 128K 换上低秩输入表和输出表（把大矩阵拆成两个小矩阵相乘来省参数），保留原本的 token、窗口、步数和 8-epoch 文档。秩为 96、192 和 384 的表，参数量分别跟 16K、32K 和 64K 的表一样；它们的 regret 降到了 0.039、0.053 和 0.074，而普通 128K 是 0.081。这个下降是可靠的。但具体跟哪个词表对齐就没那么准了：每个窗口装同样文本时，16K 和 32K 的 regret 是 0.053 和 0.039，所以各个秩跟对应词表的差距，都在换一批文档就能挪动的范围内。序列压得一样短，regret 却最多降了一半，说明光靠序列压短定不了 regret。另外，秩 96 在不重复文本上比 16K 差 0.043，但两者的 regret 差不多，这更支持参数量的解释，而不是拟合好坏的解释。不过低秩表不能拿来救场：训完 8 个 epoch，秩 96（1.507）和 192（1.478）的 8-epoch 验证损失比普通 128K（1.467）更差，秩 384（1.471）与它大致持平。每个秩只有一次运行。

Regularizing rare input rows and weights removes about half of 128K's multi-epoch regret约束少见输入行和权重，能消掉约一半 128K multi-epoch regret

We suspect that much of 128K's multi-epoch regret comes from rare tokens' input rows. The intuition is simple: the input row of " memor" updates only when " memor" appears, so across epochs it sees the exact same contexts over and over. To test this, we build each rare input row from the mean of its 8K pieces' rows (using " mem" and "or" for " memor"). This removes nearly half of the regret 128K has beyond 8K. Freezing rare input rows after epoch 1 removes nearly two fifths. However, both fixes get much of this by fitting unique text worse: piece-built rows get 1.4092 on unique text versus 1.3854 for plain 128K, while their 8-epoch loss moves only from 1.4667 to 1.4615. Technique D below changes 1-epoch loss by just 0.006. Yet an untargeted control that swaps in random input tokens starting at epoch 5 reaches nearly the same 8-epoch loss (1.446 versus 1.447). So these runs do not isolate the rare input rows. Figure 8 shows the 81 remedy setups we tested at 128K.

我们猜 128K 的 multi-epoch regret 有很大一部分出在少见 token 的输入行上。道理很简单：" memor" 的输入行只有碰到 " memor" 才更新，多训几个 epoch 就会反复看同一批上下文。为了验证，我们试着拿 8K 片段输入行的均值拼出少见输入行（比如 " memor" 就取 " mem" 和 "or" 的均值），这能消掉 128K 超出 8K 那部分 regret 的近一半；在第 1 个 epoch 后冻住少见输入行，也能消掉近五分之二。但这两种固定做法之所以能缩窄差距，很大程度是因为把不重复文本的拟合搞差了：用片段拼出来的输入行在不重复文本上是 1.4092（普通 128K 是 1.3854），而 8-epoch 损失只从 1.4667 微降到 1.4615。下面要讲的技术 D 只让 1-epoch 损失变了 0.006；但我们还跑了个不挑少见 token 的对照，从第 5 个 epoch 起随机换掉输入 token，8-epoch 损失也差不多（1.446 对 1.447）。所以这些实验还没有把少见输入行单独区分出来。图 8 画了我们在 128K 上试过的 81 种补救配置。

Five techniques drive the progress in Figure 8. Technique A adds a slowly updated residual to each piece-built rare input row. Technique B resamples token splits each epoch using BPE-dropout (randomly skipping merges so the same text splits into different tokens) without hurting 1-epoch fit. Technique C shuffles document order each epoch. Used alone, Technique A still hurts unique-text fit, and Techniques B and C each cut less than a fifth of the multi-epoch regret. Technique D replaces a rare input row with the mean of its 8K pieces (" mem" and "or" for " memor") on half of its training occurrences, blocks gradients to the pieces, and uses the token's own row at evaluation. Most of D's gain comes from blocking the rare row's own gradient on that half of occurrences. The pieces' mean keeps the context intact. Over three runs, random replacement with piece gradients enabled gets 1.4474 after 8 epochs and 1.3916 at 1 epoch; with piece gradients blocked, it gets 1.4440 and 1.3934.

图 8 的进展靠五项技术推进。技术 A 在拼好的少见输入行上加一个慢速更新的残差；技术 B 每个 epoch 用 BPE-dropout（合并时随机跳过一些规则，让同一句话每次切出不同的 token）重新切分，且不伤 1-epoch 拟合；技术 C 每个 epoch 打乱文档顺序。单拉出来用时，技术 A 还是会伤不重复文本的拟合，技术 B 和 C 各自只能消掉不到五分之一的 multi-epoch regret。技术 D 在训练时把少见输入行一半的出现位置换成 8K 片段的均值（" memor" 就换 " mem" 和 "or" 的均值），同时拦住传向片段的梯度，评测时再换回它自己的输入行。主要收益其实来自在那一半位置拦住了少见输入行自己的梯度，片段均值则让上下文保持完整。放开片段梯度时，随机替换跑三次运行的均值是 8-epoch 1.4474、1-epoch 1.3916；拦住梯度后分别是 1.4440 和 1.3934。

Our two best setups cut about half of 128K's multi-epoch regret. The first is Technique E, which adds an exponential moving average of the weights at evaluation on top of C and D. It brings 8-epoch loss down to 1.430, just 0.045 bits per byte above plain 128K's 1-epoch loss on unique text, cutting 45 percent of the regret (one run; a close variant with double the learning rate gets 1.432 over two runs, or 43 percent). The second setup combines tied input-output tables, a warmup-stable-decay learning-rate schedule (warmup, constant plateau, then decay), and the same weight averaging. It reaches 1.425 in one run, cutting 51 percent of the regret. The gap between 1.425 and 1.430 is within single-run noise, so we treat the two setups as tied. Both slightly hurt unique-text fit (1.3928 and 1.3891 versus 1.3854), and both leave about half the regret intact. On the output side, freezing rare output rows after epoch 1 hurt 1-epoch fit, while self-distillation and tied tables helped a bit. Our experiment page lists each technique's individual results. All these runs use MuonH, which has no weight decay.

目前最好的两套方案能消掉约一半的 128K multi-epoch regret。一套是技术 E：在 C 和 D 的基础上，评测时改用权重的指数滑动平均，把 8-epoch 损失压到 1.430，相对普通 128K 不重复文本的差距只剩 0.045 比特/字节，消掉了 45% 的 regret（一次运行；学习率翻倍的相近变体跑了两次运行，落到 1.432，即消掉 43%）。另一套把共享输入输出表、warmup-stable-decay 学习率调度（先预热、再恒定、最后衰减）和同样的权重平均绑在一起，单次运行落到 1.425，消掉了 51%。1.425 和 1.430 的差距在单次运行的噪声以内，我们算它们打平。这两套方案都会轻微拉低不重复文本的拟合（1.3928 和 1.3891，普通 128K 是 1.3854），也都还剩下一半左右的 regret。在输出端，第 1 个 epoch 后冻住少见输出行会伤 1-epoch 拟合，自蒸馏和共享输入输出表能帮上一点忙。实验主页记下了每项技术单跑的结果，全都用的没有权重衰减的 MuonH。

Optimal weight decay grows with the epoch count训练 epoch 越多，最优权重衰减越大

Limitations局限性

Our experiments have clear limits. Most runs use a 300m model on FineWeb-Edu for 8 epochs with MuonH, evaluated only on Paloma macro bits per byte. Our smaller BPE vocabularies simply truncate the 128K tokenizer, which also shrinks embedding parameter counts when tables are untied. Both runs in a setup use the same subset of documents, so gaps below about 0.02 bits per byte across vocabularies can come from which documents fall in that subset. Gaps below about 0.01 between single runs sit within the noise. Finally, we pick our best setup using the same Paloma score we report.

说一下实验的局限。大部分运行都只跑了 300m 模型、FineWeb-Edu 数据、8 个 epoch 和 MuonH。打分只看 Paloma 宏平均比特/字节。小号 BPE 词表都是直接截断 128K 分词器得来的，在不共享输入输出表时 embedding 参数也跟着变少。同一套配置的两次运行重复的是同一批文档，所以不同词表之间小于约 0.02 比特/字节的差距，可能只是因为 8-epoch 子集抽到了不同的文档。单次运行之间小于约 0.01 的差距在噪声以内，分不出高下。另外，挑最优配置和报最终结果用的是同一个 Paloma 分数。

Takeaways核心结论

To sum up: a large vocabulary gives rare strings like " memor" their own rows, which helps on unique text, but much of the multi-epoch regret lands on rarely seen tokens. On a 300m model trained for 8 epochs on fixed text with equal text per window, vocabularies from 256 bytes to 4K have 0.02 to 0.03 bits per byte of regret, while 128K BPE has about 0.08. This regret opens up in the second half of training, and much of it sits on tokens that rarely appear in the 8-epoch text. Our two best setups each cut about half of the 128K regret (45 and 51 percent, one run each).

小结一下：大词表给 " memor" 这种少见字符串分了专属行，在不重复文本上确实有用；但多训几个 epoch 后，multi-epoch regret 有很大一部分落在了很少出现的 token 上。300m 模型在固定文本量上训 8 个 epoch、且每个窗口装同样文本时，256 字节到 4K 词表的 regret 是 0.02 到 0.03 比特/字节，128K BPE 约 0.08。这个差距在训练后半程拉开，其中很大一部分落在 8-epoch 文本里少见的 token 上。我们最好的两套方案各消掉了约一半的 128K regret（45% 和 51%，各一次运行）。

Our practical advice: pick your vocabulary size with the epoch count in mind. If you must train a large vocabulary for many epochs, randomly replace rare input rows with the mean of their 8K pieces during training, shuffle the text each epoch, and average the weights. Together, those three steps cut 45 percent of the multi-epoch regret in one run. Alternatively, tying the input and output tables, using a warmup-stable-decay schedule, and averaging the weights cut 51 percent in one run.

给实际训练的建议：定词表大小时，先想想数据要重复训几个 epoch。如果非得拿大词表训很多个 epoch，可以在训练时随机把少见输入行换成 8K 片段的均值、每个 epoch 重新打乱文本，再做权重滑动平均；这三招合起来在一次运行中消掉了 45% 的 multi-epoch regret。或者共享输入输出表、用 warmup-stable-decay 学习率调度再加权重滑动平均，单次运行消掉了 51%。

Open questions开放问题

Beyond these regularization techniques, what else can lower multi-epoch regret in this setup? Even our best setups leave about half of it.

除了这些正则化技术，还有什么办法能在这个设置下降低 multi-epoch regret？最好的方案仍然留下了约一半。

Could a dynamic tokenizer like H-Net work well here? In one run, our one-stage H-Net has under two fifths of 128K's multi-epoch regret, with chunks of about 4 bytes against about 4.7 for 128K's tokens. We leave a detailed study to future work.

H-Net 这样的动态分词器会不会很适合这种训练？在一次运行中，我们的单阶段 H-Net 块长约 4 字节（128K 的 token 约 4.7 字节），multi-epoch regret 不到 128K 的五分之二。详细研究留作后续工作。

Related work相关工作

Acknowledgements致谢

Compute came from Princeton Language and Intelligence on the Della cluster. We thank the Marin project for its codebase, tokenizer, and training recipe, and Xingyu Dang for suggestions on the weight decay runs.

实验算力来自普林斯顿语言与智能中心（PLI）的 Della 集群。感谢 Marin 项目的代码库、分词器和训练配方，感谢 Xingyu Dang 对权重衰减实验的建议。

References参考文献

Citation

@misc{chai2026vocabmultiepoch,
  title        = {Larger Tokenizers Suffer More Multi-Epoch Regret},
  author       = {Chai, Wenhao},
  year         = {2026},
  howpublished = {Blog post},
  url          = {https://wenhaochai.com/blogs/vocab-multi-epoch.html}
}