On June 27, DeepSeek published a paper called DSpark in the GitHub repository deepseek-ai/DeepSpec. The release included MIT-licensed training code and two ready-to-use sets of weights. The paper also drew more than 700 points on Hacker News.
DSpark tackles a familiar fork in speculative decoding. In speculative decoding, a small draft model guesses a block of tokens, and the target model verifies them in one forward pass. Correct guesses are essentially free; wrong ones are rolled back. The approach has been popular for a couple of years, but it has been stuck on a trade-off:
- For accuracy, the draft model has to generate token by token, as Eagle3 does. The guesses are accurate, but drafting cost grows with block length.
- For speed, the draft model can emit a whole block in parallel, as DFlash does. It is fast, but each position guesses independently, and acceptance falls sharply toward the end of the block.
Fast but inaccurate, accurate but slow. DSpark tries to remove that either/or. Its answer is to use both routes: a parallel backbone for speed and a very light serial head for accuracy, plus a scheduler that watches the GPU and decides how many tokens to verify each round. On real DeepSeek-V4 traffic, the paper reports a 60–85% per-user generation speedup over the previous MTP-1 baseline at equal throughput, with lossless output.
One point should be clear up front: DSpark is not a new model. It is a serving optimization. It reuses existing DeepSeek-V4 weights and adds a draft module in front. It does not change the model's capabilities; it changes how tokens are emitted.
The underlying arithmetic of speculative decoding
To understand why DSpark is designed this way, start with the cost accounting. There are two roles: a draft model and a target model. The draft model proposes a block of tokens, and the target model runs one forward pass to verify them. Verification uses rejection sampling: accept the longest valid prefix, then add one extra token. Because this rule preserves the target distribution exactly, quality should not drop in theory. Speed may improve, or the work may be wasted.
Per-token latency can be written as:
L = (T_draft + T_verify) / τ
T_draft is drafting time, T_verify is verification time, and τ is the number of accepted tokens per cycle. To reduce L, there are only three levers: make drafting faster (lower T_draft), make drafting more accurate (raise τ), and make verification smarter (avoid spending T_verify on tokens that are bound to be rejected). Earlier methods usually pulled one or two of these levers. DSpark pulls all three.
A parallel skeleton and a small serial head
DSpark's draft generation has two stages. First, a parallel heavyweight backbone (DFlash in the paper) produces base logits for every position in the block at once. This inherits the parallel route's main advantage: draft cost stays nearly flat as the block grows, and the first token is highly accurate. Second, a lightweight serial head adds a context-dependent bias at each position before sampling. By default this is a Markov head that looks only at the immediately preceding token and uses a rank-256 low-rank decomposition to keep parameter count small; even with a large vocabulary it is cheap. In the paper's example, after the first position samples of, the head raises the probability of course and lowers problem. In other words, later guesses get to glance at what came before.
The effect accumulates position by position: the parallel backbone protects hit rate at the start, while the serial head holds acceptance deeper into the block instead of letting it collapse in the second half as pure parallelism does. The paper mentions an optional RNN head that can track the whole prefix, but the gain is marginal, so the released version uses the Markov head.
One training detail matters: the target model is frozen, and the draft head reuses its embedding and output head. The key loss is total-variation loss. Minimizing that distance is equivalent to directly maximizing draft acceptance, so the objective is well aligned.
On Qwen3 4B/8B/14B, DSpark's average acceptance length is 30.9% / 26.7% / 30.0% higher than Eagle3 and 16.3% / 18.4% / 18.3% higher than DFlash. A 2-layer DSpark beats a 5-layer DFlash. The serial head is almost free: moving the draft block from 4 to 16 adds only 0.2–1.3% latency per round, while acceptance length can rise by up to 30%.
Verification that watches the GPU
Making drafts more accurate is not enough. In real high-concurrency serving, guessing more tokens does not automatically mean faster inference. Verifying tokens that are bound to be rejected wastes valuable batch capacity. DSpark adds two parts here.
The first is a confidence head. It scores each draft position and estimates whether that token will survive verification. Supervision comes from parsed stepwise acceptance rates. Neural networks tend to be overconfident, so the authors add Sequential Temperature Scaling as post-hoc calibration, pushing expected calibration error (ECE) from 3–8% down to about 1%.
The second is a hardware-aware prefix scheduler. At startup it measures an offline throughput curve SPS(B), where B is batch size, and uses it to allocate verification length dynamically per request. When the GPU is idle, it verifies more tokens; when the GPU is busy, it verifies fewer to protect throughput. The scheduler includes an early-stop rule to guarantee losslessness. An appendix counterexample shows that a naive global search would leak information, so the rule is not decorative.
Behavior varies with load. Under moderate load, the scheduler reportedly verifies about 4–6 tokens per request; as concurrency rises, it cuts the budget. This is a key difference from earlier generations: Eagle3 and DFlash use essentially fixed verification lengths, while DSpark is dynamic.
Together, the two moves touch all three levers: the parallel backbone lowers T_draft, the serial head and confidence head raise τ, and load-aware scheduling saves T_verify.
How it relates to MTP
Both MTP and DSpark want to generate multiple tokens at once, but their approaches differ. MTP tends to make the model architecture itself support multi-token output. DSpark is a pure inference-time speculative decoding framework that sits outside an existing model. More interestingly, the production baseline in the paper is MTP-1, DeepSeek's earlier single-token setup. DSpark is meant to replace MTP-1, and the 60–85% speedup is measured against it.
The two released weight sets, DeepSeek-V4-Pro-DSpark and DeepSeek-V4-Flash-DSpark, are attached to existing V4 weights. The Hugging Face cards include minimal inference examples; the target model does not need retraining. The release configuration is called DSpark-5: a 5-token draft block with the Markov head.
The barrier to reproducing it is high
The training code, DeepSpec, is MIT-licensed and runs in three stages: data preparation, training, and evaluation. The default setup assumes a single machine with 8 GPUs. Evaluation covers nine datasets including gsm8k, math500, aime25, humaneval, mbpp, and livecodebench, and the target models include Qwen3 and Gemma. The repository also bundles three draft models—DSpark, DFlash, and Eagle3—for side-by-side comparison.
There is a practical catch: the target cache needed to reproduce training is enormous. Under the default Qwen3-4B setting, the paper and repository indicate it is roughly 38 TB. Most teams will use the released weights and run inference rather than train a draft model from scratch.
Returns also vary by task. Code generation already has high acceptance, and the scheduler is willing to verify longer prefixes, so streaming output from coding agents becomes noticeably smoother. For open chat, after one round of confidence-threshold adjustment, acceptance rises from 45.7% to 95.7%. Math reasoning sits in between, rising from 76.9% to 92.5%. The more structured and predictable the workload, the more it benefits.
Four parts, two jobs
Returning to the opening either/or: DSpark actually contains four cooperating pieces, and their boundaries are clear.
- The draft model handles guessing. DSpark splits it into a parallel backbone and a serial head, taking speed and accuracy separately.
- The target model handles judging. It only verifies and does not change the output distribution; that is where losslessness comes from.
- The confidence head scores each guess and tells the system which tokens are worth verifying.
- The load scheduler watches the GPU and adjusts verification length between busy and idle periods, spending compute where it matters.
The split is simple: draft and target govern quality, while confidence head and scheduler govern cost. DSpark's real contribution is not inventing speculative decoding. It is retuning each stage of the pipeline so that fast, accurate, and cheap no longer have to fight one another.
Zoom out, and the 2026 open-model inference race has quietly shifted from who has the stronger model to who can serve it more cheaply. MTP routes, speculative decoding routes, and various draft-model designs are all competing for the same cost budget. DSpark offers a fairly complete open sample that has already run on real DeepSeek-V4 traffic. Whether it can reproduce that 60–85% on other models and hardware will depend on lighter-weight community reproductions that do not require 38 TB.