DFlash 2 — 3× more tokens on Qwen3.8-27B, same output

I wrote Qwen 3.8 27B because the model is finally good enough to stay local. The next ceiling isn’t quality — it’s throughput. An agent that reads, plans, and calls tools for hours burns tokens at a rate chat never did. Every token, in classic decoding, costs a full forward pass. DFlash 2 (Inco AI, 18 Aug 2026) hits exactly that: parallel draft + one-pass verify, output provably unchanged vs the target.

The Inco AI post →

Why I care

On a 27B the bottleneck is no longer “is the model strong enough” — it’s “can I serve it fast enough for an agent”. DFlash 1 (Jan 2026) already ships in SGLang, vLLM, TensorRT-LLM, and llama.cpp: NVIDIA measured up to 15× throughput on Blackwell, Google 3× tok/s on TPUs, and CoreWeave’s Kimi K2.7 Code endpoint — fastest for that model on Artificial Analysis — runs DFlash by default. 3.5M Hugging Face downloads. DFlash 2 doesn’t invent a new stack: it recovers the slack still left in parallel drafting.

• The draft is no longer autoregressive: the whole block, every position, in one pass.
• DFlash 2: +16–25% acceptance length vs DFlash, ~1% extra cycle latency.
• On Qwen3.8-27B: 2.7–3.4× autoregressive throughput (batch 1), same text.
• Two day-one drafters: incoai/Qwen3.8-27B-DFlash2 and Muse Glimmer.

The idea, without the paper

Classic speculative decoding: a small model guesses a block, the large one verifies it in one forward. Good guesses → several tokens per pass. Bad ones get dropped. For years the draft itself stayed one token at a time. DFlash made drafting one-pass. DFlash 2 plugs the two leaks that remained: wrong pick among good candidates, and suffix decay (the tail of the block dies).

The number that sold me: on a 5-layer DFlash (Qwen3-4B, GSM8K), top-1 is right 85.4% of the time at position 0 — but the right token is in the top 16 99.5% of the time. The drafter already knows. You just need a coherent path through those lists. A top-16 oracle would lift acceptance from 4.27 → 6.79. DFlash 2 adds a pairwise path selector (+2M params, +0.6% latency) instead of an autoregressive correction head like DSpark (+77.8M, +9.6%). *Choosing is cheaper than predicting.*

Suffix decay they treat as a local problem: a two-tap conv (Canon / short-conv style) before and after every attention and MLP. +3% params, +0.7% latency, and a 5-layer almost matches a 15-layer. Selector + conv together: +1.3% cycle for +1 accepted token per pass on average.

The numbers that matter on Qwen3.8-27B

That’s the drafter I’d actually plug in. Versus the model’s native MTP and a community DSpark, on Inco’s benches (block 8, official Qwen sampling):

• Mean acceptance: MTP 4.28 · DSpark 3.62 · DFlash 2 4.80.
• GSM8K: 5.46 vs MTP 5.02. MATH-500: 5.28 vs 4.72. MBPP: 4.79 vs 3.99.
• Batch-1 throughput (H200, SGLang, model card): 2.67–3.43× vs AR — 184–236 tok/s where AR sits at ~69.
• Concurrency 8: still 2.3–2.8×. At 32 the gain collapses (1.0–1.45×) — expected: the batch already fills the GPU.
• Muse Glimmer: 3.1–4.6×, mean 5.70 vs official DFlash 4.44.

Lossless: greedy matches the target exactly; sampling via rejection sampling keeps the same distribution. You’re not trading quality for speed. You pay ~1% cycle for one extra token per verify.

Run it

Already wired into SGLang, vLLM (PR), llama.cpp (PR), Ollama (PR), and oMLX. Shortest path today: SGLang + the Hub drafter:

pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
  --speculative-num-draft-tokens 8

• vLLM: method: dflash, num_speculative_tokens: 7 — still on a PR (vllm#52816).
• llama.cpp: --spec-type draft-dflash + GGUF incoai/Qwen3.8-27B-DFlash2-GGUF.
• Mac: oMLX (prebuilt) or experimental Ollama, draft incoai/Qwen3.8-27B-DFlash2 on an MLX 4-bit target.
• The drafter is not a standalone LLM (~2B). It only runs inside a speculative server.

Where it lands in my stack

On the Qwen 3.6 server and 3.8 in Studio / MLX, an agent’s real cost is tok/s × loop time. 3× at batch 1 is a Hermes / MCP session that fits the window instead of timing out. At high concurrency the gain melts — if you already serve a full batch, DFlash 2 isn’t magic. If you serve one agent at a time (local / one card), that’s the lever.

Takeaway

DFlash 2 isn’t a new model. It’s a drafter that makes the 27B you already run usable for agents without changing a prompt or the output distribution. Inco is blunt: inference hasn’t hit the floor. For me the real test is wiring incoai/Qwen3.8-27B-DFlash2 onto the Qwen3.8 I already serve — and measuring tok/s, not the blog.

• Source: inco.ai/blog/dflash2/
• Drafter: huggingface.co/incoai/Qwen3.8-27B-DFlash2
• Qwen 3.8: /blog/qwen-3-8-27b
• Local Studio: /blog/unsloth-studio