GLM-5.3-Flash vs Qwen3.8-Flash-Next: Two Labs, One Recipe

Two frontier open-weight models shipped within roughly a day of each other this week. Z.ai released GLM-5.3-Flash, a 320B-parameter multimodal MoE with 18B active parameters. Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125B model with 6B active parameters that previews the Qwen4 architecture.

The two teams designed these systems independently. Their configs read like near-copies anyway. Both use a 3:1 hybrid of linear and full attention. Both pick context with a compressed indexer capped at 2048 tokens. Both widen the residual stream into four gated branches. Both train with the Muon optimizer, splitting fused parameter matrices before orthogonalization. Convergent evolution is normal in machine learning; this is convergent evolution on a tight, specific recipe.

The two releases in brief

GLM-5.3-Flash is the first natively multimodal model in the GLM-5 series, released under the MIT license on Hugging Face. Before the reveal, Z.ai tested it anonymously on OpenRouter — the stealth run became the most popular model of the week, and the community, not the lab, broke the story. It was trained on a 30T-token multimodal corpus and serves a 1M-token context window. Z.ai says it beats GLM-5.2 across benchmarks at one-tenth the price while approaching Claude Opus 4.8 on coding and agentic workloads. List pricing: $0.15/M input, $0.50/M output tokens.

Qwen3.8-Flash-Next plays the role Qwen3-Next played for Qwen3.5: an early public preview of the next architecture family. The model card lists a 125B main model plus a 51B n-gram embedding table, with 6B parameters activated per token. Native context is 262,144 tokens, extensible to 1M with YaRN. Training reputedly cost about one-ninth the compute of Qwen3.7-Plus. Its technical report is titled "On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability." Treat every performance and price claim here as vendor-stated — independent benchmarks for Flash-Next barely exist yet, as the model gap analysts note.

Shared trick 1: three of every four attention layers are linear

Attention is the expensive part of a transformer, and the KV cache is what grows with your context. Both labs threw the bulk of it away. GLM-5.3-Flash stacks 45 layers — 34 linear-attention layers and 11 full-attention ones, per the shipped config already merged into llama.cpp. Qwen3.8-Flash-Next stacks 48 layers as a repeating block of 3 Gated DeltaNet layers plus 1 Qwen Sparse Attention layer, per the vLLM recipe. Same 3:1 ratio.

Linear layers get their name from the trick: instead of a KV cache that grows with input length, they compress all history into a fixed-size recurrent state, so compute per token stays flat no matter how long the prompt gets. GLM uses Kimi Delta Attention (KDA), the design Moonshot AI introduced in Kimi Linear, with a fine-grained per-channel decay gate. Qwen uses its own Gated DeltaNet (GDN) gated at the per-head level. Different granularity, same delta-rule family, same job.

The remaining quarter of layers handle precise long-range retrieval. GLM uses DeepSeek-style NoPE multi-head latent attention (MLA). Qwen uses grouped-query attention inside QSA. That is where the KV cache actually lives — and where the second shared trick kicks in.

Shared trick 2: compress 4x, then keep 2048 tokens

Neither model lets its full-attention layers attend over the entire context. Both attach a small learned indexer that scores chunks of history and keeps only the winners.

GLM's sparse layers use a 32-head lightning indexer with top-2048 selection, descended from DeepSeek's DSA. To tame indexer cost at 1M-token contexts, Z.ai introduces IndexPool, which compresses four indexer key vectors into one via weighted pooling before scoring. Qwen's QSA works at micro-block granularity: a lightweight indexer scores 4-token blocks and keeps the top 512 — which is exactly 2048 tokens. Two implementations, one budget. Qwen credits QSA with up to 7.6x prefill and 4.9x decoding speedups over full attention at 1M tokens, via an NVIDIA reference on GB300 NVL72.

The structural idea, side by side:

# What actually shipped into the weights
glm-5.3-flash:
  layers: 45
  linear: 34     # Kimi Delta Attention, per-channel decay gate
  full: 11       # NoPE MLA  (qk_rope_head_dim = 0)
  sparse_indexer: lightning 32-head, top-2048
  residual: mHC, 4 gated branches

qwen3.8-flash-next:
  layers: 48
  linear: 36     # Gated DeltaNet, per-head gate
  full: 12       # QSA, grouped-query
  sparse_indexer: micro-block 4-token, top-512 blocks (= 2048 tokens)
  residual: Gated Residual, 4 branches

On GLM's side the payoff is large. Versus the full GLM-5.3, Z.ai reports the Flash architecture cuts attention compute roughly 3x and KV cache size 4.4x, while nearly halving active parameters (18B vs 32B) and layers (45 vs 92).

Shared trick 3: four residual streams instead of one

Both models abandon the single residual stream that has defined transformers since 2017 and widen it into four parallel branches, with gates controlling what each block reads and writes.

GLM adopts Manifold-Constrained Hyper-Connections (mHC), a DeepSeek-originated design, tuned to 4 branches in the shipped weights. Qwen wrote its own variant, Gated Residual, which modulates flow through 4 widened streams with an element-wise data-dependent read gate and a per-branch scalar write gate. Qwen argues Gated Residual drops the extra branch-mixing step Hyper-Connections needs, cutting memory-access overhead, and that the gates suppress activation outliers well enough for FP8 residual storage. The interesting part: the team ablated both approaches and found them roughly equal. Two labs, two implementations, one verdict — four gated streams beat one.

Shared trick 4: Muon, with fused matrices split first

Both models train with the Muon optimizer. And both apply the same subtle refinement: fused projection matrices get split back into their independent transformations before Muon orthogonalizes them. Qwen documents splitting fused QKV, SwiGLU, and GDN projections — Muon on genuine 2-D linear maps, AdamW on embeddings, routers, and low-rank parameters. They also refit their scaling laws for the new architecture and dropped batch-size warmup entirely after measuring that warmup cost 18.8% more optimizer steps without improving results. When two labs independently delete a training ceremony that everyone assumed was standard, that is worth noting.

Where they disagree: positional encoding

The one clean split is rotary position embeddings in the full-attention layers. GLM-5.3-Flash drops them: the config sets qk_rope_head_dim = 0, making its sparse MLA layers fully NoPE. Position info flows implicitly through the recurrent linear layers.

Qwen tried the same thing and kept RoPE. Their technical report explains why: NoPE produced no measurable difference during pre-training, but after post-training the NoPE variant often failed to stop generating. That is a genuinely useful cautionary result for the field. Pre-training loss curves can hide behavioral defects that only show up after RLHF-stage tuning.

MiniMax says no

The recipe is bigger than two labs. DeepSeek pioneered the sparse-indexer-plus-2048-budget pattern with DSA in DeepSeek-V3.2-Exp, and mHC is a DeepSeek design now shipping in GLM. Moonshot's Kimi supplied the exact linear-attention layer GLM adopted. Chinese open models are cross-pollinating and landing on shared settings.

The notable dissenter is MiniMax. During M2 development the team tested linear and sliding-window attention at scale and found severe deficits in multi-hop reasoning, especially beyond 32K context after SFT — so M2 shipped with full softmax attention in every layer. For M3 they adopted MiniMax Sparse Attention, which sparsifies softmax attention via block selection without a single linear-attention layer. So the field has not fully settled: Z.ai, Qwen, DeepSeek, and Kimi are betting a 3:1 linear hybrid preserves reasoning; MiniMax's ablations say it does not, at least on their stack.

What to watch next

For most developers, the practical takeaway is simple: the cheap-and-fast tier of open models is getting structurally cheaper. A 6B-active model being that much of a budget headline is itself the story — the same pattern we covered with tokenizers and GGUF tooling playing out at the model-architecture layer.

Two things are worth tracking. First, what Qwen4 does when it lands on top of the Flash-Next blueprint — the Qwen3-Next gamble paid off across three full model series, so this preview is more than a teaser. Second, whether the 3:1 recipe holds up under independent evaluation; right now GLM-5.3-Flash has real outside benchmarks while Flash-Next's record is still self-reported. If the ratio survives scrutiny, we will see a lot more 3:1 hybrids in 2027.