GLM-5.2 Q4: prefill fails at ≥64K context on M3 Ultra 512GB (KV payload staging failed)

[wbllcbbg-ai]

Environment

• Hardware: Apple M3 Ultra, 512GB unified memory, 819GB/s bandwidth
• Model: GLM-5.2-UD-Q4_K_RoutedQ4K.gguf (404GB, from antirez/GLM-5.2-GGUF on HuggingFace)
• Engine: ds4-server, built from latest main (Metal backend)
• Context: --ctx 524288 (512K)
• Command: ./ds4-server -m GLM-5.2-UD-Q4_K_RoutedQ4K.gguf --ctx 524288 --port 8321 --kv-disk-dir ~/.ds4/server-kv --kv-disk-space-mb 262144

What works

• Short context (2K, 16K): prefill and decode work correctly
  • 2K: prefill 116 t/s, decode 16.0 t/s
  • 16K: prefill 112 t/s, decode 14.2 t/s
• Memory is sufficient (model 404GB + graph ~50GB = ~455GB, well within 512GB)
• Quality: needle tests pass when thinking mode is disabled

What fails

At ≥64K context, prefill fails with:

ds4-server: kv cache skipped tokens=10240 reason=continued because KV payload staging failed: se...
ds4-server: kv cache discarded reason=prefill-failed

After the failure, the engine stops responding to new requests (appears hung).

Reproducible on a fresh process. Same hardware runs DeepSeek V4 Flash MXFP4 at 1M context without issues.

Log excerpt

0907 02:11:17 ds4-server: kv cache skipped tokens=10240 reason=continued because KV payload staging failed: se...
0907 02:35:35 ds4-server: chat ctx=2048..79031:76983 prompt start
0907 02:35:35 ds4-server: kv cache discarded reason=prefill-failed file=...kv

Notes

• This might be related to GLM's IndexShare architecture or the compact DSA KV layout at larger contexts
• The model loads fine and memory guard passes at 512K ctx (required: ~455GB, budget: 480GB)
• Happy to provide more logs or test patches

[wbllcbbg-ai]

Update: resolved on current main (f62ca29) on the same M3 Ultra 512GB.

Reproduced the original failure on my previous build (base 84cc882): prefill at ≥64K died with KV payload staging failed. After rebuilding at f62ca29 and launching with --power 100 (as the README now requires for GLM):

• 2K: prefill OK, decode ~15 t/s
• 16K: prefill ~131 t/s avg, needle retrieval 4/4
• 64K: full 77.5K-token prefill completes (~79–131 t/s), decode 11.2 t/s at 64K context, needle retrieval 5/5, server stays healthy across the whole run

I can't cleanly separate whether it was the attention scratch alignment fix (fb2abb2), the SSD cache priority work, or --power 100 itself — but the combination works on this machine. Happy to provide the old logs if useful; otherwise feel free to close.