fix(gdn): only reject THD packed_seq_params, allow BSHD#52
Merged
Conversation
Zhichenzzz
force-pushed
the
fix/1292-gdn-bshd-packed
branch
from
8b0dd99 to
2dc2612
Compare
Megatron passes a bshd-format PackedSeqParams uniformly even when no real packing is happening, which GDN can safely ignore. Restrict the NotImplementedError to genuine THD packing instead of any packed_seq_params, so GDN runs under the default bshd path.
Zhichenzzz
force-pushed
the
fix/1292-gdn-bshd-packed
branch
from
c2ab1aa to
fc7ae0e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GatedDeltaNet.forwardblanket-rejects any non-Nonepacked_seq_params, but Megatron passes aPackedSeqParamsobject to every attention module uniformly — even in BSHD mode (qkv_format="bshd"). This crashes Qwen3.5/3.6 GDN withNotImplementedError: GDN does not support packed sequence for now.Fix
Guard on
packed_seq_params.qkv_format == "thd"specifically, matching upstream NVIDIA/Megatron-LM PR NVIDIA#2645. BSHD now proceeds (GDN ignores the params and computes seq_len from shapes); genuine THD packing still raises (full THD support is a follow-up port of PR NVIDIA#2645).Validation
Built the real
GatedDeltaNeton GPU and exercised the guard with realPackedSeqParams:GDN does not support packed sequenceFixes radixark/miles#1292