Skip to content

[codex] Fix Qwen3.5 MRoPE VLM model#2819

Draft
eligotts wants to merge 1 commit into
mainfrom
fix/qwen35-mrope-model
Draft

[codex] Fix Qwen3.5 MRoPE VLM model#2819
eligotts wants to merge 1 commit into
mainfrom
fix/qwen35-mrope-model

Conversation

@eligotts

Copy link
Copy Markdown
Contributor

Summary

Fixes the custom Qwen3.5 MoE VLM path to match the MRoPE behavior expected by the upstream/HF-vLLM-style implementation.

  • Adds a Qwen3.5 MRoPE position builder that uses renderer-provided mm_token_type_ids plus image_grid_thw instead of trainer 1D positions.
  • Replaces the generic rotary embedding path with Qwen3.5-compatible interleaved MRoPE handling.
  • Makes VLM forward compute image MRoPE internally, reject invalid 2D image positions, validate image token/feature counts, and accept explicit packed seq_lens for future packed multimodal batches.
  • Adds targeted unit coverage for text-only positions, single-image positions, packed segment resets, packed-vs-standalone equivalence, video rejection, and HF rotary parity.

Validation

  • FLA_TILELANG=0 uv run pytest tests/unit/utils/test_sequence.py tests/unit/train/models/test_qwen3_5_moe_mrope.py tests/unit/train/models/test_qwen3_5_moe_vlm.py tests/unit/train/models/test_qwen3_5_moe.py
  • Also run as part of the combined focused suite: FLA_TILELANG=0 uv run pytest tests/unit/utils/test_sequence.py tests/unit/orchestrator/test_batch.py tests/unit/train/test_model_forward.py tests/unit/train/models/test_qwen3_5_moe_mrope.py tests/unit/train/models/test_qwen3_5_moe_vlm.py tests/unit/train/models/test_qwen3_5_moe.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant