Release/0.23.0 by kiran-thumma · Pull Request #1058 · ROCm/vllm

kiran-thumma · 2026-07-14T04:49:57Z

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…AMD Zen CPUs (vllm-project#41813) Signed-off-by: R <Ganesh.R@amd.com> Signed-off-by: Harshal Adhav <harshal.adhav@amd.com> Signed-off-by: Aakar Dwivedi <aadwived@amd.com> Co-authored-by: R <Ganesh.R@amd.com> Co-authored-by: Harshal Adhav <harshal.adhav@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Michael Goin <mgoin64@gmail.com>

…ct#43956) Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>

Co-authored-by: nightcityblade <nightcityblade@gmail.com>

…mpletion protocol validators (vllm-project#36254) Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

…lm-project#44035) Signed-off-by: esmeetu <jasonailu87@gmail.com> Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: esmeetu <jasonailu87@gmail.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

…ct#42730) Signed-off-by: wcy <233313160abc@gmail.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com>

Signed-off-by: UranusSeven <109661872+UranusSeven@users.noreply.github.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Will.hou <1205157517@qq.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

…model. (vllm-project#43770) Signed-off-by: zzt <zengzetang.zzt@antgroup.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>

… fixture (vllm-project#44146) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

…erformance improvement (vllm-project#43706) Signed-off-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

…llm-project#43992) Signed-off-by: Madeesh Kannan <madeeswaran.kannan@jetbrains.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

…4161) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Micah Williamson <micah.williamson@amd.com> Co-authored-by: Micah Williamson <micah.williamson@amd.com>

…3779) Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

… updates (vllm-project#40096) Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

…ject#44234) Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…-project#44256) Signed-off-by: JartX <sagformas@epdcenter.es>

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…nnector (vllm-project#43742) Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

…ct#44266) Signed-off-by: Alec Flowers <aflowers@nvidia.com>

vllm-project#44017) Signed-off-by: sfeng33 <4florafeng@gmail.com>

…llm-project#44131) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…44699) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

vllm-project#36423) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

…project#37149) Signed-off-by: Yan Ma <yan.ma@intel.com>

…ct#44540) Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com> Signed-off-by: Kunshang Ji <jikunshang95@gmail.com> Co-authored-by: Kunshang Ji <jikunshang95@gmail.com>

…roject#44041) Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>

Signed-off-by: JaredforReal <w13431838023@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

… registry (vllm-project#44707) Signed-off-by: Terrencezzj <terrence@cohere.ai> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: JaredforReal <w13431838023@gmail.com>

…in MooncakeStoreConnector KV events (vllm-project#44103) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>

… VLLM_TORCH_COMPILE_LEVEL (vllm-project#44378) Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

Signed-off-by: Bram Vanroy <2779410+BramVanroy@users.noreply.github.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

Signed-off-by: charlifu <charlifu@amd.com>

…ction (vllm-project#44454) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

…-project#42736) Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>

) Signed-off-by: SunskyXH <sunskyxh@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io>

…roject#44805) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>

…t#39562) Signed-off-by: Holworth <kangqihan17@mails.ucas.ac.cn> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>

) Signed-off-by: Ma Jian <jian1.ma@intel.com>

Signed-off-by: shen-shanshan <467638484@qq.com>

…ect#44771) Signed-off-by: Yan Ma <yan.ma@intel.com>

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…i-K2.5 (vllm-project#42978) Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> (cherry picked from commit baacbfc)

…roject#45169) Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> (cherry picked from commit bfe1001)

…erts (vllm-project#44679) Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> (cherry picked from commit 5b6b536)

…ct#45204) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com> (cherry picked from commit 40e065e)

…ject#45401) Signed-off-by: khluu <khluu000@gmail.com> (cherry picked from commit e3e31e5)

…strumentator crash on serve startup A fresh pip install pulls fastapi 0.137, whose new _IncludedRouter route type (a BaseRoute with no .path) breaks prometheus-fastapi-instrumentator's route iteration -> 'vllm serve' dies at startup with: AttributeError: '_IncludedRouter' object has no attribute 'path' requirements/common.txt pinned fastapi[standard]>=0.115.0 with no upper bound. Cap to <0.137; lift once the instrumentator handles _IncludedRouter (trallnag/prometheus-fastapi-instrumentator#370). Signed-off-by: avesed <trevor.lijunlin@gmail.com>

WoosukKwon and others added 30 commits May 30, 2026 09:40

[MRV2] Support breakable CUDA graph (vllm-project#44050)

27fa5aa

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[CI/Build] Enable Step3p7ForConditionalGeneration testing (vllm-proje…

6bdabba

…ct#43956) Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>

docs: fix MLA attention docstring examples (vllm-project#44118)

8b8546d

Co-authored-by: nightcityblade <nightcityblade@gmail.com>

[Misc] Use VLLMValidationError consistently in chat completion and co…

f46e6be

…mpletion protocol validators (vllm-project#36254) Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>

[MRV2] Remove Eagle's dedicated CUDA graph pool (vllm-project#44078)

4721bb3

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

[Docs] Replace broken video url in examples (vllm-project#44159)

1fd8bd0

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

[CPU][RISC-V] Add missing RVV cpu_types helpers for WNA16 (vllm-proje…

98f1279

…ct#42730) Signed-off-by: wcy <233313160abc@gmail.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com>

fix: glm5.1 pp model loading (vllm-project#42944)

1f6048a

Signed-off-by: UranusSeven <109661872+UranusSeven@users.noreply.github.com>

[Frontend] Resettle generative scoring entrypoint. (vllm-project#44153)

0910f7e

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

[Rust Frontend] Add InternLM2 tool parser (vllm-project#43481)

de21863

Signed-off-by: Will.hou <1205157517@qq.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[Bugfix] fix wrong partial_rotary_factor calculation for bailing_moe …

8796838

…model. (vllm-project#43770) Signed-off-by: zzt <zengzetang.zzt@antgroup.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>

[XPU][CI] Fix test_audio_in_video flake by using module-scoped server…

bd0aecd

… fixture (vllm-project#44146) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

[Perf] Optimize cutlass fp8 scaled mm bypassing padding, 20% kernel p…

985c97a

…erformance improvement (vllm-project#43706) Signed-off-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[Feature] Add support for JetBrains' Mellum v2 code generation model (v…

023808c

…llm-project#43992) Signed-off-by: Madeesh Kannan <madeeswaran.kannan@jetbrains.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

[Kernel][DSv4] Optimize sparse FP8 compressor kernels (vllm-project#4…

0357335

…4161) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

[ROCm][CI] Fix and stabilize EAGLE3 acceptance tests (vllm-project#41294

fd9e91d

) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Micah Williamson <micah.williamson@amd.com> Co-authored-by: Micah Williamson <micah.williamson@amd.com>

[Rust Frontend] Support streaming generate endpoint (vllm-project#4…

182c67d

…3779) Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[Frontend][Core] Add sparse NCCL weight transfer support for in-place…

266b9d9

… updates (vllm-project#40096) Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>

[BugFix][CI] Fix added _has_module tests (vllm-project#44248)

6f8b40a

Signed-off-by: Nick Hill <nickhill123@gmail.com>

[Test][BugFix] Fix double-BOS in PD+specdec acceptance test (vllm-pro…

e4cbc43

…ject#44234) Signed-off-by: Nick Hill <nickhill123@gmail.com>

[DSV4] Remove unncessary classes & functions (vllm-project#44246)

8c3cc98

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[ROCm][CI] Skip unbacked dynamic shapes tests on PyTorch < 2.11 (vllm…

48c0d13

…-project#44256) Signed-off-by: JartX <sagformas@epdcenter.es>

[DSV4] Refactor RoPE initialization (vllm-project#44262)

517e74a

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[Bugfix][Mooncake] Release GPU pin on failed store in MooncakeStoreCo…

d68f0b2

…nnector (vllm-project#43742) Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

[ROCm] Upgrade AITER to v0.1.13.post1 (vllm-project#44265)

2588ec4

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

[Bugfix][CI] Normalize NIXL connector CUDA wheel installs (vllm-proje…

816cc73

…ct#44266) Signed-off-by: Alec Flowers <aflowers@nvidia.com>

[Refactor] Move unstreamed tool-arg flush from serving layer to parser (

9affc17

vllm-project#44017) Signed-off-by: sfeng33 <4florafeng@gmail.com>

[CI] Stabilize OpenAI schema fuzzing for malformed structural tags (v…

54d0c36

…llm-project#44131) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

WoosukKwon and others added 30 commits June 6, 2026 20:37

[DSV4] Decouple DS V4 Sparse MLA Metadata from DS V3.2 (vllm-project#…

2a983c7

…44699) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[XPU] Support cpu kv offloading and tiering offloading on XPU platform (

8109664

vllm-project#36423) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

[XPU][Feature] transparent sleep mode support for XPU platform (vllm-…

3bb4697

…project#37149) Signed-off-by: Yan Ma <yan.ma@intel.com>

[XPU] add xpu branch in compressed_tensors_moe_w4a4_mxfp4 (vllm-proje…

6181e80

…ct#44540) Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com> Signed-off-by: Kunshang Ji <jikunshang95@gmail.com> Co-authored-by: Kunshang Ji <jikunshang95@gmail.com>

[Bugfix] Fix benchmark_moe.py after inplace mechanism removal (vllm-p…

9c7f774

…roject#44041) Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>

[feature] add index share feature for DSA MTP (vllm-project#44420)

32f34d3

Signed-off-by: JaredforReal <w13431838023@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[Cohere] Enable Cohere Mini Code model and update Command A-plus test…

1505b3d

… registry (vllm-project#44707) Signed-off-by: Terrencezzj <terrence@cohere.ai> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[videoloader] implement glm46v video loader (vllm-project#44417)

6ac6920

Signed-off-by: JaredforReal <w13431838023@gmail.com>

[Bugfix][Mooncake] Fix per-group block_size/block_hash and group_idx …

51ef688

…in MooncakeStoreConnector KV events (vllm-project#44103) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>

[Doc] Fix multimodal torch.compile troubleshooting to not use removed…

15652a6

… VLLM_TORCH_COMPILE_LEVEL (vllm-project#44378) Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

[CI] Stabilize the multi-audio OpenAI server path (vllm-project#44051)

f0f6805

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Dependency] Remove stale cuDNN frontend upper bound (vllm-project#42599

66ecfd0

) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>

Modify torch dependency in xpu.txt (vllm-project#43087)

3d3ba46

Signed-off-by: Bram Vanroy <2779410+BramVanroy@users.noreply.github.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

[ROCm][Kernel] Enable permute_cols for ROCm (vllm-project#44674)

228bcc4

Signed-off-by: charlifu <charlifu@amd.com>

[Kernel][Test] Make kernel tests for mamba dual-HW (CUDA + XPU) (vllm…

2ed0a96

…-project#42736) Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>

[Bugfix] Fix FunASR-Nano crash during initialization (vllm-project#44215

6124a98

) Signed-off-by: SunskyXH <sunskyxh@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io>

Added extra_repr() to pooler classes to improve debuggability (vllm-p…

5633405

…roject#44805) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>

[Bugfix]: Fix assertion in MambaManager.allocate_slots() (vllm-projec…

303916e

…t#39562) Signed-off-by: Holworth <kangqihan17@mails.ucas.ac.cn> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>

[XPU]feat: add DeepSeek-V4 XPU attention decode path (vllm-project#42953

eebce65

) Signed-off-by: Ma Jian <jian1.ma@intel.com>

[MM][CG] Simplify ViT CUDA graph interfaces (vllm-project#44484)

8fb0274

Signed-off-by: shen-shanshan <467638484@qq.com>

[XPU][Minor] format moe kernel name and add in kernel list (vllm-proj…

54c660c

…ect#44771) Signed-off-by: Yan Ma <yan.ma@intel.com>

[ROCm][CI] Stage C mirrors (vllm-project#42793)

967c5c3

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[Bugfix] [DSV4] [ROCm] Pin apache-tvm-ffi version to 0.1.10 (vllm-p…

741ba42

…roject#45169) Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> (cherry picked from commit bfe1001)

[ROCm][Bugfix] Make intermediate_pad TP-aware in rocm_aiter_fused_exp…

b2d7294

…erts (vllm-project#44679) Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> (cherry picked from commit 5b6b536)

[Docker] Fix CUTLASS DSL cu13 install order in Dockerfile (vllm-proje…

78743ab

…ct#45204) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com> (cherry picked from commit 40e065e)

[Bugfix][CPU] Don't build triton-cpu on arm64 release image (vllm-pro…

91df0fa

…ject#45401) Signed-off-by: khluu <khluu000@gmail.com> (cherry picked from commit e3e31e5)

Use release/v0.1.13.post1 branch for AITER_BRANCH

9ddef71

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Release/0.23.0#1058

Release/0.23.0#1058
kiran-thumma wants to merge 8104 commits into
mainfrom
release/0.23.0

kiran-thumma commented Jul 14, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Uh oh!

Conversation

kiran-thumma commented Jul 14, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

kiran-thumma commented Jul 14, 2026 •

edited by github-actions Bot

Loading