feat(engine): [image, audio] sglang image vllm asr engines

[OliverBryant]

#4896

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

[OliverBryant]

End-to-end benchmark: Z-Image on diffusers / vLLM / SGLang

Ran all three image engines for real on a GPU box against this PR's HEAD, same prompt and generation config, measuring client-side wall-clock of POST /v1/images/generations (3 timed runs per engine after warmup, one engine loaded at a time).

Setup

• GPU: 1x NVIDIA GeForce RTX 3090 Ti (24 GB), model pinned to a single GPU via gpu_idx
• Host env: Python 3.12, torch 2.11.0, vllm 0.24.0, sglang 0.5.7, transformers 5.14.1
• Model: Z-Image, defaults from the builtin spec (num_inference_steps=28, guidance_scale=4), 1024*1024, n=1, response_format=b64_json
• Prompt: A red bicycle leaning against a weathered stone wall at golden hour, cinematic lighting
• Each engine runs in its own model virtualenv (vLLM venv resolves the vllm-omni==0.24.* + vllm==0.24.* pair from the spec; SGLang venv is self-contained with sglang[diffusion]'s own torch 2.9.1 stack)

Single-request latency

Engine	Run 1	Run 2	Run 3	Avg	vs diffusers
diffusers	48.9 s	49.7 s	51.2 s	49.9 s	1.00x
vLLM (vllm-omni 0.24.0)	40.3 s	41.3 s	42.6 s	41.4 s	1.21x
SGLang (sglang[diffusion] 0.5.7)	43.9 s	44.9 s	46.6 s	45.2 s	1.10x

Concurrency: 4 simultaneous requests (fired at once after warmup)

Engine	Request latencies (sorted)	Makespan	Per-image under load	vs its own serial
diffusers	53.3 / 108.4 / 164.9 / 223.4 s	223.4 s	55.9 s	+12%
vLLM	40.2 / 81.3 / 123.7 / 167.4 s	167.4 s	41.9 s	+1%
SGLang	44.2 / 89.6 / 136.9 / 185.9 s	185.9 s	46.5 s	+3%

Reading the numbers:

• vLLM is the only engine that accepts requests concurrently: the actor no longer serializes calls (allow_batch=True) and every request is submitted to vLLM-Omni's engine queue immediately, with a single dispatcher routing outputs back by request id. Z-Image's pipeline does not support request batching in vllm-omni, so GPU execution is still one-at-a-time — the win is zero added overhead under load (41.9 s vs 41.4 s serial), immediate admission, and per-request error isolation rather than throughput multiplication.
• diffusers and SGLang serialize at the model actor: concurrent callers queue outside the engine. They still complete correctly, with diffusers paying ~12% per-image overhead under load.
• Net effect for 4 requests: vLLM finishes the batch 1.33x faster than diffusers and 1.11x faster than SGLang.

Cross-request GPU batching: sd3.5-medium (vllm-omni pipeline with supports_request_batch=True)

Z-Image's vllm-omni pipeline declares supports_request_batch = False (pipeline_z_image.py), and the engine refuses max_num_seqs > 1 for such pipelines, so its GPU execution is serial upstream. To verify the dispatcher composes with true request batching, sd3.5-medium (SD3 pipeline, supports_request_batch = True) was launched with max_num_seqs=4 and hit with 4 simultaneous requests:

	Latencies	Makespan
single request	5.5 s	--
4 concurrent (steady state)	5.6 / 21.46 / 21.48 / 21.48 s	21.5 s

Requests in the same wave complete with identical timestamps (21.478/21.460/21.478 s) -- they went through the DiT as one batched tensor -- and the engine logs [RequestBatch] engine init max_num_seqs=4. On this compute-saturated 3090 Ti the batched throughput matches serial FLOPs, so the makespan equals serial time; on GPUs where a single request underutilizes compute, the same path yields real throughput gains. sd3.5-medium is now registered for the vLLM engine.

All three engines produced valid, prompt-faithful 1024x1024 images (visually verified).

Bugs found and fixed while testing (pushed to this branch):

• f0c0293 — vLLM engine failed to launch: the concurrent-dispatch locks made the model instance unpicklable for the actor-subprocess handoff (cannot pickle '_thread.lock' object)
• 0e75c68 — SGLang generation crashed with Object of type UUID is not JSON serializable: xinference injects request_id as uuid.UUID and SGLang's SamplingParams accepts the field but its runtime JSON-serializes it; now stringified
• 4449556 — SGLang generation crashed with No such file or directory: 'ninja': the ninja wheel installs the binary into the venv's bin directory, which was not on PATH in the model subprocess; now prepended at load time

Environment notes (outside this PR's scope)

• The SGLang venv install currently needs XINFERENCE_VIRTUAL_ENV_SKIP_INSTALLED=0 until BUG: resolve specs with extras in uv skip_installed even if base package is installed xoscar#193 lands: with skip-installed on, a bare host sglang is treated as satisfying sglang[diffusion] and the venv comes up without the diffusion runtime
• On hosts with transformers >= 5 the huggingface-hub<1.0 pin in ENGINE_VIRTUALENV_PACKAGES["diffusers"] cannot co-resolve; the box was tested with it relaxed to <2.0 (pre-existing on main, not touched here)

[qinxuye]

LGTM