Repository navigation
Add streaming and pcm response_format to /v1/audio/speech - #1
Open
ashharadnan wants to merge 3 commits into
Open
ashharadnan wants to merge 3 commits into
ashharadnan wants to merge 3 commits into
Conversation
) Adapted to the AMD fork: /tts endpoint and CustomTTSRequest left untouched (fork never had stream on /tts; upstream restructure dropped in conflict resolution). README corrected to match fork reality. Fixed two fork-specific NameErrors the PR missed: server.py never imported asyncio (streaming generator) or struct (WAV header path).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports devnen#176 (streaming + pcm response_format on the OpenAI-compatible
/v1/audio/speechendpoint) onto this AMD fork, adapted to fork reality:CustomTTSRequesthas no stream field and upstream's /tts restructure conflicted, so only the OpenAI endpoint gained streaming.asyncio(streaming generator callsget_running_loop/run_in_executor) orstruct(streaming WAV header). Both added.openai_stream_chunk_size(default 50),openai_stream_by_default.Tested on RX 6800 XT / gfx1030, ROCm 7.2 wheels, torch 2.9.1, Turbo model (warm):
stream=true, response_format=pcm, chunk_size=50: first audio ~0.9-1.0 s vs ~6.5-7.0 s non-streaming (same total wall time, ~8 s).stream=true, response_format=wav: streaming WAV header + PCM16 frames verified.stream: byte-identical legacy behavior.Note: stream outputs engine-native 24 kHz PCM regardless of configured sample_rate; clients must wrap with their own header for pcm.
Known caveat (raw streaming wav): when
stream=trueandresponse_format=wav, the served WAV header carries the0xFFFFFFFFwildcard for both the RIFF and data sizes, and it is never patched when the stream closes. This is by design - audio sizes aren't known upfront in streaming mode - but it means the raw response is not a well-formed file for direct playback: players that trust WAV duration metadata will show a bogus duration (observed ~89,478 s) or refuse the clip. Clients consuming raw streaming wav must ignore the header and rely on the stream end or accumulated chunk length instead; usepcmif you intend to write your own header client-side, as done in the reference client for this PR's tests.