fix(pipeline): start endpoint before encoders to fix RTMP audio race condition#294
Open
theermite wants to merge 1 commit into
Open
fix(pipeline): start endpoint before encoders to fix RTMP audio race condition#294theermite wants to merge 1 commit into
theermite wants to merge 1 commit into
Conversation
EncodingPipelineOutput.startStreamUnsafe() launched the audio and video
encoder coroutines BEFORE calling endpointInternal.startStream(). On
RTMP endpoints this meant the AAC encoder's sequence header (csd-0,
AACPacketType=0) was emitted by MediaCodec within milliseconds of
start, forwarded by the init{} output consumer to the endpoint, while
the underlying RtmpClient.publish() handshake had not yet run.
RtmpEndpoint.write() silently catches all Throwables, so the failed
write of the AAC sequence header surfaced only as a single W log. The
AudioFlvStream.sentSequenceStart flag was set to true on that first
frame, so the codec config was never re-emitted. Twitch (and any RTMP
ingest) then never received an audio codec descriptor, and reported an
empty Audio Codec / SampleRate in Inspector for the whole stream while
video flowed normally.
Moving endpointInternal.startStream() to BEFORE the encoder coroutines
launch guarantees the RTMP publish() handshake and onMetaData write
complete before any encoded frame can be produced.
Observed on:
- StreamPack 3.1.2
- Android 15 / ColorOS (Realme GT Neo5)
- Twitch RTMP ingest live.twitch.tv
Co-Authored-By: Takumi "IA Dev Partner"
Owner
|
Hi, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix a race condition in
EncodingPipelineOutput.startStreamUnsafe()that prevents RTMP ingest endpoints (Twitch confirmed) from ever receiving the AAC audio sequence header (csd-0/AACPacketType=0), causing the audio track to be silently dropped while video flows normally.Symptom observed
Audio Track | Codec [empty] | Sample Rate [empty]for the entire streamWlog lineError while writing FLV dataappears in logcatRoot cause
In
core/src/main/java/io/github/thibaultbee/streampack/core/pipelines/outputs/encoding/EncodingPipelineOutput.kt,startStreamUnsafe()currently launches the audio and video encoder coroutines before callingendpointInternal.startStream():```kotlin
streamEventListener?.onStartStream()
val audioEncoderJob = audioEncoderInternal?.let {
coroutineScope.launch { it.startStream() } // <-- MediaCodec.start() inside
}
val videoEncoderJob = videoEncoderInternal?.let {
coroutineScope.launch { it.startStream() }
}
audioEncoderJob?.join()
videoEncoderJob?.join()
endpointInternal.startStream() // <-- RtmpClient.publish() inside, runs LAST
```
But:
audioEncoderInternal.startStream()callsMediaCodec.start()internally. The AAC encoder emits its sequence header (csd-0, anAudioSpecificConfigof 2 bytes for AAC LC mono 44 100) within milliseconds of start.init { }(around lines 258-272) forwards every encoded frame toendpointInternal.write(closeableFrame, audioStreamId)as soon asaudioStreamId != null.RtmpEndpoint.write()silently catches allThrowable:```kotlin
catch (t: Throwable) {
Logger.w(TAG, "Error while writing FLV data: $t")
}
```
csd-0frame arrives at the endpoint,RtmpClient.publish()has not yet handshaked. The write fails, the exception is swallowed, andAudioFlvStream.sentSequenceStartflips totrueon that first frame (it tracks "I already tried to send the header"). The codec config is never re-emitted.Video is unaffected because by the time the video encoder produces a keyframe (later than the AAC
csd-0),endpoint.startStream()has usually completed.Fix
Move
endpointInternal.startStream()to before the encoder coroutines launch, so the RTMPpublish()handshake andonMetaDatawrite are guaranteed complete before any encoded frame can be produced.```kotlin
streamEventListener?.onStartStream()
endpointInternal.startStream() // <-- moved up
val audioEncoderJob = audioEncoderInternal?.let { ... }
val videoEncoderJob = videoEncoderInternal?.let { ... }
audioEncoderJob?.join()
videoEncoderJob?.join()
```
A descriptive inline comment is added at the new position explaining the race.
Validation
Tested live against Twitch ingest (
live.twitch.tv) from the Hoso Android app (https://github.com/theermite/Hoso), Realme GT Neo5 / Android 15 / ColorOS, StreamPack 3.1.2 + this single patch (Gradle composite build):Before patch (StreamPack 3.1.2 stock):
After patch:
Scope
core/.../EncodingPipelineOutput.ktstartStream()is cheap or whosewrite()does not depend on a prior handshake)Test plan
AAC | 44100 Hzfor the audio trackstartStreamis also fast, but the principle "endpoint ready before encoders" is structurally safer regardless)🤖 Patch authored with the help of Claude Code on a real-world debugging session for Hoso.