fix(core): use raw data-start for media elements in preview visibility loop#1078
fix(core): use raw data-start for media elements in preview visibility loop#1078JamesXiaoFF wants to merge 2 commits into
Conversation
…y loop For video and audio elements, data-start is authored in global (composition-root) time — the same contract used by the render pipeline's discoverMediaFromBrowser, which reads the raw attribute directly. Previously, the visibility loop called resolveStartForElement which adds the nearest ancestor composition's global start on top, causing a double-offset that kept pip-wired media permanently hidden when the host composition did not start at t=0. Example: a pip video with data-start="45.40" inside a host composition that also starts at data-start="45.40" resolved to 90.80, so the video was always hidden during its actual [45.40, 52.46] window. Non-media elements (divs, sections, etc.) continue to use the accumulating resolver because their data-start values are local to their composition. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
0211b02 to
53e3175
Compare
Superseded — resubmitting after reproducing the bug.
Cleaning up — resubmitting as a single clean review.
There was a problem hiding this comment.
Confirmed the bug — I reproduced the double-offset on main. The pip video gets resolved to 90.80 instead of 45.40 and stays permanently hidden. Good catch.
Two things I'd want before merging:
The fix only covers the visibility loop, but refreshRuntimeMediaCache (init.ts:1284) and resolveMediaWindowEndSeconds (timeline.ts:223) resolve media start times the same way. Without patching those too, the video shows up at the right moment but its playback timing is still off. Fixing this in resolveHostOffsetForElement instead would cover all three call sites in one shot.
Also worth adding a test for the auto-injected data-start="0" case — a video with no explicit timing inside a host at data-start="10" should still appear at t=10, not t=0. If that already works, a quick test proving it would round this out nicely.
Add also a regression test in the producer tests to avoid regressions.
Narrow the raw data-start read to media elements without data-hf-auto-start (explicitly authored global coordinates). Elements with auto-injected data-start="0" remain composition-local via the resolver. Apply consistently across all three consumers: - visibility loop (init.ts) - refreshRuntimeMediaCache start/duration (init.ts) - resolveMediaWindowEndSeconds (timeline.ts) Add regression test for auto-injected data-start="0" inside a late-starting host to prove it doesn't regress.
miguel-heygen
left a comment
There was a problem hiding this comment.
Pushed a follow-up commit addressing the review feedback:
- Narrowed the raw read to media elements without data-hf-auto-start so auto-injected timing stays composition-local
- Extended the fix to refreshRuntimeMediaCache and resolveMediaWindowEndSeconds in timeline.ts
- Added regression test for data-start="0" inside a late-starting host
All 998 core tests pass.
|
I had to do a new PR including a regression test with LFS! Thanks for contributing! |
Problem
When a pip-wired
<video>or<audio>element has adata-startvalue in global (composition-root) time and lives inside a sub-composition that also starts at that same global time, the preview visibility loop was keeping it permanently hidden.Root cause: The visibility loop called
resolveStartForElement, which adds the nearest ancestor composition's global start on top of the element's owndata-start. For a pip video withdata-start="45.40"inside a host atdata-start="45.40", the resolved start became45.40 + 45.40 = 90.80— well past the video's actual window.Why media elements are different: The render pipeline's
discoverMediaFromBrowserreadsdata-starton<video>and<audio>as a raw global timestamp (not composition-relative). The preview visibility loop must use the same semantics to stay consistent with render output.Fix
In the visibility loop inside
init.ts, media elements (video,audio) now readdata-startdirectly from the DOM attribute instead of going throughresolveStartForElement. Non-media elements (divs, sections, etc.) continue to use the accumulating resolver because theirdata-startvalues are local to their composition timeline.Test
Added a regression test in
init.test.tsthat reproduces the exact double-offset scenario: a pip video at globaldata-start="45.40"inside a host composition starting at45.40must be visible att=46and hidden att=53.