feat: BROS-833: Video vectors#9504
Merged
robot-ci-heartex merged 29 commits intodevelopfrom Mar 13, 2026
Merged
Conversation
✅ Deploy Preview for label-studio-docs-new-theme canceled.
|
✅ Deploy Preview for heartex-docs canceled.
|
✅ Deploy Preview for label-studio-playground ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for label-studio-storybook ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Member
Author
|
/fmt |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class support for vector (polyline/polygon) annotations on video frames, including new schema tags, frontend region/tool implementations, and backend label extraction adjustments.
Changes:
- Introduces new
VideoVector/VideoVectorLabelscontrol tags and registers a newVideoVectorRegionModelregion type. - Implements video vector rendering + editing via
KonvaVector, including keyframe-based interpolation over frames. - Updates editor plumbing (result types, area dispatch, view tag registry) and docs/schema to recognize the new annotation types.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| web/libs/editor/src/tools/index.js | Exposes the new VideoVector tool from the tools entrypoint. |
| web/libs/editor/src/tools/VideoVector.js | Adds a dedicated drawing tool for video vector regions (click/dblclick workflow). |
| web/libs/editor/src/tags/visual/View.jsx | Allows videovector / videovectorlabels tags in the View tag whitelist. |
| web/libs/editor/src/tags/object/Video/VideoVector.jsx | New Konva overlay shape for rendering/editing video vectors with percent↔pixel conversion. |
| web/libs/editor/src/tags/object/Video/VideoRegions.jsx | Routes stage mouse events to the vector tool and renders VideoVectorShape for the new region type. |
| web/libs/editor/src/tags/object/Video/Video.js | Adds videoVectorControl and addVideoVectorRegion() creation path. |
| web/libs/editor/src/tags/object/Video/HtxVideo.jsx | Treats video as “supports regions” when vector controls exist. |
| web/libs/editor/src/tags/control/index.js | Registers VideoVectorModel + VideoVectorLabelsModel exports. |
| web/libs/editor/src/tags/control/VideoVectorLabels.jsx | Implements combined VideoVector+Labels control. |
| web/libs/editor/src/tags/control/VideoVector.js | Implements VideoVector control model and tag registration. |
| web/libs/editor/src/tags/control/Vector.js | Updates Vector docs comment to remove the curves param mention. |
| web/libs/editor/src/tags/control/Label.jsx | Allows Label to be nested under VideoVectorLabels. |
| web/libs/editor/src/regions/index.js | Registers VideoVectorRegionModel in the global regions union/exports. |
| web/libs/editor/src/regions/VideoVectorRegion.jsx | Adds the new video vector region model with keyframe interpolation + shape updates. |
| web/libs/editor/src/regions/Result.js | Adds videovector / videovectorlabels as recognized result types/values. |
| web/libs/editor/src/regions/Area.js | Routes video results to VideoVectorRegionModel when sequence items include vertices. |
| web/libs/editor/src/components/SidePanels/DetailsPanel/RegionItem.tsx | Shows “incomplete region” warning based on region.incomplete. |
| web/libs/editor/src/components/Node/Node.tsx | Adds node metadata (name/icon/content) for video vector regions. |
| web/libs/editor/src/components/KonvaVector/eventHandlers/mouseHandlers.ts | Avoids internal point creation logic when point addition is externally managed. |
| web/libs/editor/src/components/KonvaVector/KonvaVector.tsx | Tweaks “drawing disabled” logic for externally-managed point creation + ghost-line state. |
| web/libs/editor/src/components/ImageView/LabelOnRegion.jsx | Makes video label rendering safer and respects showLabels via optional chaining. |
| web/libs/core/src/lib/utils/schema/tags.json | Adds schema definitions for VideoVector / VideoVectorLabels and updates allowed tags list; removes curves from Vector schema. |
| label_studio/projects/models.py | Updates project label validation / extraction to account for new video vector types. |
| docs/source/includes/tags/videovectorlabels.md | Adds docs for VideoVectorLabels parameters. |
| docs/source/includes/tags/videovector.md | Adds docs for VideoVector parameters. |
| docs/source/includes/tags/vector.md | Removes curves from Vector docs table. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Member
Author
|
/fmt |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 26 out of 26 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
web/libs/editor/src/components/SidePanels/DetailsPanel/RegionItem.tsx:73
RegionItemnow relies onregion.incompleteto show the “Incomplete …” warning, but many region models (e.g. PolygonRegionModel, RectRegionModel, VideoRectangleRegionModel) don’t define anincompleteview, so the warning will no longer appear while those regions are being drawn. Consider falling back to the previousregion.isDrawingcheck (or defining a defaultincompletegetter in a shared mixin likeAreaMixin) so in-progress drawings still show the warning consistently.
{region.incomplete && (
<div className={cn("detailed-region").elem("warning").toClassName()}>
<IconWarning />
<div className={cn("detailed-region").elem("warning-text").toClassName()}>
Incomplete {region.type?.replace("region", "") ?? "region"}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
nass600
approved these changes
Mar 12, 2026
yyassi-heartex
approved these changes
Mar 12, 2026
Contributor
yyassi-heartex
left a comment
yyassi-heartex
left a comment
There was a problem hiding this comment.
looks good over all, just had a question and a minor nitpick
Member
Author
|
/fmt |
Member
Author
|
/git merge
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces support for vector-based annotations on video frames by adding new
VideoVectorandVideoVectorLabelstags, along with their associated backend and frontend logic. It also updates documentation, schema definitions, and region handling to accommodate these new annotation types. Additionally, the PR implements keyframe-based interpolation for smooth vector transitions across video frames.The most important changes are:
New Video Vector Annotation Support:
VideoVectorandVideoVectorLabelstags to the schema (tags.json) with comprehensive attribute definitions for vector annotation on videos, including support for keyframe interpolation, skeleton mode, and customizable appearance. [1] [2]VideoVectorRegionModelin the frontend, including logic for keyframe-based interpolation of vector shapes, region actions, and registration in the region registry. [1] [2] [3] [4] [5] [6] [7] [8]Documentation Updates:
VideoVectorandVideoVectorLabelstags. [1] [2]Backend and Data Model Adjustments:
VideoVectorLabelsin project configuration and annotation label extraction, ensuring compatibility with existing video annotation workflows. [1] [2]Frontend Display and Usability Improvements:
Schema Cleanup:
curvesattribute from the schema for relevant tags, possibly to avoid redundancy or confusion, as curve support is now handled differently. [1] [2]