Skip to content

Add PerfLab benchmark experiments - #1151

Draft
cvolkcvolk wants to merge 11 commits into
mainfrom
cvolk/feature/perflab-benchmarks
Draft

Add PerfLab benchmark experiments#1151
cvolkcvolk wants to merge 11 commits into
mainfrom
cvolk/feature/perflab-benchmarks

Conversation

@cvolkcvolk

@cvolkcvolk cvolkcvolk commented Aug 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Add reproducible non-OSMO PerfLab workloads.

Detailed description

  • Instruments Experiment Runner phases with timing and reproducibility artifacts.
  • Adds eight isolated workloads, including same-object and ten-fruit controls on the registered Maple-table environment.
  • Documents the exact local and PerfLab command, sweeps, repeats, artifacts, and failure handling.
  • Keeps 10 steps as a smoke-only value and the reduced camera dimensions TBD before handoff.

@cvolkcvolk
Add Experiment Runner benchmark logging
bd2dd36 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Add camera resolution overrides
359f1b1 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Add PerfLab benchmark experiments
2890e62 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Document concrete PerfLab commands
33ea22f 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
cvolkcvolk force-pushed the cvolk/feature/perflab-benchmarks branch from df878cf to 33ea22f Compare August 27, 2026 11:42
@cvolkcvolk
cvolkcvolk changed the base branch from cvolk/feature/timer-utility to main August 27, 2026 11:42
@cvolkcvolk
Use registered environment for object benchmarks
24c7f89 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Document first PerfLab benchmark runs
851aa67 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Simplify PerfLab runs and select OpenPI GPU
2c15aa8 
Remove smoke, repeat, and attempt variables from the operator flow. Add explicit OpenPI GPU selection without changing server networking.

Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Add Cosmos and object PerfLab runs
cb75ad9 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Simplify PerfLab run instructions
72c8c87 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Document PerfLab timing output
b0cd0b3 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
@cvolkcvolk
Use PerfLab branch in checkout steps
3e48720 
Signed-off-by: Clemens Volk <cvolk@nvidia.com>
This was referenced Sep 9, 2026
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexmillane alexmillane Awaiting requested review from alexmillane alexmillane will be requested when the pull request is marked ready for review alexmillane is a code owner
@xyao-nv xyao-nv Awaiting requested review from xyao-nv xyao-nv will be requested when the pull request is marked ready for review xyao-nv is a code owner
@qianl-nv qianl-nv Awaiting requested review from qianl-nv qianl-nv will be requested when the pull request is marked ready for review qianl-nv is a code owner
@peterd-NV peterd-NV Awaiting requested review from peterd-NV peterd-NV will be requested when the pull request is marked ready for review peterd-NV is a code owner
@zhx06 zhx06 Awaiting requested review from zhx06 zhx06 will be requested when the pull request is marked ready for review zhx06 is a code owner
@viiik-inside viiik-inside Awaiting requested review from viiik-inside viiik-inside will be requested when the pull request is marked ready for review viiik-inside is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@cvolkcvolk