Skip to content

sensing: add the SENSING SG8A SIPL camera plugin - #998

Draft
jiwenc-nv wants to merge 1 commit into
jiwenc/sensing-cuda-ipcfrom
jiwenc/camera-cuda
Draft

sensing: add the SENSING SG8A SIPL camera plugin#998
jiwenc-nv wants to merge 1 commit into
jiwenc/sensing-cuda-ipcfrom
jiwenc/camera-cuda

Conversation

@jiwenc-nv

@jiwenc-nv jiwenc-nv commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Description

Stack 3/3 — depends on #1078 (which depends on #1077). Review that order; this PR's diff is only the plugin.

New C++ plugin camera_plugin_sensing for the SENSING SG8A-AGON-G2Y-A1 GMSL carrier on a Jetson AGX Orin (JetPack 7.2.1 / L4T R39.2.1). It captures two Astra SHW5G at 2560×1984 @ 60 through SIPL and either encodes H.264 on the V4L2 M2M engine or publishes frames over the CUDA IPC transport from #1078.

The carrier ships no kernel sensor drivers and no /dev/video* — SIPL owns the sensors from userspace — so there is no daemon, no per-boot module load and no V4L2 controls.

Four SIPL requirements look arbitrary and are not: call ordering around Init(), mandatory ICP buffers, NeedCpuAccess on the producer attribute list, and REC709_ER on ISP0. Each is commented where it is enforced; getting any of them wrong fails silently or with a status code that names nothing.

--add-stream sensor=N is the SIPL pipeline index (0 and 1), not the GMSL link index, CSI virtual channel, or JSON sensorInfo.id — all of which are 2 and 3 on this rig.

Auto-exposure saturates with sensor gain pinned at 1.0. That reproduces in NVIDIA's own nvsipl_camera, so it is upstream; tracked in #1076.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Testing

AGX Orin, JetPack 7.2.1 / L4T R39.2.1, SG8A carrier with two Astra SHW5G:

  • Both sensors at 60 fps, zero drops, rendered by camera_viz in window mode and in XR.
  • Runs as the ordinary container user: no root, no capabilities, only video / i2c / gpio.
  • --list-sensors and ctest -R sipl_query assert pipeline indices and geometry with no camera attached.
  • SKIP=check-copyright-year pre-commit run --all-files passes.

H.264 recording and the MCAP metadata path are not yet verified end to end on this rig.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix/feature works (or explained why not)
  • I have signed off all my commits (git commit -s) per the DCO

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

if self._sock is not None:
try:
self._sock.close()
except OSError:
@jiwenc-nv

Copy link
Copy Markdown
Collaborator Author

/preview-docs

@github-actions

Copy link
Copy Markdown
Contributor

✅ Preview deployed: https://NVIDIA.github.io/IsaacTeleop/preview/pr-998/

@jiwenc-nv jiwenc-nv changed the title sensing: add the SENSING SG10A GMSL camera plugin sensing: add the SENSING SIPL GMSL camera plugin Sep 4, 2026
@jiwenc-nv jiwenc-nv changed the title sensing: add the SENSING SIPL GMSL camera plugin sensing: add the SENSING SG8A SIPL camera plugin Sep 4, 2026
@jiwenc-nv
jiwenc-nv changed the base branch from main to jiwenc/sensing-cuda-ipc September 4, 2026 04:13
@jiwenc-nv
jiwenc-nv force-pushed the jiwenc/camera-cuda branch 2 times, most recently from d4b596c to 65fbfe8 Compare September 4, 2026 04:50
@jiwenc-nv
jiwenc-nv force-pushed the jiwenc/camera-cuda branch 2 times, most recently from 3c5ccdd to 0dce783 Compare September 4, 2026 21:39
New C++ plugin camera_plugin_sensing for the SENSING SG8A-AGON-G2Y-A1 GMSL
carrier on a Jetson AGX Orin (JetPack 7.2.1 / L4T R39.2.1). It captures two
Astra SHW5G at 2560x1984@60 through SIPL and either encodes H.264 on the
V4L2 M2M engine or publishes frames over the CUDA IPC transport.

The carrier ships no kernel sensor drivers and no /dev/video*: SIPL owns the
sensors from userspace, so there is no daemon, no per-boot module load and no
V4L2 controls. Capture runs as the ordinary container user, needing only the
video, i2c and gpio groups.

Four SIPL requirements look arbitrary and are not -- call ordering around
Init(), mandatory ICP buffers, NeedCpuAccess on the producer attribute list,
and REC709_ER on ISP0. Each is commented where it is enforced; getting any of
them wrong fails silently or with a status code that names nothing.

sensing.rst and the plugin README carry the rig details. shw5g.json is a
verbatim copy of the vendor platform config; configs/README.md records its
provenance and the rule against editing it in place.

Auto-exposure saturates with sensor gain pinned at 1.0. That reproduces in
NVIDIA's own nvsipl_camera, so it is upstream; tracked in #1076.

Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants