Summary
Research whether to replace the camera driver v4l2_camera with usb_cam. This
came out of the L1 depth-perception latency work: v4l2_camera turned out to be
limited in ways usb_cam is not, but a driver swap has real integration and
re-validation cost, so we want a decision backed by measurement, not a blind
switch.
Why this came up
While profiling the depth pipeline we tried to cap the camera at 20 fps (to cut Pi
CPU + Wi-Fi; it does not help depth latency, which is inference-bound). We found:
- The pinned
v4l2_camera 0.7.1 (the only version on robostack-jazzy) has no
frame-rate parameter and never calls VIDIOC_S_PARM — confirmed in the binary
(params are only video_device, image_size, pixel_format, output_encoding, camera_info_url, camera_frame_id). So capture rate is not settable in-driver.
- Current per-frame cost on the Pi: USB YUYV → (v4l2) YUYV→RGB8 → (image_transport)
RGB8→BGR→JPEG for /image_raw/compressed, i.e. two colour conversions + a JPEG
encode every frame (the "slow conversion yuv422_yuy2 => rgb8" WARN). Setting
output_encoding: yuv422_yuy2 doesn't help — the compressed plugin just converts
to BGR itself before encoding.
usb_cam 0.8.1 is available on robostack-jazzy and has a native framerate
param plus flexible pixel_format (incl. MJPEG) and more camera controls, with a
larger user base / better docs.
Research questions
- Does the UGREEN webcam expose MJPG @ 640x480 (
v4l2-ctl --list-formats-ext)?
- Can
usb_cam deliver the camera's native MJPEG straight to /image_raw/compressed
without the YUYV→RGB→JPEG round-trip? (This, not the frame-rate cap, is the real
potential Pi/Wi-Fi win.)
- Measured Pi CPU % and SoC temperature under
pixi run launch: v4l2_camera
(YUYV, ~28 fps) vs usb_cam (MJPEG, 20 fps). Is the camera actually straining the Pi?
- Wi-Fi bandwidth of the compressed stream in each config.
- Integration cost: re-map params in
mote_launch.py + robot.yaml; verify the
/image_raw/compressed topic name the depth node subscribes to, the
camera_info_url / ~/.mote/camera_calibration.yaml calibration plumbing, frame
ids, timestamping, and QoS. Note usb_cam pulls in ffmpeg/libavcodec (heavier dep).
Non-goal / context
- This is not a depth-latency optimisation — end-to-end latency is dominated by
off-board inference (measured ~458 ms quiet, ~2 s under CPU contention). The camera
driver only affects Pi CPU / Wi-Fi. Decide on those grounds.
Acceptance
A recommendation (switch / don't switch) backed by the measured Pi CPU + temp +
bandwidth numbers above, and — if switching — a scoped list of the launch/config
changes and the hardware re-validation steps.
Summary
Research whether to replace the camera driver
v4l2_camerawithusb_cam. Thiscame out of the L1 depth-perception latency work:
v4l2_cameraturned out to belimited in ways
usb_camis not, but a driver swap has real integration andre-validation cost, so we want a decision backed by measurement, not a blind
switch.
Why this came up
While profiling the depth pipeline we tried to cap the camera at 20 fps (to cut Pi
CPU + Wi-Fi; it does not help depth latency, which is inference-bound). We found:
v4l2_camera0.7.1 (the only version on robostack-jazzy) has noframe-rate parameter and never calls
VIDIOC_S_PARM— confirmed in the binary(params are only
video_device, image_size, pixel_format, output_encoding, camera_info_url, camera_frame_id). So capture rate is not settable in-driver.RGB8→BGR→JPEG for
/image_raw/compressed, i.e. two colour conversions + a JPEGencode every frame (the "slow conversion yuv422_yuy2 => rgb8" WARN). Setting
output_encoding: yuv422_yuy2doesn't help — the compressed plugin just convertsto BGR itself before encoding.
usb_cam0.8.1 is available on robostack-jazzy and has a nativeframerateparam plus flexible
pixel_format(incl. MJPEG) and more camera controls, with alarger user base / better docs.
Research questions
v4l2-ctl --list-formats-ext)?usb_camdeliver the camera's native MJPEG straight to/image_raw/compressedwithout the YUYV→RGB→JPEG round-trip? (This, not the frame-rate cap, is the real
potential Pi/Wi-Fi win.)
pixi run launch: v4l2_camera(YUYV, ~28 fps) vs usb_cam (MJPEG, 20 fps). Is the camera actually straining the Pi?
mote_launch.py+robot.yaml; verify the/image_raw/compressedtopic name the depth node subscribes to, thecamera_info_url/~/.mote/camera_calibration.yamlcalibration plumbing, frameids, timestamping, and QoS. Note
usb_campulls in ffmpeg/libavcodec (heavier dep).Non-goal / context
off-board inference (measured ~458 ms quiet, ~2 s under CPU contention). The camera
driver only affects Pi CPU / Wi-Fi. Decide on those grounds.
Acceptance
A recommendation (switch / don't switch) backed by the measured Pi CPU + temp +
bandwidth numbers above, and — if switching — a scoped list of the launch/config
changes and the hardware re-validation steps.