Skip to content

Research: evaluate replacing v4l2_camera with usb_cam #19

Description

@MJohnson459

Summary

Research whether to replace the camera driver v4l2_camera with usb_cam. This
came out of the L1 depth-perception latency work: v4l2_camera turned out to be
limited in ways usb_cam is not, but a driver swap has real integration and
re-validation cost, so we want a decision backed by measurement, not a blind
switch.

Why this came up

While profiling the depth pipeline we tried to cap the camera at 20 fps (to cut Pi
CPU + Wi-Fi; it does not help depth latency, which is inference-bound). We found:

  • The pinned v4l2_camera 0.7.1 (the only version on robostack-jazzy) has no
    frame-rate parameter
    and never calls VIDIOC_S_PARM — confirmed in the binary
    (params are only video_device, image_size, pixel_format, output_encoding, camera_info_url, camera_frame_id). So capture rate is not settable in-driver.
  • Current per-frame cost on the Pi: USB YUYV → (v4l2) YUYV→RGB8 → (image_transport)
    RGB8→BGR→JPEG
    for /image_raw/compressed, i.e. two colour conversions + a JPEG
    encode every frame (the "slow conversion yuv422_yuy2 => rgb8" WARN). Setting
    output_encoding: yuv422_yuy2 doesn't help — the compressed plugin just converts
    to BGR itself before encoding.
  • usb_cam 0.8.1 is available on robostack-jazzy and has a native framerate
    param plus flexible pixel_format (incl. MJPEG) and more camera controls, with a
    larger user base / better docs.

Research questions

  1. Does the UGREEN webcam expose MJPG @ 640x480 (v4l2-ctl --list-formats-ext)?
  2. Can usb_cam deliver the camera's native MJPEG straight to /image_raw/compressed
    without the YUYV→RGB→JPEG round-trip? (This, not the frame-rate cap, is the real
    potential Pi/Wi-Fi win.)
  3. Measured Pi CPU % and SoC temperature under pixi run launch: v4l2_camera
    (YUYV, ~28 fps) vs usb_cam (MJPEG, 20 fps). Is the camera actually straining the Pi?
  4. Wi-Fi bandwidth of the compressed stream in each config.
  5. Integration cost: re-map params in mote_launch.py + robot.yaml; verify the
    /image_raw/compressed topic name the depth node subscribes to, the
    camera_info_url / ~/.mote/camera_calibration.yaml calibration plumbing, frame
    ids, timestamping, and QoS. Note usb_cam pulls in ffmpeg/libavcodec (heavier dep).

Non-goal / context

  • This is not a depth-latency optimisation — end-to-end latency is dominated by
    off-board inference (measured ~458 ms quiet, ~2 s under CPU contention). The camera
    driver only affects Pi CPU / Wi-Fi. Decide on those grounds.

Acceptance

A recommendation (switch / don't switch) backed by the measured Pi CPU + temp +
bandwidth numbers above, and — if switching — a scoped list of the launch/config
changes and the hardware re-validation steps.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions