Skip to content

Use with AI

Use with AI

Perception and compute ​

Connect viewpoint → capture host → calibrated observation → application. Control executes robot operations; your application processes images and decides the task.

Viewpoints ​

Head and wrist views cover the scene and grasp. The eight-camera example adds viewpoints around a PLEXA One S1 on the Gessmann M1 AMR.

At the grasp
ZED X Mini · GMSL2 · example positions

Perception for your application

At the grasp

Three stereo cameras combine head context with close tool views. Each stream carries its identity, time and calibration into the grasp logic.

Three views → time / frame alignment → grasp logic

Camera reference: resolution and depth ​

Stereo camera · wide 2.2 mm opticsImage per eyeDepth range · ideal rangeSpecified depth error
ZED X Mini · 50 mm baseline1920 × 1200 at 15/30/60 fps0.1–8 m · 0.1–4 m< 1.6% at 2 m; < 3% at 4 m
ZED X · 120 mm baseline1920 × 1200 at 15/30/60 fps0.3–20 m · 0.3–12 m< 0.8% at 2 m; < 4% at 12 m

Depth ranges and errors apply to the stated optics and reference distances. Texture, lighting, motion and calibration determine application results; narrow optics have different ranges. Compare cameras · Depth specifications.

Capture and compute ​

Camera pathHost and processing
ZED X Mini / ZED XGMSL2 → compatible capture / deserializer → Jetson → ZED SDK
ZED Mini / ZED 2iUSB 3 → supported NVIDIA host → ZED SDK
OAK 4 DEthernet / USB → DepthAI, including device-side processing

A Jetson can publish GMSL2 observations to a separate x86 application host. The ZED Box 2 Max offers eight native GMSL2 inputs. Its capture bandwidth supports eight stereo cameras at 1920 × 1200 / 30 fps or four at 60 fps; real-time processing also depends on SDK modules and Jetson power mode. Choose workstation, industrial PC or embedded compute by GPU memory, bandwidth, power and cooling.

From image to observation
  1. 01
    CameraHead · wrist · scene
  2. 02
    Capture / driverGMSL2 · USB · Ethernet
  3. 03
    Application hostGPU PC · IPC · Jetson
  4. 04
    ObservationTime · frame · result
Compare camera drivers
EXPLORE THE CONCEPTInteractive teaching model
  1. 01ZED X Mini
  2. 02GMSL2 + Jetson
  3. 03Network stream
  4. 04x86 application

GMSL2 needs compatible capture hardware and a supported Jetson. An x86 consumer uses a separate qualified capture host and network path.

Intrinsics, extrinsics and clock mapping give the image stream its metric, spatial and temporal meaning. Explore the three mappings.

Planning profiles for capture paths and data mapping. Document each physical camera with its identity and calibration record.

Data available to your application ​

ObservationRepresentation and context
Stereo / colourLeft/right or rectified images, image mode and capture timestamp
Metric depth32-bit floating-point Z-depth aligned to the left image, explicit units; optional confidence map and XYZ/RGBA point cloud
System contextSensor identity, clock mapping, task-frame transform, calibration version, validity and measured observation age

The ZED SDK depth API exposes measurement data; its 8-bit depth preview is for display. Your application consumes observations over its chosen process, network or ROS 2 channel and requests robot operations through the documented Robot APIs.

A configured SICK nanoScan3 Pro I/O provides the separate protective path. Field dimensions and stopping behaviour belong to Safety & commissioning; camera observations feed the application logic. Keep capture settings and sensor evidence with the configuration.