Perception and compute
Connect viewpoint → capture host → calibrated observation → application. Control executes robot operations; your application processes images and decides the task.
Viewpoints
Head and wrist views cover the scene and grasp. The eight-camera example adds viewpoints around a PLEXA One S1 on the Gessmann M1 AMR.
Perception for your application
At the grasp
Three stereo cameras combine head context with close tool views. Each stream carries its identity, time and calibration into the grasp logic.
Three views → time / frame alignment → grasp logic
Camera reference: resolution and depth
| Stereo camera · wide 2.2 mm optics | Image per eye | Depth range · ideal range | Specified depth error |
|---|---|---|---|
| ZED X Mini · 50 mm baseline | 1920 × 1200 at 15/30/60 fps | 0.1–8 m · 0.1–4 m | < 1.6% at 2 m; < 3% at 4 m |
| ZED X · 120 mm baseline | 1920 × 1200 at 15/30/60 fps | 0.3–20 m · 0.3–12 m | < 0.8% at 2 m; < 4% at 12 m |
Depth ranges and errors apply to the stated optics and reference distances. Texture, lighting, motion and calibration determine application results; narrow optics have different ranges. Compare cameras · Depth specifications.
Capture and compute
| Camera path | Host and processing |
|---|---|
| ZED X Mini / ZED X | GMSL2 → compatible capture / deserializer → Jetson → ZED SDK |
| ZED Mini / ZED 2i | USB 3 → supported NVIDIA host → ZED SDK |
| OAK 4 D | Ethernet / USB → DepthAI, including device-side processing |
A Jetson can publish GMSL2 observations to a separate x86 application host. The ZED Box 2 Max offers eight native GMSL2 inputs. Its capture bandwidth supports eight stereo cameras at 1920 × 1200 / 30 fps or four at 60 fps; real-time processing also depends on SDK modules and Jetson power mode. Choose workstation, industrial PC or embedded compute by GPU memory, bandwidth, power and cooling.
- CameraHead · wrist · scene
- Capture / driverGMSL2 · USB · Ethernet
- Application hostGPU PC · IPC · Jetson
- ObservationTime · frame · result
Compare camera drivers
- 01ZED X Mini
- 02GMSL2 + Jetson
- 03Network stream
- 04x86 application
GMSL2 needs compatible capture hardware and a supported Jetson. An x86 consumer uses a separate qualified capture host and network path.
Intrinsics, extrinsics and clock mapping give the image stream its metric, spatial and temporal meaning. Explore the three mappings.
Planning profiles for capture paths and data mapping. Document each physical camera with its identity and calibration record.
Data available to your application
| Observation | Representation and context |
|---|---|
| Stereo / colour | Left/right or rectified images, image mode and capture timestamp |
| Metric depth | 32-bit floating-point Z-depth aligned to the left image, explicit units; optional confidence map and XYZ/RGBA point cloud |
| System context | Sensor identity, clock mapping, task-frame transform, calibration version, validity and measured observation age |
The ZED SDK depth API exposes measurement data; its 8-bit depth preview is for display. Your application consumes observations over its chosen process, network or ROS 2 channel and requests robot operations through the documented Robot APIs.
A configured SICK nanoScan3 Pro I/O provides the separate protective path. Field dimensions and stopping behaviour belong to Safety & commissioning; camera observations feed the application logic. Keep capture settings and sensor evidence with the configuration.