I am currently working on creating a digital twin of a real-world ZED X camera in NVIDIA Isaac Sim. I noticed that Stereolabs provides certified support and extensions for the ZED X within Isaac Sim, so I wanted to ask for guidance regarding camera intrinsics configuration.
While trying to replicate the real camera intrinsics by modifing values in Isaac Sim under ZED X camera properties section PinholeOpenCV, I encountered an issue related to the optical center parameters. Specifically, when I use the real-world optical center X from left camera value (cx = 961.542), the depth values generated through the ZED SDK appear incorrectly scaled and inconsistent.
For the purposes of my research, I temporarily set the optical center X from left camera equal to the optical center X value from right camera which is 971.565, which stabilizes the depth output. However, this introduces noticeable lateral offsets in object detection, especially at larger distances when comparing the outputs with real world ZED X camera.
I would like to ask whether there is a specific procedure or recommended approach for correctly configuring camera intrinsics in Isaac Sim so that the simulated ZED X matches the real-world camera as accurately as possible.
A real ZED camera cannot be simulated in Isaac Sim by entering custom intrinsics: the calibration parameters used by the ZED SDK are hardcoded per camera model and cannot be overridden from the simulator side. This is an intentional design choice to prevent misuse of the feature, so any value you change in the PinholeOpenCV block of the camera prim is not propagated to the SDK’s depth pipeline. That’s why your edits to cx produced inconsistent depth and lateral offsets — the rendering side and the SDK side are using two different sets of intrinsics.
In practice, this is not a limitation for the typical use cases of the digital twin. Even if it were possible to inject the exact intrinsics of your physical unit, the simulation would still not reproduce reality one-to-one (sensor noise, optical effects, rolling shutter, etc. are not modeled at that level). For evaluating and testing algorithms — perception, depth-based logic, navigation, control loops — the fixed model-level intrinsics are sufficient and consistent.
If your goal specifically requires the exact intrinsics of your physical unit (for example, projecting real-camera detections onto simulated geometry), the right approach is to keep the simulation on the SDK’s fixed model intrinsics and handle the small per-unit offset on your side, rather than trying to match them inside Isaac Sim.