Sorry if this has been covered elsewhere. I looked around and did not see anyone with this issue.
A little background. I am using 4 Zed X with Jetson Orin AGX 64. When I first set this system up, the Jetson was unable to handle all 4 cameras in ROS/ROS2 reliably. I ended up setting the Jetson up as streamer, and then processed the stream on a larger GPU (A6000) on the same vehicle. This worked pretty reliably, but even the A6000 seemed to struggle if I ran all 4 cameras in ROS at anything higher than NEURAL_LIGHT.
Fast forward a bit, and things got remarkably better with SDK updates. 5.1+ has been a lot better on the big computer. But I was dropping many frames when recording SVOs. I thought this might be because it gets compressed to stream, then decompressed on the host then recompressed to record (though I am not sure if this is correct).
I decided to try setting up the Jetson again with updated SDK to see how it did, thinking if it could handle compressing for streaming it should be able to handle compressing for SVO. I I have fresh JP6.2/ROS2 Humble install. I had to set the ROS2 wrapper settings very “light weight” in order to get this to work. I set resolution to HD1200, capture rate to 30 FPS, and NEURAL_LIGHT. I set all the publishing frequencies to < 1Hz. I set the compression to Lossy H265. I recorded a few minutes like this and had no dropped frames. I went out and drove the vehicle and recorded again. Suddenly I have tons of dropped frames. I had an effective frame rate in the SVO < 2Hz. I attribute this to maybe the compression can do some caching when the vehicle is stationary that it can not do when in motion? But this seems excessive.
On top of that, it seems I can not open the cameras at all using HD1080 and they often fail to open in SVGA. I feel like I am doing something really wrong.
Is there a better way to handle recording through ROS2? I can look into writing a simple recording only script, but integration with ROS is nice to consolidate the recording calls to a single location. (I am using this data with Lidar/GNSS/etc… all working in ROS).
What would cause the cameras to not open when using resolutions other than HD1200?
Should a Jetson Orin be able to stream 4 cameras reliably or am I asking for too much from the computer?
When I call record I set:
bitrate: 0
target_fps: 0
compression_mode: 0
input_transcode: false
Which should choose appropriate defaults I believe? My zed config file and a terminal log launch using HD1200 followed by a failed launch using HD1080 are also attached.
launch.log (274.7 KB)
One last thing. I should mention that this is custom carrier board (NRU-230) but the drivers do not seem to be an issue, at least getting connected to the cameras initially.
zed_record.yaml (23.5 KB)
Thank you!
Benjamin