I’m developing a multi-camera video compositing application for vehicles using ZED X One cameras on NVIDIA Jetson Orin AGX. The application needs to dynamically switch between different camera view layouts at runtime based on operator commands, similar to switching between different security camera monitoring views during operation.
Currently I’m using nvarguscamerasrc elements feeding into an nvcompositor for real-time video composition, followed by hardware H.265 encoding and UDP streaming. The pipeline works well for static layouts, which produces 30fps encoded streams. However, I’m encountering significant issues when attempting to dynamically reconfigure the compositor layout properties at runtime.
The core problem is that when I try to update nvcompositor sink pad properties like xpos, ypos, width, height, and alpha to change camera positioning and scaling during live operation, the video feed freezes completely. The pipeline doesn’t crash and no errors are reported, but data flow stops entirely after the property updates. I can see in my logs that the g_object_set calls complete successfully and querying the properties back shows they were set correctly, but no new frames are produced.
I’ve tried multiple approaches including pausing the pipeline before updates, using pad blocking mechanisms, sending custom events to force buffer flushing, and calling gst_pad_mark_reconfigure, but none restore proper data flow. The issue appears to be that nvcompositor doesn’t handle dynamic property changes gracefully during live operation.
Given that ZED cameras have their own GStreamer plugins, I’m wondering if using zedxonesrc instead of nvarguscamerasrc might provide better support for dynamic layout switching. Specifically, I’d like to know if the ZED GStreamer plugins offer any advantages for runtime video composition scenarios, such as better integration with standard GStreamer compositor elements or built-in support for dynamic layout changes.