Nvcompositor freezes on dynamic pad-resize when switching multi-camera view modes

I’m developing a multi-camera video compositing application for vehicles using ZED X One cameras on NVIDIA Jetson Orin AGX. The application needs to dynamically switch between different camera view layouts at runtime based on operator commands, similar to switching between different security camera monitoring views during operation.

Currently I’m using nvarguscamerasrc elements feeding into an nvcompositor for real-time video composition, followed by hardware H.265 encoding and UDP streaming. The pipeline works well for static layouts, which produces 30fps encoded streams. However, I’m encountering significant issues when attempting to dynamically reconfigure the compositor layout properties at runtime.

The core problem is that when I try to update nvcompositor sink pad properties like xpos, ypos, width, height, and alpha to change camera positioning and scaling during live operation, the video feed freezes completely. The pipeline doesn’t crash and no errors are reported, but data flow stops entirely after the property updates. I can see in my logs that the g_object_set calls complete successfully and querying the properties back shows they were set correctly, but no new frames are produced.

I’ve tried multiple approaches including pausing the pipeline before updates, using pad blocking mechanisms, sending custom events to force buffer flushing, and calling gst_pad_mark_reconfigure, but none restore proper data flow. The issue appears to be that nvcompositor doesn’t handle dynamic property changes gracefully during live operation.

Given that ZED cameras have their own GStreamer plugins, I’m wondering if using zedxonesrc instead of nvarguscamerasrc might provide better support for dynamic layout switching. Specifically, I’d like to know if the ZED GStreamer plugins offer any advantages for runtime video composition scenarios, such as better integration with standard GStreamer compositor elements or built-in support for dynamic layout changes.

Hi @Lrnts
Welcome to the StereoLabs community

The zedxonesrc source element provides all the features available with the sl::CameraOne object of the ZED SDK.
You can read more in the GStreamer documentation.

From the documentation and some preliminary tests, it appears that zedxonesrc lets me adjust exposure, gain, white-balance and the AEC/AGC ROI on the fly, but geometry parameters like resolution, frame-rate or sensor mode will remain fixed once the pipeline is opened and any change still requires a reset and recreation which introduces a stream interruption. Dynamic layout tasks such as hiding, showing, moving or resizing the feed therefore continue to be handled downstream in the compositor by updating position, dimension, or visibility properties. From what I understand, replacing nvarguscamerasrc with zedxonesrc would grant the ZED SDK functionality but still leave layout management unchanged. Does this match your understanding, or is there a recommended way to push new caps through the pipeline without a full teardown?

This is expected. They are not runtime parameters, you must close the camera and re-open it with new settings if required.

The only way to achieve what you need is by using Image Processing elements.

1 Like