ZedX not auto adjusting?

Software ROS1
1

I have ZedX cameras set up as remote streamers on a Jetson Orin. I receive the stream a different PC using the ROS wrapper. The cameras are opened through Docker containers on the Jetson if that is important. This all works well, except that if I start the cameras while the vehicle is indoors and drive outside(or vice versa) the cameras don’t auto adjust their parameters and I end up with an image that is either blown out or very dark. I did not have this issue when using zed2i directly on the PC.

I am setting autowhite balance and auto exposure in the common.yml and I don’t believe I am over writing this anywhere.
Do I need to set these parameters when opening the cameras on the Jetson or does the receiver have control to set these?
Is this a known issue?

Remote Streamer: Jetson Orin AGX 64, SDK 5.0.1, L4T 36.3

Host: Ubuntu 20, SDK 5.0.1

Thanks!
Benjamin

2

Hi @benijohn
Welcome to the StereoLabs community.

What version of the ZED X Driver are you using?
Please upgrade the ZED SDK to the latest v5.0.5.

3

Hey Walter,

My apologies for the delay. I am using driver version 1.3. I will try upgrading to the newer SDK.

I also did run a couple of experiments and confirmed that when using ZED_Explorer, I can manually adjust the settings. I also tried opening the camera directly on the Jetson(instead of through a container) and I get the same behavior, the camera does not automatically adjust the exposure when lighting changes.
I will update here if the newer SDK changes anything.

Thanks!

Benjamin

4

Updating to 5.0.5 did not change the behavior. Do I need to set these settings when the camera is opened? Or connecting through ROS should handle setting those values? From what I can tell when I open the stream through ROS the settings are being taken from the yaml file. I would assume this means that auto white balance and auto exposure should also be taken from the file.

Is there something else I can try or look at?

Thanks,

Benjamin

5

Hello,

Is there something I can try to debug this, or this expected behavior when using the ROS wrapper to open remote streams?

Thanks,

Benjamin

6

Hi @benijohn,

Sorry for the delay.

This is indeed not the expected behavior, auto-exposure should be functional be default regardless of being used in streaming or not.

Can you confirm that the auto exposure is working as expected in ZED_Explorer or another application, with the setting set to ON?

7

@mattrouss thanks for your reply. I ran some tests using ZED_Explorer, and confirmed that yes the auto exposure works when connecting through explorer, but there was some interesting behaviors that may or may not be related to the issue with it no working in ROS.

I tried three different configurations in ZED_Explorer:

  1. connecting directly to the camera

2. connecting to the stream on the same computer(Jetson)

3. connecting to the stream from the remote computer(X64)

When connecting to the streams, passing the stream parameters -s -p did not work(I might be using these wrong) but I passed -s IPAddress -p portnumber and on both machines it said it could not find a camera, however opening ZED_Explorer and using the GUI to connect to the stream worked on both local and remote computers. Additionally, though the auto exposure was working in all three cases, the camera settings were different for the streaming cameras as opposed to the camera opened directly.

On the streaming cameras(same for both local and remote) the camera settings read:
Gain: 0
Exposure: 0

Exp Range: 0-0

Analog Gain Range: 0-0

Digital Gain Range: 0-0

Denoising: 0

Exp. Comp: 0

However, when connecting directly to the camera these values were different. In this case(darkish room) they read:

Gain: 34
Exposure: 92

Exp Range: 28-66000

Analog Gain Range: 1000-16000

Digital Gain Range: 1-256

Denoising: 50

Exp. Comp: 50

also, if I turned off auto exposure and adjust the exposure values and then turn auto exposure back on, it would return to these values. So it does appear to be working in this case.

One more thing that may or may not be of interest. In both cases when connecting to the stream, I received a warning in the terminal:

Corrupted frame chunk received (recv: 196 / expected 16100)

additionally when opening stream on the Jetson I got an additional output:
Metadata timeout. the size is equal to 196 instead of 21960

Let me know if there is anything else you like me to try or further information you need.

Thank you!

Benjamin

8

Hi @benijohn,

In order to use a streaming input in the ros2 wrapper, please use the following command:

ros2 launch zed_wrapper zed_camera.launch.py stream_address:="127.0.0.1" stream_port:=30000

You should be able to see the log
[component_container_isolated-2] [INFO] [zed.zed_node]: === LOCAL STREAMING OPENING ===

Please note that when opening a stream, the ros2 wrapper will set video parameters according to what has been set in the config yaml files, overriding any parameters set while opening the camera.

9

Hey sorry for any confusion I have caused. In this case I am using ROS1 and I do have my launch file configured to open the camera streams. In that case I do point it to a config file which has the settings:

auto_exposure_gain: true

auto_whitebalance: true

I can see they are being set using rosparam get. So I am pretty sure the cameras are set to use auto. Is there something else I should look at to see if the camera is being properly set to use auto?

Thanks!

Benjamin

10

Hi @benijohn,

The best way to know where this is going wrong is to open a camera in the ROS1 wrapper without streaming. With this you will see if it’s a parameter propagation issue in the ROS1 wrapper, or an issue with the streaming.

11

Hello,
Sorry for the very long delay in getting back to you on this. I somehow corrupted my Jetson due a Firefox usse and I have only just got it back up and mostly running again. Since re installing L4T, I moved the streams from Docker to system services. Other than that everything should be mostly the same. Because I am using JetPack6 I was not able to install ROS, so ran these experiments on ROS2. Opening the camera natively in ROS2, everything works great and I can see the exposure is changing in rviz. If I open the camera locally in ros2 but use the local stream as the source it still seems to work but seems quite a bit slower(maybe the encoding/decoding is slowing it down?) But I did get some different warnings in the terminal when using the stream. I am attaching the terminal outputs for both launching from stream and launching natively(GMSL). When using the stream I got a lot of this:

[component_container_isolated-2] [WARN] [1756667761.578329753] [zed.zed_node]: Error setting AUTO DIGITAL GAIN RANGE: MODULE NOT COMPATIBLE WITH CAMERA
[component_container_isolated-2] [WARN] [1756667761.912284572] [zed.zed_node]: Error setting AUTO EXPOSURE TIME RANGE: MODULE NOT COMPATIBLE WITH CAMERA
[component_container_isolated-2] [WARN] [1756667761.912379007] [zed.zed_node]: Error setting AUTO ANALOG GAIN RANGE: MODULE NOT COMPATIBLE WITH CAMERA
[component_container_isolated-2] [WARN] [1756667761.912400928] [zed.zed_node]: Error setting AUTO DIGITAL GAIN RANGE: MODULE NOT COMPATIBLE WITH CAMERA

I opened the camera like this:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx stream_address:=130.127.116.149 stream_port:=30004

Could this have anything to do with the issue I am seeing when opening the streams in ROS1?

Thanks!

Benjamin

12

Hi @benijohn

This is a known issue. A fix will be available with ZED SDK v5.1.

13

@Myzhar do you the error I am seeing or the issue with Exposure not adjusting when moving to different lighting conditions?

I have retested the system on the vehicle and can confirm that the exposure adjustment does work with ROS2, but not with ROS1. This is when streaming to a remote system and using the ZedWrapper on the remote system. So, will the ROS1 issue be resolved in 5.1 or should I look at moving to ROS2?

Thanks!

14

The ZED ROS Wrapper code is no longer maintained.
You can debug it to search for the eventual bug and push a pull request if you fix it.
We will check and merge the PR, but we are no longer working on that code because ROS has reached EOL.