Inconsistent Timestamp

Hi,

We are working with a Zed2i.
We try to geolocate photos taken with the camera.

We use a GPS RTK, to get associated lon/lat to a timestamp. Both the GPS acquisition and Zed acquisition are launched by the same script, on the same computer (so same gmtime). We are driving a car.
When we compare the recorded timestamps, we observe a 4.6s delay between the Zed image and the GPX track. i.e. the Zed timestamp is late.

We are sure there is a delay : we compared the frames with photos from Google Street view and found the Zed frame is late, compared to where we are supposed to be according to the GPS.

When reading the documentation Camera Class Reference | API Reference | Stereolabs we understand that a time difference of ~0.3s may be normal, due to transfer between zed and the PC ram. But we do not understand the 4.6s delay…

We benchmarked the GPS and we absolutely trust its measurements.

Do you have any idea how to improve the Zed timestamp ?

Thanks

Hi @Francois
the ZED SDK uses the operating system time as a reference for ZED data timestamping.
If the OS clock is not synchronized with the GPS clock, then the ZED and GPS data will be misaligned.

You must disable the NTP server, so the OS does not update the system time by using the internet servers and configure a service to use the GPS as the time reference.
You can read more, for example, here:

Hi,

I double checked, and the problem does not come from the GPS, which uses the system time. (we get a timestamp from satellites, but it is not the timestamp we use).

I can ensure you, the timestamp gap does not come from the GPS.

Here, I read " When requesting the TIME_REFERENCE.IMAGE timestamp, the UNIX nanosecond timestamp of the latest grabbed image will be returned. This value corresponds to the time at which the entire image was available in the PC memory."

What could cause a delay ? Disk writing speed ? Too many images in RAM and a queue that cannot be consumed in real time ?

Are you saving the GPS data and the ZED data using the same machine?
A delay of 4.6s cannot be justified by buffering or by write speed because the stamping is performed as soon as the data is available to the SDK.
I need more information regarding the configuration of your processes to better understand the possible causes of the problem.

Yes.

I need more information regarding the configuration of your processes to better understand the possible causes of the problem.

What information do you need ?

To sum it up, we have two subprocesses, running in parallel.

  1. The GPS gets lon/lat coordinates of the car, and writes them in a .gpx file with the system timestamp
  2. The Zed Recorder records a .svo and saves it to disk. (15FPS, 2k)

Note that the acquisition may last up to an hour, and longer in the future.

Later, when we postprocess these GPS/SVO files, and compare results with Google Street View.
For a given timestamp, there is a space difference (~6m, 55 frames or 3.8s, depends how you count it).

Hi,
Can you precise the OS (windows, linux or jetson) ? as the camera acquisition (i.e. timestamping) is not made the same way.

1 Like

I work on a Ubuntu 22.04

Erratum. The acquisition was made on a Windows PC.

My apologies

As it happens, the problem may come from the computer itself.

I made several tests. On a screen, I real-time print the current timestamp.

I have the Zed record the screen. All the acquisitions are made with HD2K/15fps, via the Zed recorder.

Then, I calculate the time difference between the timestamp printed on the screen, and the svo timestamp I got with zed.get_timestamp(sl.TIME_REFERENCE.IMAGE).get_milliseconds()

  • On my computer (Ubuntu), I have a timedelta of approx. 75ms
  • On a mini PC (Ubuntu), with lesser performances, the timedelta is approx. 375ms

On my computer :

frame camera timestamp timestamp on the screen delta
100 1674729002.484 1674729002.38 0.104
500 1674729029.204 1674729029.09 0,114
1000 1674729062.41 1674729062.488 0,078
1500 1674729097.288 1674729097.298 0.01
2000 1674729130.497 1674729130.516 0,019
2500 1674729163.77 1674729163.867 0,097
3000 1674729197.178 1674729197.284 0,106

On the mini PC :

frame camera timestamp timestamp on the screen delta
100 1671192866.015 1671192866.491 0.475
500 1671193029.854 1671193030.25 0,396
1000 1671193236.913 1671193237.055 0,142
1500 1671193462.348 1671193462.73 0,382
2000 1671193685.793 1671193686.124 0,331
2500 1671193924.279 1671193924.82 0,541

Which part of these PC do you think is responsible for this behaviour ? Should we improve disk writing speed, CPU frequency, or RAM speed to get closer to real-time acquisitions ? Is an OS better than another ?

Hi, this is really important for us. How can we have the smallest timedelta possible, and make it constant ?

I’m not related to Stereolab, so my comment is just based on what we do in our projects that may be helpful for you.

We’re using a GPS and ZED 2 on a Jetson where the Jetson doesn’t have access to the internet, and when it’s turned off, its clock loses track of time. What we do is to update the clock on the Jetson by grabbing the first GPS observation (our GPS update rate is 1 Hz). This way, we’ve never had any issue with synching the GPS and camera times. The issue you’ve come across may, however, be due to the way your multiprocessing pipeline is working, and maybe some kind of lock that happens when you retrieve images or do some other processing that causes the camera to fail to catch up the GPS.

Thanks, this may be a cause of the problem. We’ll try.