Fusion and objects' position

Software
1

Hello,
I am trying to locate centroid of an object i see in the image.
I understood how to translate from camera coordinates to lat/long but I am struggling a bit on obtaining the camera coordinates of desired points.
I have the following questions:

  1. If I am using only one camera and no gnss is there a difference between camera computed position and fusion one? (e.g fusion uses the IMU inside the camera to improve localization)?
  2. If I enable gnss fusion, and I get the positions from :
fusion.get_position(fused_position)

will it be different from the position I get from zed.get_position or not since the gnss is employed only to compute the calibration between VIO and GNSS? I mean does gnss data somehow influence the position even in world coordinates (not lat/long)?

  1. Ultimately what I need is:
fusion.retrieve_measure(point_cloud_fusion,uuid,sl.MEASURE.XYZ)

because I need the real world coordinates of objects I detect in the image. When I do that how do I know which frame the retrieved measure refers to ? if I run

point_cloud_fusion.timestamp.get_nanoseconds()

I always get 0. Furthermore sometimes I get nan data.
I could do

zed.retrieve_measure(point_cloud,sl.MEASURE.XYZ)

but then I would get less accurate positioning right?

2

Hi @Prospecto,

  1. If you are using only one camera, the output pose from the camera and fusion will be the same, as only the positional tracking of the camera is taken into account. The positional tracking from the camera already uses the IMU data to track its position by default

  2. Fusing the camera tracking and GNSS data will have an impact on the position and accuracy, as fusion will use whichever is the most reliable to update the global position. World and lat/long coordinates represent the same data, just in a different coordinate system.

  3. All fusion data is returned in “Fusion World” which is determined by the GNSS/camera calibration, and the global positioning algorithm. So this seems to be the reference frame that you wish to retrieve your data in. By retrieving it with zed.retrieve_measure you will retrieve it in “camera world” which has not been corrected by GNSS data and therefore less accurate yes.

3

But if I do:

fusion.retrieve_measure(point_cloud_fusion,uuid,sl.MEASURE.XYZ)

How do I know the timestamp of the image of which i am retrieving the measure?

Actually the best thing would be being able to retrieve the measure of a specified timestamp in order to obtain the measure after the gnss and vio calibration has been performed. Is that possible?

4

The issue you are experiencing has been fixed internally, and is currently planned to appear in the version 4.2 of the ZED SDK.

In the meantime, I can suggest using:

fusion.getCurrentTimestamp()

https://www.stereolabs.com/docs/api/classsl_1_1Fusion.html#ad380aebc08e4784f8f574b7c04ecdfc5

5

I get the following

Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'pyzed.sl.Fusion' object has no attribute 'getCurrentTimestamp'here

I think that the function you are talking about is only for C++ and not for python…

6

Hi @Prospecto,

Sorry the methods are written in snake_case in the python API, so it should be: get_current_timestamp, however I have noticed that the method is actually missing from the API.

I have logged the issue internally so we can fix this method as well

7

So right now there’s no way I can retrieve the fusion timestamp?
Or is it missing just from the docs? Because I tried fusion.get_current_timestamp() but stil not getting anything.

8

That is correct there is no way to retrieve the timestamp from fusion, these will be available in the next patch of the ZED SDK.

I’ve checked and the python method is missing from the API entirely, not only the docs.

9

Ok, thanks for the infos.
So, is there a way I can know which image I am processing? I am doing object detection and I want to locate objects in real world. If I am gonna use fusion.retrieve_image and fusion.retrieve_measure are they gonna be synchronized?

10

All data coming from fusion after running fusion.process with a SUCCESS error code will be synchronized.

This applies to image, depth, positional tracking and body tracking data.