I was able to sucessfully fuse GNSS with VIO, but I am not able to get the fused data in lat long?

[tarun]

I used GeoPose() command to get this:

<pyzed.sl.GeoPose object at 0x000001F915789A80>
<pyzed.sl.GeoPose object at 0x000001F915789940>
<pyzed.sl.GeoPose object at 0x000001F915789A80>
<pyzed.sl.GeoPose object at 0x000001F915789940>

Can someone please help me on how to get the fused lat, long output?. Thank you

[tarun]

GNSS-VIO fusion.ipynb (14.9 KB)

[mattrouss]

Hi @tarun,

Please take a look at the documentation for the GeoPose object: GeoPose Class Reference | API Reference | Stereolabs

This contains the latitude and longitude information that you are looking for.

[tarun]

That isn’t working for me, not sure why.
But I was able to get the lat lng data via math calculations.

Thank you!!

[mattrouss]

Can you please share more info on what you were not able to do, as this may help other users who have a similar issue?

You can retrieve the sl.LatLng object which contains latitude, longitude and altitude data using:

current_geopose = sl.GeoPose()
current_geopose_satus = fusion.get_geo_pose(current_geopose)

# Retrieve LatLng object
lat_lon = current_geopose.latlng_coordinates()
print(f"Lat: {lat_lon.get_latitude()}, Lon: {lat_lon.get_longitude()}, Alt: {lat_lon.get_altitude()}")

[tarun]

So, I have acquired GNSS data from my mobile phone and transmitted it to my Laptop through WiFi.
I have used to lat, lng data and VIO from ZED to localize my wheeled mobile robot in global map (Lat, Lng based earth map). I have used the GeoPositioning tutorial code from ZED python Github.

Problems I am facing:

1. The fused lat, lng data is not calibrated properly, im not sure if its a sensor problem or the way i collected the data. If the way i collected is the problem, I have planned to use a gimbal and collect a more stable data in few days. In the picture I have attached the way I took is along the path way, but its showing like I have gone through buildings. For few cases I was able to correct it with 2d rotation matrix, but while collecting multiple iterations, the change in angle is not similar. So not sure what im gonna do.
   

   

2. I am a undergraduate student whos working on the project, So there can be a lot of mistakes in my code, and also few parts of the code I am not able to understand. If anyone can please teach me on how it works, and how to correct it (I am open for any form of teaching, Zoom calls, anything…).

3. I used the code fix that you have provided which gives me TypeError: ‘pyzed.sl.LatLng’ object is not callable.

GNSS-VIO fusion (1).ipynb (10.5 KB)

Thank you.

[tarun]

Update,

I used an DJI Ronin gimbal to stabilize the ZED 2i and collect the data, but still there seems to be an offset, not sure what to do now.

This is the ground raw GPS data (using an iphone 11 with GPS2IP application running on it).

The black and white dots are the VIO-GPS fused data, and the red line is how I actually walked (the red line I added using MS paint so that you can have a better understanding).

Sincerely,
Tarun.

[mattrouss]

Would it be possible for you to share the SVO + GNSS data of these sequences in order to try and reproduce on our end?

We have worked on improvements on our geotracking algorithms for the future release of the ZED SDK, which should limit such issues.

[tarun]

I don’t have the SVO file but these are the data that I collected.
But please let me know if I want any other data, I can collect them.
Is my code 100% correct maybe something wrong there?
Output_Data.xlsx (43.2 KB)

[mattrouss]

Hi @tarun,

I took a look at your code, the issues you are having are because you set the covariance values to 1 in the diagonal.

These values are used as “confidence” values to fuse the GNSS and ZED SDK tracking data, where low values are confident, and high values are not confident.
Can you please try lowering this value and check if this changes the behavior.

If this does not work, can you please share the values reported by this method:
get_current_gnss_calibration_std, to see if this is a GNSS calibration initialization issue.

[tarun]

Hello @mattrouss ,

Thank you for taking your time in between your busy schedule,

I went about lowering the diagonal value by changing the eph & epv values to lesser numbers, as shown below,

I did 3 iterations with different values i.e,

1. eph = 0.5, epv = 1; for which i got the output data like this:
   

   

2. eph = 0.3, epv = 0.3
   

   

3. eph = 0.01, epv = 0.01
   

   

There seem to be not much in change, I am yet to try the get_current_gnss_calibration_std method, will get back to you once I did that.
Meanwhile I have collected the .svo file and raw_gps data as you asked previously, maybe try fusing it at your end and please share the results. (the data are not time synchronized i.e, I started the GPS receiver 10-20 seconds before the stereo recording)

HD1080_SN30507109_18-20-28.svo

The drive has the SVO video.
raw_latlng.txt (28.5 KB)

This has the lat, lng, easing, northing.

Thank you,
Tarun

[mattrouss]

I’ve noticed in your code that you do not provide the timestamped gnss data when ingesting to Fusion. This is necessary information that is required by the ZED SDK in order to synchronize and fuse GNSS and camera data.