I am new to the ZED SDK and currently working on a project to generate point cloud data with a real-world coordinate system. I am using a ZED 2i camera with ZED SDK 4.2.5 to capture data in SVO2 format using the ZED Explorer tool.
For GNSS data, I am using an Advanced Navigation Spatial GPS configured in GPIO NMEA format. I have installed gpsd services on Ubuntu, and while GNSS data is being displayed correctly, I am encountering an issue where a GNSS JSON file is not being generated.
Additionally, I noticed that the “epv” (estimated vertical position error) value is not present in the GPIO NMEA format.
Could you please suggest the correct approach to generate the GNSS JSON file and guide me on how to handle the missing “epv” value in this format?
Any advice or recommendations would be greatly appreciated!
This sample does not write the GNSS data explicitely as a separate JSON file, but stores the data inside the SVO2 using our custom data feature, allowing you to have both ZED and external data inside the same file, and also to have this data synchronized.
Here is the method that is being used:
You can then retrieve the GNSS data with the API as well, with this method:
Ingested GNSS data into SVO2 file using the ZED camera and gnss.
Verified the data using the playback module, and everything looks correct.
We are able to generate KML files (raw_gnsss.kml & fused_position.kml) from the svo2 file, and the geolocation data is correctly creating paths from the points, visible on the map which states the ingestion is correct.
Additionally, we tried creating a PointCloud using the ZEDfu tool, and while we are getting a PointCloud, it is not georeferenced.
Next, our goal is:
To create a georeferenced PointCloud from the SVO file using the GNSS data.
My question is:
How can we generate a georeferenced PointCloud from the SVO file? We would greatly appreciate any resources or steps that can guide us through this!
Hi @nitingarg,
Based on my understanding of your issue, you want to use the ZED SDK to generate a geo-referenced point cloud. Is that correct?
Unfortunately, the ZEDfu tool does not currently support this feature. However, you can achieve this by writing some additional code. The process should be manageable. Here’s how you can do it:
Retrieve the Point Cloud: Use the ZED SDK functions to obtain the point cloud in the camera coordinate frame.
Convert to Geo-Referenced Coordinates: Utilize the GNSS-VIO calibration from the Fusion module to transform the point cloud into a geo-referenced coordinate system.
Here are some useful links to help you get started:
You’re absolutely right! My goal is to generate a geo-referenced point cloud with real-world coordinates in UTM or ECEF.
Here’s my current workflow:
I have an SVO2 file that includes latitude, longitude, and altitude (EPSG:4326).
I can successfully generate a point cloud (PLY) in camera coordinates using the spatial mapping module or ZEDfu.
From the SVO2 file, I can extract a geo-referenced KML file, which opens correctly.
I have also applied the Fusion module, and it works as expected.
However, I’m still unsure how to write the transformed (geo-referenced) point cloud back to a PLY file in UTM or ECEF coordinates.
I’m attaching my Python script for review. Could you please guide me on the necessary steps to properly convert and save the geo-referenced point cloud?
If I understand correctly, you’re asking how to write a geo-referenced point cloud into a PLY file. While this isn’t my area of expertise, I can offer some guidance.
Writing a geo-referenced point cloud to a PLY file shouldn’t differ much from writing a regular point cloud. You can write each point in x-y-z coordinates. However, due to the absolute coordinate system, the viewer used to read the PLY file might struggle with the large coordinate values, as it may expect smaller numeric values.
If this is the case, I suggest switching from ECEF (Earth-Centered, Earth-Fixed) coordinates to ENU (East, North, Up) coordinates, which will preserve lower numeric values.
Regarding your question about converting your point cloud from a ZED camera to a geo-coordinate system, you can use the Camera2Geo function. This will provide you with latitude and longitude coordinates. Then, you can use our converter to obtain ECEF coordinates.
Dear @TanguyHardelin,
Thank you for your reply it was really helpful that you focus to use ENU
I understand the functionalities you have discussed here for Camera2Geo & Converter, Let me explain in depth first:
Steps:
We have a svo2 file captured from global_localization/recording/recording.py module with GNSS ingested into it having GNSS key as “gnss_json”
Instead of opening the camera directly we started the initiation of camera from svo2 file using the below snippet:
file_gnss_position = sl.GeoPose
py_translation = sl.Translation()
pose_data = sl.Transform()
text_translation = ""
text_rotation = ""
init_params = sl.InitParameters(depth_mode=sl.DEPTH_MODE.ULTRA,
coordinate_units=sl.UNIT.METER,
coordinate_system=sl.COORDINATE_SYSTEM.IMAGE)
init_params.set_from_svo_file(input_svo_file)
# create the camera that will input the position from its odometry
zed = sl.Camera()
status = zed.open(init_params)
if status != sl.ERROR_CODE.SUCCESS:
print("Camera Open: " + repr(status) + ". Exit program.")
exit()
# Enable positional tracking:
positional_init = zed.enable_positional_tracking()
if positional_init != sl.ERROR_CODE.SUCCESS:
print("[ZED][ERROR] Can't start tracking of camera : " + repr(status) + ". Exit program.")
exit()
# set up communication parameters and start publishing
communication_parameters = sl.CommunicationParameters()
communication_parameters.set_for_shared_memory()
zed.start_publishing(communication_parameters)
Then we have started the Positional Tracking module by the following code:
tracking_params.enable_imu_fusion = True
tracking_params.set_gravity_as_origin = True
err = zed.enable_positional_tracking(tracking_params)
if (err != sl.ERROR_CODE.SUCCESS):
print("Camera positional tracking: " + repr(status) + ". Exit program.")
exit()
camera_info = zed.get_camera_information()
# step 2
# init the fusion module that will input both the camera and the GPS
fusion = sl.Fusion()
init_fusion_parameters = sl.InitFusionParameters()
init_fusion_parameters.coordinate_system = sl.COORDINATE_SYSTEM.IMAGE
init_fusion_parameters.coordinate_units = sl.UNIT.METER
init_fusion_parameters.output_performance_metrics = True
init_fusion_parameters.verbose = True
init_fusion_parameters.timeout_period_number = 66
fusion.init(init_fusion_parameters)
# Enable positional tracking for Fusion object
positional_tracking_fusion_parameters = sl.PositionalTrackingFusionParameters()
positional_tracking_fusion_parameters.enable_GNSS_fusion = True
gnss_calibration_parameters = sl.GNSSCalibrationParameters()
gnss_calibration_parameters.target_yaw_uncertainty = 1e-2
gnss_calibration_parameters.enable_translation_uncertainty_target = True
gnss_calibration_parameters.target_translation_uncertainty = 15e-2
gnss_calibration_parameters.enable_reinitialization = True
gnss_calibration_parameters.gnss_vio_reinit_threshold = 5
gnss_calibration_parameters.gnss_antenna_position = np.asarray([0.01,.01,0.01])
positional_tracking_fusion_parameters.gnss_calibration_parameters = gnss_calibration_parameters
fusion.enable_positionnal_tracking(positional_tracking_fusion_parameters)
uuid = sl.CameraIdentifier(camera_info.serial_number)
#Subscribe fusion to camera
print(f"Subscribing to CAMERA Seial : {uuid.serial_number} with {communication_parameters.comm_type}")
status = fusion.subscribe(uuid, communication_parameters, sl.Transform(0,0,0)) #Initial Ref to 0
if status != sl.FUSION_ERROR_CODE.SUCCESS:
print("Failed to subscribe to", uuid.serial_number, status)
exit(1)
print(f"GNSS Calibration Status : {fusion.get_geo_tracking_calibration()}" )```
Along with these we have accomplised fusion object, now the question is that How can I get FusedPointCloud data from fusion object in a way that it can be exported to a ply or object file.
Consider to check the script inline with this message that will help us understand the situation better.
Also, you can find the inline script here :
[modified_sample.py|attachment](upload://u9qQQ1xy3dDqdImsVEx3g6RCa3J.py) (14.9 KB)
TIA
It appears that the upload didn’t go through correctly.
Regarding your issue, you might want to use the retrieveSpatialMapAsync function from the fusion object. If this function doesn’t meet your needs, you can still use the retrieveMeasure function from the Camera object and then use the Camera2Geo function from the fusion object to convert it to the Geo world. This should work as expected.
Apologies for not uploading the file properly earlier.
Regarding your suggestion, I couldn’t find the retrieve_spatial_map_async function in the Fusion object in Python. Could you confirm if this function is available in the latest ZED SDK (4.2.5) in python api?
As for using the retrieveMeasure function from the Camera object and then applying Camera2Geo from the Fusion object, could you clarify the recommended workflow? Here’s my understanding:
point_cloud = sl.Mat() zed.retrieve_measure(point_cloud, sl.MEASURE.XYZRGBA)
then how it can be used to write my desired result.
It appears that this function is not currently available in our Python wrapper. I will report this issue, and it should be included in the next release. In the meantime, you can use the function from the Camera class, as it should meet your needs.
Regarding the workflow, it is quite straightforward. Once you obtain your point cloud referenced to your current camera position, you can call Camera2Geo for each point in it. Here are the steps to follow for each point in the point cloud:
Construct a transform and set it as the identity.
Set the translation of this transform with the position of the point.
Call Camera2Geo with this transform as input.
Using this method, you should be able to reference every point in your point cloud in ENU (using the pose_data attribute of GeoPose) or in LatLong (using the latlng_coordinates attribute of GeoPose), which can then be converted to ECEF using our converter.
Thanks for clearing the points,
I just want to know if is there any way in the Converter to directly save the transformed coordinates in a ply file format of we have to write it as a text file.
“According to your reply Camera2Geo function takes the pose object and returns the GeoPose object”. My question is, how can I create a pose object for each point?
A prompt answer will be really appreciated.
I think you forget to have a look at the previously attached program file, it would be great if you review this for me and point out where I am lacking.
