If using REFERENCE_FRAME::world to grab the point cloud with a depth_maximum_distance set to for example 100, when the camera travels 50 meters the max distance it will see along it’s travelling vector is 50 meters. I’m having an issue where the object the camera is mounted to is travelling 100s if meters so I get no points in my point cloud once the object travels far!
I don’t think this is intended behaviour, to me a depth maximum distance implies distance from the camera, NOT from the origin regardless of where the camera travels
Could I also request that the team adds a “REFERENCE_FRAME::GRAVITY_ALIGNED” mode when grabbing the point cloud? As is when I use REFERENCE_FRAME::WORLD, I need to retranslate all my points back to the origin and twist it to look a constant direction as my algorithms are tuned best around the origin. Though this currently doesn’t work due to the issue above.
I can use REFERENCE_FRAME::Camera, then rotate my points to be gravity aligned myself. However this doesn’t work when the tracking is disabled or visual tracking is unable to track itself for whatever reason (This can be an issue for my application due to dust and other factors) and AFAIK I cannot directly grab the gravity vector from the sensor. I can see that the initial angle of the sensor is defaulted to be gravity aligned, but once it rotates that initial world rotation is no longer gravity aligned and it won’t update when the tracking isn’t working.
Hi @Ozone,
thank you for the suggestion; I added it to the list of feature requests for the ZED SDK team.
You can retrieve a gravity-aligned orientation directly from the camera IMU, independently of the Positional Tracking module, so it keeps working even when tracking is disabled or visual tracking is lost because of dust.
After calling getSensorsData, the SensorsData::IMUData::pose field provides the orientation estimated by the internal accelerometer and gyroscope fusion; roll and pitch are referred to the gravity vector, so you can use this rotation to align the point cloud retrieved with REFERENCE_FRAME::CAMERA.
If you prefer to estimate gravity yourself, the linear_acceleration field contains the calibrated accelerometer data, gravity included; note that this raw value is not available when using SVO or streaming input.
You can find more details in the Sensors API documentation; remember to use TIME_REFERENCE::IMAGE to get the IMU data synchronized with the point cloud.