Global Localization (GNSS Fusion) fails with DEPTH_MODE.NONE even though GEN_3

Software Python
, ,
1

First of all, thank you very much for the great SDK and for maintaining this community — it’s been a pleasure to build with.

Note: part of the experimentation/verification below was carried out with the help of Claude Code, and this post itself was translated/drafted from my original question with AI assistance. Sorry if any part reads a bit awkwardly — please feel free to ask me to clarify anything.

Summary

I’m trying to minimize GPU/CPU load by disabling depth computation (DEPTH_MODE.NONE) while still using POSITIONAL_TRACKING_MODE.GEN_3, which works perfectly on its own (see “Test 1” below). However, when I add the Fusion module for GNSS-based Global Localization on top of the same camera, Camera::startPublishing() fails outright as soon as depth is disabled, with the internal message:

[ZED][ERROR] [Grab]  This feature requires an active depth map, but depth computation is disabled (MODE_NONE). Enable depth computation in the initialization parameters.

I could not find this requirement documented anywhere (Global Localization Overview, Troubleshooting page, Fusion Overview, or the Camera.hpp/Fusion.hpp header comments only show sample code that happens to use DEPTH_MODE::ULTRA, without stating that NONE is unsupported).

My questions

  1. Is it fundamentally impossible to run Global Localization / GNSS Fusion (Camera::startPublishing() + Fusion::subscribe() + Fusion::ingestGNSSdata()) with DEPTH_MODE::NONE? Or is there a workaround/parameter I’m missing to make it work without an active depth map?
  2. If it is indeed impossible, why? What internal data does startPublishing()/the Fusion pipeline actually need from the depth module, given that GEN_3 positional tracking itself performs its own feature-based triangulation independently of the DEPTH_MODE-configured dense depth map (confirmed working with DEPTH_MODE::NONE — see Test 1)?
  3. If depth data is indeed being consumed internally by the Fusion/Global Localization pipeline, does the choice of depth mode (PERFORMANCE / QUALITY / ULTRA / NEURAL_LIGHT / NEURAL / NEURAL_PLUS) have a measurable effect on Global Localization accuracy? i.e., is it worth spending the extra compute on ULTRA/NEURAL_PLUS for better geo-pose accuracy, or is any active depth mode functionally equivalent from Fusion’s point of view (just needed to be “on”)?

Test environment

  • ZED SDK: 5.3.0
  • Camera: ZED 2i (serial 33004600, FW 1523), connected via USB3 (confirmed 5000 Mbps enumeration)
  • Host: NVIDIA Jetson (Linux 5.15.148-tegra, aarch64), Python 3.10, pyzed bindings
  • Positional tracking mode: sl.POSITIONAL_TRACKING_MODE.GEN_3
  • GNSS receiver: u-blox GNSS receiver over USB (/dev/ttyACM0), NMEA parsed directly with pyserial + pynmea2 (no gpsd running on this box), confirmed valid 3D fix (12 satellites, fix quality 1, HDOP 0.87) at real-world coordinates in Tokyo, Japan (lat ≈ 35.6955°N, lon ≈ 139.5958°E)

What I tried and observed

Test 1 — GEN_3 Positional Tracking alone, DEPTH_MODE.NONE (no Fusion)

init_params = sl.InitParameters()
init_params.depth_mode = sl.DEPTH_MODE.NONE
zed.open(init_params)

pt_params = sl.PositionalTrackingParameters()
pt_params.mode = sl.POSITIONAL_TRACKING_MODE.GEN_3
zed.enable_positional_tracking(pt_params)   # -> SUCCESS

Result: works as expected. enable_positional_tracking() succeeds, get_position() returns POSITIONAL_TRACKING_STATE.OK on every frame over a 15s run, and the reported pose visibly tracks real hand movement of the camera (up to ~0.46 m translation). retrieve_measure(sl.MEASURE.DEPTH) correctly returns INVALID FUNCTION PARAMETERS, confirming depth truly isn’t being computed.

Test 2 — Add Fusion / GNSS Global Localization on top, still DEPTH_MODE.NONE

init_params.depth_mode = sl.DEPTH_MODE.NONE
zed.open(init_params)
zed.enable_positional_tracking(pt_params)   # mode=GEN_3, -> SUCCESS

fusion = sl.Fusion()
fusion.init(sl.InitFusionParameters())      # -> SUCCESS

communication_parameters = sl.CommunicationParameters()  # default INTRA_PROCESS
pub_status = zed.start_publishing(communication_parameters)
# -> ERROR_CODE.INVALID_FUNCTION_CALL

uuid = sl.CameraIdentifier(zed.get_camera_information().serial_number)
sub_status = fusion.subscribe(uuid, communication_parameters, sl.Transform())
# -> FUSION_ERROR_CODE.MEMORY_ALREADY_USED   (looks like a cascading failure from the failed publish)

positional_tracking_fusion_parameters = sl.PositionalTrackingFusionParameters()
positional_tracking_fusion_parameters.enable_GNSS_fusion = True
fusion.enable_positionnal_tracking(positional_tracking_fusion_parameters)
# -> SUCCESS (misleadingly, since publishing/subscribing already failed)

...
fusion.ingest_gnss_data(gnss_data)
# -> FUSION_ERROR_CODE.MODULE_NOT_ENABLED, on every single frame for 30s

SDK log at the moment start_publishing() is called:

[ZED][ERROR] [Grab]  This feature requires an active depth map, but depth computation is disabled (MODE_NONE). Enable depth computation in the initialization parameters.

Reproduced identically on 2 separate runs.

Test 3 — Control test: same code, DEPTH_MODE.PERFORMANCE instead of NONE

Only the depth mode was changed, everything else identical:

init_params.depth_mode = sl.DEPTH_MODE.PERFORMANCE

Result: start_publishing() → SUCCESS, fusion.subscribe() → SUCCESS, fusion.ingest_gnss_data() → SUCCESS for 16/~30 GNSS samples over the same 30s window (the remaining ones were rejected with GNSS_DATA_COVARIANCE_MUST_VARY, which is a separate, expected issue on my end since my NMEA reader currently reports a fixed covariance value rather than a varying one — not related to depth). The real GNSS coordinates (Tokyo area) were successfully ingested.

What I’ve already checked

  • Global Localization Overview — sample code always sets a real depth_mode (e.g. ULTRA), but never states NONE is disallowed.
  • Global Localization Troubleshooting — covers MODULE_NOT_ENABLED, INVALID_COVARIANCE, INVALID_TIMESTAMP, GNSS_DATA_NEED_FIX, GNSS_DATA_COVARIANCE_MUST_VARY, but nothing about depth mode.
  • Fusion Overview — describes startPublishing() conceptually but doesn’t mention any depth precondition.
  • The Doxygen comments for Camera::startPublishing() in Camera.hpp don’t mention depth either.
  • Searched the forums for the exact error string and for MODULE_NOT_ENABLED + depth — found one thread with the same NOT_ENABLE ingest error, but that case was caused by the user simply forgetting to call start_publishing(), unrelated to depth mode.

Any clarification from the SDK team on the three questions above would be greatly appreciated. Thanks!