Python API: grab() and reset_positional_tracking() hold the GIL, blocking other Python threads (IMU thread stalls ~2.8 s)

Software Python
1

Hi,

In pyzed, other Python threads in the same process stop running while grab() or reset_positional_tracking() is running. It looks like the wrapper doesn’t release the GIL during these calls.

Setup: ZED X Mini, SVGA @ 60 fps, GEN_3 tracking with area memory, ZED SDK [version], Python 3.10, Jetson AGX Orin, JetPack [version].

What I see:

  • An IMU thread polling get_sensors_data(CURRENT) at 200 Hz only publishes at the camera’s 60 Hz.
  • During reset_positional_tracking() (2.8 s), it publishes nothing.
  • A plain heartbeat thread that only calls time.sleep(0.005) and never touches the camera also stalls for 2768 ms during the reset.

Repro:

python

import threading, time
import pyzed.sl as sl

zed = sl.Camera()
init = sl.InitParameters()
init.camera_resolution = sl.RESOLUTION.SVGA
init.camera_fps = 60
assert zed.open(init) == sl.ERROR_CODE.SUCCESS
tp = sl.PositionalTrackingParameters()
tp.enable_area_memory = True
zed.enable_positional_tracking(tp)

def heartbeat():
    last = time.monotonic()
    while True:
        time.sleep(0.005)
        now = time.monotonic()
        if now - last > 0.05:
            print(f"heartbeat stalled {1e3 * (now - last):.0f} ms")
        last = now
threading.Thread(target=heartbeat, daemon=True).start()

rt = sl.RuntimeParameters()
for i in range(300):
    zed.grab(rt)
    if i == 150:
        zed.reset_positional_tracking(sl.Transform())
zed.close()

Question: Does pyzed hold the GIL during these calls on purpose? Would it be safe to release it (with nogil:) so other threads, such as an IMU reader, keep running?

Thanks!

2

Hi @bajaj10
Welcome to the StereoLabs community-

thank you for the detailed report and the reproducer.

This is only partially correct. I checked the source code of the ZED Python API:

  • Camera.grab() releases the GIL (with nogil:) since ZED SDK v5.0.
  • Camera.get_sensors_data() releases the GIL since ZED SDK v5.1.
  • Camera.reset_positional_tracking() does not release the GIL in any version, including the latest v5.4.0.

During reset_positional_tracking() (2.8 s), it publishes nothing.
A plain heartbeat thread that only calls time.sleep(0.005) and never touches the camera also stalls for 2768 ms during the reset.

This is caused by the missing GIL release in reset_positional_tracking(). I’m going to report it to the StereoLabs development team so it can be fixed in a future release.

In the meantime, you can patch the wrapper locally since it’s open source. Clone the repository for your ZED SDK version and apply these two changes.

In src/pyzed/sl_c.pxd:

ERROR_CODE resetPositionalTracking(const Transform &path) nogil

In src/pyzed/sl.pyx, class Camera:

def reset_positional_tracking(self, Transform path) -> ERROR_CODE:
    cdef c_ERROR_CODE err
    with nogil:
        err = self.camera.resetPositionalTracking(path.transform[0])
    return _error_code_cache.get(<int>err, ERROR_CODE.FAILURE)

Then rebuild and install it by running python -m pip install . from the repository root, as described in the README.

With an up-to-date wrapper this should not be caused by the GIL, because both grab() and get_sensors_data() release it.

Your post still has [version] placeholders. Please share:

  • the exact ZED SDK version,
  • the pyzed version (python3 -m pip show pyzed),
  • the JetPack version.

A pyzed package that doesn’t match the installed ZED SDK version could explain this behavior.