Zed 4.0.6 - Broken python library

robots · November 9, 2023, 4:47pm

With https://stereolabs.sfo2.cdn.digitaloceanspaces.com/zedsdk/4.0/ZED_SDK_Ubuntu20_cuda11.8_v4.0.6.zstd.run pyzed fails to import. This previously worked on 4.0.6, but no longer does.

> python
Python 3.10.13 
>>> import pyzed.sl as sl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /...python3.10/site-packages/pyzed/sl.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2sl6Camera19getRegionOfInterestERNS_3MatENS_10ResolutionE
>>>

This happens even after re-running

> python /usr/local/zed/get_python_api.py

robots · November 9, 2023, 4:50pm

This is fixed in 4.0.7, but that neural depth model performs worse.

alassagne · November 10, 2023, 9:52am

Hi,
This is because get_python_api.py returns the latest python module, which is now 4.0.8. The compatibility was not there between 4.0.7 and 4.0.6 because the API changed within the same minor version. We will avoid that in future patches, it was because of the early access. Sorry again.
If you still want to use 4.0.6 and the python module, you can build it yourself from the repo GitHub - stereolabs/zed-python-api: Python API for the ZED SDK. It’s actually quite easy.

It’s very suprising that you found difference between NEURAL version 4.0.6 and 4.0.7 since it did not change. Is it still the case with 4.0.8 ? Can you send a SVO so that we can investigate ?

robots · November 10, 2023, 1:51pm

I can still get an SVO if needed, but the changelog did say the network changed. 4.0.7 Confidence Ranges Changed - #4 by Myzhar

The issue now is in scenes that would’ve previously had points almost everywhere, we get large patches of completely empty space when using low confidence_thresholds

alassagne · November 10, 2023, 4:17pm

Indeed, the confidence ranges was retuned - however if you modify the confidence, you can get the same quality as you had before. It’s just that low confidences are now really low, whereas they were still quite high before.

robots · November 10, 2023, 4:26pm

The issue is it’s really hard to know what the mapping from old confidence values to new confidence values is. The changelog and docs provided no guidance on this issue. If you can provide more guidance now it’d be very helpful

alassagne · November 10, 2023, 4:27pm

Of course ! Can you tell me what your old confidence value was ?

robots · November 10, 2023, 4:52pm

It was 0. On 4.0.7, 0 returns no values.

alassagne · November 10, 2023, 6:08pm

Can you try with the value 1 ?

robots · November 13, 2023, 10:32pm

I have tried many confidence values. None consistently perform in a similar manner. Do you have the python whl for 4.0.6? I would like to manually install it. The depth model 4.0.7+ is not viable.

robots · November 13, 2023, 11:08pm

I’m also ok with building it from source https://github.com/stereolabs/zed-python-api/tree/master/src if you can direct me to the correct commit.

alassagne · November 14, 2023, 10:21am

Hi,
Really sorry about this. I can reproduce your behavior a little:

This two images are taken from 4.0.5 and 4.0.8. The one with more black is 4.0.6 at 0%, the other is 4.0.8 at 1%. I guess the confidence is still too high for you ? If you can send a SVO file, it can really help us understand the issue better. I thought that you actually had less points than before, but in my tests I get more.

What you can try is to remove the NEURAL model from /usr/local/zed/resources, to force a reoptimizations. You’ll be sure you have a fresh installation.

In the meantime, here is the python wrapper for 4.0.6.
pyzed.zip (929.6 KB)

robots · November 14, 2023, 2:32pm

Do you have the 4.0.6 wheel file? Alternatively, how do I convert the zip you sent to a wheel.

I was able to build the 4.0.8 egg from source, but didn’t see how to produce a wheel file from that. The building also failed for me for sdk versions <4.0.8 on any of the 5 most recent commits of the python-sdk repo.

I observed a similar change in the number of points between 4.0.6 and 4.0.7. In almost every case there were more points in 4.0.7+ with a confidence threshold of 1% compared to <4.0.7 with a confidence of 0. However, in <5% of cases the 4.0.7+ model with confidences ranging between (1 and 5) would catastrophically return no points for about 1/4 of the entire scene. These scenes were all within the min/max depth runtime parameters and similar to the scenes we had successfully operated in with <4.0.7.

I’d like to send you an SVO, but don’t have any available and struggle to reproduce this issue whenever SVO recording is an option. Is there a way to use the SDK normally/live and also record SVOs?

Also if you have an email/slack/discord we could use to communicate I can share more details than I am comfortable doing on this forum.

alassagne · November 14, 2023, 4:08pm

Hi again,

You can just replace the files I sent you in the repo from github (latest vesion). Then do what is in the readme.
Alternatively, send me you platform and python version and I will build you it. I guess, Python3.10, Ubuntu20 ? If that’s the case, here it is:
pyzed-4.0-cp310-cp310-linux_x86_64.whl (622.9 KB)

robots · November 15, 2023, 4:18pm

Thank you for the wheel. I’ll work on getting SVO files, but it will take a while

robots · January 1, 2024, 10:11pm

Hi, I tried again to build the wheels for python 3.11, but still ran into tons of issues. Could you compile a ubuntu 20, python 3.11 wheel for me? It would be incredibly helpful. Thank you

Also is there a docker container you’d recommend for compiling things?

robots · January 4, 2024, 6:47am

I spent some more time on this. I was able to build the wheel for 3.11 ubuntu 20 with 4.0.8, but not with the files you sent for 4.0.6. To get it to work for 4.0.8 I

Built off stereolabs/zed:4.0-devel-cuda11.8-ubuntu20.04 docker image with a python3.11-dev virtual environment
Installed the 4.0.6 SDK
Swapped in the files you sent
Installed all required python packages
Ran python setup.py bdist_wheel
When running setup.py I get a bunch of errors like

------------------------------------------------------------

pyzed/sl.pyx:10670:89: 'c_REFERENCE_FRAME' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    # \param reference_frame: Defines the reference from which you want the pose to be expressed. Default : \ref REFERENCE_FRAME "REFERENCE_FRAME.WORLD".
    # \param uuid: If set to a sender serial number (different from 0), this will retrieve position projected on the requested camera if \ref position_type is equal to \ref POSITION_TYPE "POSITION_TYPE.FUSION" or raw sender position if \ref position_type is equal to \ref POSITION_TYPE "POSITION_TYPE.RAW".
    # \param position_type: Select if the position should the fused position re-projected in the camera with uuid or if the position should be the raw position (without fusion) of camera with uui.
    # \return POSITIONAL_TRACKING_STATE is the current state of the tracking process.
    def get_position(self, camera_pose : Pose, reference_frame : REFERENCE_FRAME = REFERENCE_FRAME.WORLD, uuid: CameraIdentifier = CameraIdentifier(), position_type : POSITION_TYPE = POSITION_TYPE.FUSION):
        return POSITIONAL_TRACKING_STATE(<int>self.fusion.getPosition(camera_pose.pose, <c_REFERENCE_FRAME>(<int>reference_frame.value), uuid.cameraIdentifier, <c_POSITION_TYPE>(<int>position_type.value)))
                                                                                                                                                                 ^
------------------------------------------------------------

pyzed/sl.pyx:10670:161: 'c_POSITION_TYPE' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    # \return sl.Transform is the calibration found between VIO and GNSS during calibration process.
    ## 
    def get_geo_tracking_calibration(self) -> Transform:
        # cdef c_Transform tmp
        tf_out = Transform()
        tmp = <c_Transform>(self.fusion.getGeoTrackingCalibration())
               ^
------------------------------------------------------------

pyzed/sl.pyx:10731:15: 'c_Transform' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    # \return sl.Transform is the calibration found between VIO and GNSS during calibration process.
    ## 
    def get_geo_tracking_calibration(self) -> Transform:
        # cdef c_Transform tmp
        tf_out = Transform()
        tmp = <c_Transform>(self.fusion.getGeoTrackingCalibration())
              ^
------------------------------------------------------------

pyzed/sl.pyx:10731:14: Casting temporary Python object to non-numeric non-Python type
Traceback (most recent call last):
  File "/root/zed-python-api/src/setup.py", line 194, in <module>
    extList = cythonize(extension, compiler_directives=cython_directives)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/311-env/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
    cythonize_one(*args)
  File "/root/311-env/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: pyzed/sl.pyx

Any suggestions on how to build it properly?

fjiang · January 4, 2024, 8:58am

Hi @robots

The files @alassagne send you should not be compatible with Cython 3.0 (since the ZED SDK 4.0.6 is not).

You can either:

downgrade Cython to 0.29 and use this setup.py (6.6 KB)

OR

try to update sl.pyx to be compatible with Cython 3.0 (it might be simple as replacing

from sl_c cimport ( String, to_str, Camera as c_Camera, ERROR_CODE as c_ERROR_CODE, toString

with

from .sl_c cimport ( String, to_str, Camera as c_Camera, ERROR_CODE as c_ERROR_CODE, toString

if I remember correctly)