• DepthAI

  • cannot start device in python3.9 environment from 3d slicer

Hi,

I have a simple code that creates a pipeline that captures both L+R mono images and the disparity map, using OAK-D SR module.
This code works perfectly when running from python3.10 environment (Distro: Linux Mint 21.3 Virginia base: Ubuntu 22.04 jammy).
When I am running the code within 3D slicer scripted module environment (python 3.9), the same pipeline is stuck in the creation (dai.Device(pipeline)).

This is the debug dump I get when there is a successful run:

[2024-12-12 22:18:15.973] [depthai] [debug] Python bindings - version: 2.29.0.0 from build: 2024-11-24 09:54:26 +0000

[2024-12-12 22:18:15.973] [depthai] [debug] Library information - version: 2.29.0, commit: d6a37a5ba8ba2ee187fabc9208b813db5f75d4a7 from 2024-11-22 15:48:55 +0100, build: 2024-11-24 09:39:00 +0000, libusb enabled: true

[2024-12-12 22:18:15.978] [depthai] [debug] Initialize - finished

[2024-12-12 22:18:16.094] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.28.tar.xz' open: 3ms, archive read: 116ms

[2024-12-12 22:18:16.643] [depthai] [debug] Resources - Archive 'depthai-device-fwp-4d360b5c56225f23e9a3d3a3999ce46c90cfdeaf.tar.xz' open: 2ms, archive read: 666ms

/home/ajax/dev/cflow/cflow_slicer_ext/cflow_pc/point_cloud/test_camera_depth_post_processing.py:43: DeprecationWarning: HIGH_DENSITY is deprecated, will be removed in future releases and replaced with DEFAULT.

depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)

[2024-12-12 22:18:20.538] [host] [debug] Device - OpenVINO version: universal

[14442C101130B1D200] [3.2] [1734034700.538] [host] [debug] Device - BoardConfig: {"camera":[],"emmc":null,"gpio":[],"imu":null,"logDevicePrints":null,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoDelay":true},"nonExclusiveMode":false,"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"manufacturer":"","maxSpeed":4,"pid":63035,"productName":"","vid":999},"usb3PhyInternalClock":null,"uvc":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null}

libnop:

0000: b9 12 b9 07 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 bd 00 bd 00 b9 02 00 01 ba 00 be be bb 00 bb

0020: 00 be be be be be be be 00 bb 00 be be

[2024-12-12 22:18:21.508] [depthai] [debug] Searching for booted device: DeviceInfo(name=3.2, mxid=14442C101130B1D200, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only

[14442C101130B1D200] [3.2] [1.270] [system] [info] Memory Usage - DDR: 0.12 / 333.26 MiB, CMX: 2.04 / 2.50 MiB, LeonOS Heap: 7.50 / 81.70 MiB, LeonRT Heap: 2.89 / 39.87 MiB / NOC ddr: 25 MB/s

[14442C101130B1D200] [3.2] [1.270] [system] [info] Temperatures - Average: 31.22C, CSS: 32.73C, MSS 30.56C, UPA: 30.56C, DSS: 31.04C

…

When Running within the 3D slicer, I get the following debug dump:

[2024-12-12 22:29:03.376] [depthai] [debug] Python bindings - version: 2.29.0.0 from build: 2024-11-24 09:50:04 +0000

[2024-12-12 22:29:03.376] [depthai] [debug] Library information - version: 2.29.0, commit: d6a37a5ba8ba2ee187fabc9208b813db5f75d4a7 from 2024-11-22 15:48:55 +0100, build: 2024-11-24 09:39:00 +0000, libusb enabled: true

[2024-12-12 22:29:03.380] [depthai] [debug] Initialize - finished

[2024-12-12 22:29:17.994] [host] [debug] Device - OpenVINO version: universal

[14442C101130B1D200] [3.2] [1734035357.995] [host] [debug] Device - BoardConfig: {"camera":[],"emmc":null,"gpio":[],"imu":null,"logDevicePrints":null,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoDelay":true},"nonExclusiveMode":false,"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"manufacturer":"","maxSpeed":4,"pid":63035,"productName":"","vid":999},"usb3PhyInternalClock":null,"uvc":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null}

libnop:

0000: b9 12 b9 07 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 bd 00 bd 00 b9 02 00 01 ba 00 be be bb 00 bb

0020: 00 be be be be be be be 00 bb 00 be be

The sample code is below (stuck on the last line):

        # Closer-in minimum depth, disparity range is doubled (from 95 to 190):
        extended_disparity = False
        # Better accuracy for longer distance, fractional disparity 32-levels:
        subpixel = False
        # Better handling for occlusions:
        lr_check = False

        # Create pipeline
        pipeline = dai.Pipeline()

        # Define sources and outputs
        monoLeft = pipeline.create(dai.node.MonoCamera)
        monoRight = pipeline.create(dai.node.MonoCamera)
        depth = pipeline.create(dai.node.StereoDepth)
        xout = pipeline.create(dai.node.XLinkOut)
        sync = pipeline.create(dai.node.Sync)
        xout.setStreamName("disparity")

        xout_01 = pipeline.create(dai.node.XLinkOut)
        xout_01.input.setBlocking(False)
        xout_01.setStreamName("mono")

        # Properties
        monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
        monoLeft.setFps(10)
        monoLeft.setCamera("left")

        monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
        monoRight.setCamera("right")
        monoLeft.setFps(10)

        # Create a node that will produce the depth map (using disparity output as it's easier to visualize depth this way)
        depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
        depth.setLeftRightCheck(lr_check)
        depth.setExtendedDisparity(extended_disparity)
        depth.setSubpixel(subpixel)

        config = depth.initialConfig.get()
        config.postProcessing.median = dai.MedianFilter.KERNEL_7x7
        config.postProcessing.speckleFilter.enable = True
        config.postProcessing.speckleFilter.speckleRange = 10
        config.postProcessing.temporalFilter.enable = False
        config.postProcessing.spatialFilter.enable = True
        config.postProcessing.spatialFilter.holeFillingRadius = 20
        config.postProcessing.spatialFilter.numIterations = 1
        config.postProcessing.thresholdFilter.minRange = 0
        config.postProcessing.thresholdFilter.maxRange = 10000
        config.postProcessing.decimationFilter.decimationFactor = 1
        depth.initialConfig.set(config)

        # Linking
        monoLeft.out.link(depth.left)
        monoLeft.out.link(sync.inputs['rgb'])
        monoRight.out.link(depth.right)
        depth.disparity.link(xout.input)
        monoLeft.out.link(xout_01.input)

        # Connect to device and start pipeline
        with dai.Device(pipeline) as device:

Thank you for your help.

Hi,

have you tried running the code with python 3.9 outside of the 3D Slicer environment? It seems like there might be a dependency issue or conflict between 3D Slicer's Python environment and DepthAI's libraries.

Hi, thanks for your reply!

It works using a standalone python 3.9 environment outside of 3D Slicer

here is the list of packages comparison between the different environments:

package                    clean version    slicer version
-------------------------  ---------------  ----------------
certifi                    2024.8.30        2023.7.22
charset-normalizer         3.4.0            3.3.0
depthai                    2.29.0.0         2.29.0.0
idna                       3.10             3.4
numpy                      2.0.2            1.26.1
packaging                  24.2             23.2
pip                        24.2             23.3
pyparsing                  3.2.0            3.1.1
python-dateutil            2.9.0.post0      2.8.2
requests                   2.32.3           2.31.0
retrying                   1.3.4            1.3.4
scipy                      1.13.1           1.11.3
setuptools                 75.1.0           68.2.2
six                        1.17.0           1.16.0
typing_extensions          4.12.2           4.8.0
urllib3                    2.2.3            2.0.6
wheel                      0.44.0           0.41.2
ConfigArgParse             1.7              N/A
Flask                      3.0.3            N/A
Jinja2                     3.1.4            N/A
MarkupSafe                 3.0.2            N/A
PyYAML                     6.0.2            N/A
Pygments                   2.18.0           N/A
Werkzeug                   3.0.6            N/A
addict                     2.4.0            N/A
asttokens                  3.0.0            N/A
attrs                      24.2.0           N/A
blinker                    1.9.0            N/A
click                      8.1.7            N/A
comm                       0.2.2            N/A
contourpy                  1.3.0            N/A
cycler                     0.12.1           N/A
dash                       2.18.2           N/A
dash-core-components       2.0.0            N/A
dash-html-components       2.0.0            N/A
dash-table                 5.0.0            N/A
decorator                  5.1.1            N/A
exceptiongroup             1.2.2            N/A
executing                  2.1.0            N/A
fastjsonschema             2.21.1           N/A
fonttools                  4.55.3           N/A
importlib_metadata         8.5.0            N/A
importlib_resources        6.4.5            N/A
ipython                    8.18.1           N/A
ipywidgets                 8.1.5            N/A
itsdangerous               2.2.0            N/A
jedi                       0.19.2           N/A
joblib                     1.4.2            N/A
jsonschema                 4.23.0           N/A
jsonschema-specifications  2024.10.1        N/A
jupyter_core               5.7.2            N/A
jupyterlab_widgets         3.0.13           N/A
kiwisolver                 1.4.7            N/A
matplotlib                 3.9.4            N/A
matplotlib-inline          0.1.7            N/A
nbformat                   5.10.4           N/A
nest-asyncio               1.6.0            N/A
open3d                     0.18.0           N/A
opencv-python              4.10.0.84        N/A
pandas                     2.2.3            N/A
parso                      0.8.4            N/A
pexpect                    4.9.0            N/A
pillow                     11.0.0           N/A
platformdirs               4.3.6            N/A
plotly                     5.24.1           N/A
prompt_toolkit             3.0.48           N/A
ptyprocess                 0.7.0            N/A
pure_eval                  0.2.3            N/A
pyquaternion               0.9.9            N/A
pytz                       2024.2           N/A
referencing                0.35.1           N/A
rpds-py                    0.22.3           N/A
scikit-learn               1.6.0            N/A
stack-data                 0.6.3            N/A
tenacity                   9.0.0            N/A
threadpoolctl              3.5.0            N/A
tqdm                       4.67.1           N/A
traitlets                  5.14.3           N/A
tzdata                     2024.2           N/A
wcwidth                    0.2.13           N/A
widgetsnbextension         4.0.13           N/A
zipp                       3.21.0           N/A
CouchDB                    N/A              1.2
Deprecated                 N/A              1.2.14
GitPython                  N/A              3.1.37
Pillow                     N/A              10.1.0
PyGithub                   N/A              2.1.1
PyJWT                      N/A              2.8.0
PyNaCl                     N/A              1.5.0
SimpleITK                  N/A              2.2.1rc2.dev4
cffi                       N/A              1.16.0
chardet                    N/A              5.2.0
cryptography               N/A              41.0.4
debugpy                    N/A              1.8.9
dicomweb-client            N/A              0.59.1
gitdb                      N/A              4.0.10
pycparser                  N/A              2.21
pydicom                    N/A              2.4.3
smmap                      N/A              5.0.1
vtk                        N/A              9.2.20230607
wrapt                      N/A              1.15.0

Hi,

thanks for the update! Since the standalone Python environment works fine, I suggest trying to sync the version of numpy in the 3D Slicer environment to match the standalone setup.

If that doesn’t resolve the issue, consider adding other missing packages from the standalone environment to the Slicer setup. These differences in package versions and dependencies might be causing the conflict.

Let me know how it goes, and we can explore further if needed!

Thank you for your advice.

I tried your advice, and it didn't seems to help (assuming it was done correctly).
I also tried running "external" python3.9 with the environment from Slicer, and it worked.
I also tried running with the bin/PythonSlicer executable (that uses the slicer environment), and it worked.

I tried running the code in a different process (using multiprocessing) but it looks like slicer does not work with multiprocessing.

Thanks

Looks like the problem is with Slicer itself. Since it works with python3.9 and PythonSlicer, the issue is likely how Slicer handles the environment in the scripted module. Maybe try using subprocess to run the DepthAI code outside and see if that helps.