OAK-FFC ToF 33D

jakaskerl

Hi @vuk16
Not sure where you got that cam_test.py script. I can confirm that on latest main branch the TOF node is created with pipeline.create(dai.Node.Tof).

git checkout main
git pull --recurse-submodules

Thanks,
Jaka

vuk16

Hi @jakaskerl,

I found it in official Luxonis documentation (https://docs.luxonis.com/projects/hardware/en/latest/pages/DM1090/). When I am running this one Could you tell me if there is any script that can be used to get ToF data? When I am running tof_depth.py I am getting weird image and the error message:
[14442C109187CAD600] [1.3] [20.962] [ToF(1)] [error] Orientation not supported! Only normal and 180 flip(not mirrored) can be used.

When you carefully look at the image, you can see that the image contains multiple same images, so I guess it is somehow a problem of setting resolution?

Best regards,

Vuk

jakaskerl

Hi @vuk16
The script should work without issues. Have you changed it in any way, can you send the script?

Thanks,
Jaka

vuk16

jakaskerl ,

I have not changed anything in the script.

#!/usr/bin/env python3

import cv2import depthai as daiimport numpy as np
pipeline = dai.Pipeline()
cam_a = pipeline.create(dai.node.Camera)# We assume the ToF camera sensor is on port CAM_Acam_a.setBoardSocket(dai.CameraBoardSocket.CAM_B)
tof = pipeline.create(dai.node.ToF)
# Configure the ToF nodetofConfig = tof.initialConfig.get()print(tofConfig)# tofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MINtofConfig.depthParams.freqModUsed = dai.RawToFConfig.DepthParams.TypeFMod.MAXtofConfig.depthParams.avgPhaseShuffle = FalsetofConfig.depthParams.minimumAmplitude = 3.0tof.initialConfig.set(tofConfig)# Link the ToF sensor to the ToF nodecam_a.raw.link(tof.input)
xout = pipeline.create(dai.node.XLinkOut)xout.setStreamName("depth")tof.depth.link(xout.input)
# Connect to device and start pipelinewith dai.Device(pipeline) as device: print('Connected cameras:', device.getConnectedCameraFeatures()) q = device.getOutputQueue(name="depth")
while True: imgFrame = q.get() # blocking call, will wait until a new data has arrived depth_map = imgFrame.getFrame()

Colorize the depth frame to jet colormap

depth_downscaled = depth_map[::4]

non_zero_depth = depth_downscaled[depth_downscaled != 0] # Remove invalid depth values

if len(non_zero_depth) == 0:

min_depth, max_depth = 0, 0

else: min_depth = np.percentile(non_zero_depth, 3)

max_depth = np.percentile(non_zero_depth, 97)

depth_colorized = np.interp(depth_map, (min_depth, max_depth), (0, 255)).astype(np.uint8)

depth_colorized = cv2.applyColorMap(depth_colorized, cv2.COLORMAP_JET)

cv2.imshow("Colorized depth", depth_colorized)
if cv2.waitKey(1) == ord('q'): break

Thanks,

Vuk

jakaskerl

vuk16 cam_a.setBoardSocket(dai.CameraBoardSocket.CAM_B)

? looks like a change 🙂. The camera socket should be CAM_A as described in the comment above.

Thanks,
Jaka

vuk16

Hi jakaskerl,

If I connect camera to the CAMR port on the board (which should be equivalent to CAM_A), and if I use CAM_A in the line of the code you mention, I get device not found error. If I am using CAMR port and CAM_C in the line of the code, then I do not get device not found error, but I get weird data like described above (CAML port and CAM__B in the code). The only way camera provides me with the correct depth map is if I connect it to the CAMC port on the board and use CAMA in the code. In that case I get a depth map of a good quality, but the only issue is that the depth map is blinking. Have you ever experienced this blinking issue?
Note: In the code it is stated "We assume the ToF camera sensor is on port CAMA". In my understanding word assume does not mean that it has to be connected to the CAMA, but that if it is connected to the CAM_A and we do not change the code, then the camera should work (which is here not the case, because camera works only when connected to the CAMC port)

Best regards,

Vuk

jakaskerl

Hi @vuk16
You're right. I was using SR POE and assumed you were as well. I'm confused as to what board you are using. You said FFC-3P, but what is this CAMR port?

3P should have LEFT RIGHT and CENTER(color?), which is CAM_B, CAM_C, CAM_A respectively.

I have not experienced any blinking issues. Could you record a video of this happening so we can forward this to FW team?

Thanks,
Jaka

vuk16

Hi jakaskerl,

Thanks for the fast reply. I am using OAK-FFC 3P board and it has three ports named CAM_R, CAM_L and CAM_C. "3P should have LEFT RIGHT and CENTER(color?), which is CAMB, CAM_C, CAM_A respectively." This is true, but in API documentation CAM_R is CAM_A. However, now when I am able to get an image, I am sending you the video of flickering.

screencast-from-03-05-2024-100946-am.webm

5MB

Additionally, I recorded the video with raw data, and it is also flickering (For some reason it does not allow me to upload it here). All artificial ambient light was off during the experiment

Best regards,

Vuk

jakaskerl

Hi @vuk16

vuk16 This is true, but in API documentation CAM_R is CAM_A.

Can you link this please so we can fix it?

Regarding the flickering. The sensor flickers when it cannot asses the depth of the image since that's how it recalibrates. I experienced similar thing, but the flickering dissapeared when pointing at a surface (where max depth visible was 5m; if pointed at window, I would get flickering issues).
Can you confirm that is the case for you as well?

Thanks,
Jaka

vuk16

Hi jakaskerl,

I will take a look later, and I will link it when I find it again.
Regarding the flickering, I tried pointing to a surface that was smaller than 5 meters (I tried different surface distances), but flickering is always there. There was no window or glass in the FOV. When I try to upload the video I get the request payload was too large although the video is only two seconds long.

Best regards,

Vuk

jakaskerl

Hi @vuk16
Thanks.

vuk16 When I try to upload the video I get the request payload was too large although the video is only two seconds long.

Try online compression tools.

Thanks,
Jaka

vuk16

Hi jakaskerl,

I still cannot upload the video even after compressing. Here is couple of consecutive frames, where you can see the flickering effect.

Best regards,
Vuk

jakaskerl

Hi @vuk16
Tof is still WIP. The flicker issue will get fixed in future releases. Sorry for the inconvenience.

Thanks,
Jaka