Pipeline sometimes not being launched as expected

erik

Yes one camera will be blocked and won't recover. The tricky part is the issue not appear for all the pipelines with IMX477. But as comparison, with IMX214, NO issue for all the pipelines.

In the latest C++ MRE, I have 3 different nodes

• image_publisher_node: color camera nodes and detection nodes for CAM_B and CAM_C, no ROS.
• image_publisher_ros_node: color camera nodes and detection nodes for CAM_B and CAM_C, also ROS bridges to publish image and detection to ROS.
• image_publisher_ros_single_detection_node: color camera nodes for both cameras, only one camera has detection node. ROS publishing image from two cameras and detection from one camera.

In term of severity of the issue

For image_publisher_node, the blocking issue can be seen directly from the screen, it happens around 1 out of 10

For image_publisher_ros_node, we need pipeline_graph to see the FPS. It happens around 2 out of 5

For image_publisher_ros_single_detection_node, we need pipeline_graph to see the FPS, I have NOT see the blocking happen yet

Also, like I mentioned a while ago, the issue rarely happens with the python MRE even with IMX477 (1 out of 30?). But I did see it happen before.

So in summary

• The inconsistency of the issue with IMX477 modules in different pipelines, make it looks like it is more on the firmware side.
• But absolutely no issue with IMX214 in all pipelines make it looks like IMX477 hardware is the one to blame.

Sorry for the mess. Let me know what do you think.

Thanks

Lincoln

erik

Another information may or may not be relevant. Even though the Luxonis shop is still branding the module as IMX477, but Arducam actually updated the sensor to IMX577, see the description on their own shop for the same module. And the cameras I received and am testing is the updated version with IMX577 sensors. So in order to reproduce the issues, you may need to make sure using the same version.

Thanks

Lincoln

Hi erik,

Thanks for trying to repro the issue. I really appreciate it. Just to make sure I did it the right way. What arguments did you provide to launch the container while reproducing the C++ MRE? Here is my command

docker container run --net=host -it --rm \
    --privileged \
    -e DISPLAY=$DISPLAY \
    -e QT_X11_NO_MITSHM=1 \
    -v /dev/bus/usb:/dev/bus/usb \
    -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
    -v $HOME/.Xauthority:/root/.Xauthority:rw \
    -v $HOME/data/:/data:rw \
    --device /dev/i2c-1 \
    depthai-ros \
    bash

Also, I guess there are no issue with all three nodes while using IMX477?

Thanks

Lincoln

Hi @jakaskerl, @DaniloPejovic, @erik, @Luxonis-Adam,

Sorry for continuously spamming you guys with this issue.

Try to make everyone's life easier, I built a public docker image (amd64 and arm64) with depthai ros noetic and the MRE. And I created a github repo to launch the MRE with pipeline_graph output with one command. So to reproduce, simply three steps

• Clone this repo by git clone https://github.com/lincolnxlw/depthai_dual_cam_ros_detector.git.

• Download my prebuilt docker image by docker pull lincolnxlw/my-depthai-ros.

• Run the main script in the repo's root folder ./run.sh.

I also provided details of my testing result in the README. When using IMX577, around 30% failure rate for two different hosts (my laptop and CM4). 0% failure rate when using IMX214.

@DaniloPejovic, could you reconfirm with the prebuilt docker image that there are no issues with using IMX477? If yes, do you have the model number for the IMX477 you used (should be something like B0346)? I will need to request the exact same model from Arducam and with the Chinese New Year around the corner, I may only have one shot to get a production batch before the New Year.

Thanks

Lincoln

Hi jakaskerl,

Thanks for the confirmation of the issue. The depthai precompiled 64 bit pi OS is in Debian bookworm and ROS noetic is not supported natively. I can try to build it from source but it can be messy. Do you guys have an 64bit pi OS with ROS noetic built-in I can use?

Our product's software is designed using a container-based architecture, so not using container will involve heavy system redesign. I will test without container for the MRE now of course. But any chance the team look into this ROS and container issue with IMX577/IMX477?

Thanks and let me know what you think,

Lincoln