Pipeline sometimes not being launched as expected

Llincolnxlw · Jan 19, 2024

Another information may or may not be relevant. Even though the Luxonis shop is still branding the module as IMX477, but Arducam actually updated the sensor to IMX577, see the description on their own shop for the same module. And the cameras I received and am testing is the updated version with IMX577 sensors. So in order to reproduce the issues, you may need to make sure using the same version.

Thanks

Lincoln

erik · Jan 20, 2024

Hi @lincolnxlw ,
We just ordered the exact same cameras (from Arducam), hopefully they will arrive in the next 2 weeks and we'll try to repro then. We tried with the IMX477 many times and didn't encounter the issue.
Thanks, Erik

Llincolnxlw · Jan 21, 2024

Hi erik,

Thanks for trying to repro the issue. I really appreciate it. Just to make sure I did it the right way. What arguments did you provide to launch the container while reproducing the C++ MRE? Here is my command

docker container run --net=host -it --rm \
    --privileged \
    -e DISPLAY=$DISPLAY \
    -e QT_X11_NO_MITSHM=1 \
    -v /dev/bus/usb:/dev/bus/usb \
    -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
    -v $HOME/.Xauthority:/root/.Xauthority:rw \
    -v $HOME/data/:/data:rw \
    --device /dev/i2c-1 \
    depthai-ros \
    bash

Also, I guess there are no issue with all three nodes while using IMX477?

Thanks

Lincoln

jakaskerl · Jan 21, 2024

Hi lincolnxlw
I was not able to setup a ros environment since I am running MacOS, but tried it with python and ran the script 300+ times (had it on script) and encountered no issues. Will try with 577s once I receive them. The sensor should be fully supported (https://docs.luxonis.com/projects/hardware/en/latest/pages/articles/supported_sensors/), but was likely not tested to such an extent.

Thanks,
Jaka

Llincolnxlw · Jan 22, 2024

Hi jakaskerl

Thanks for the extensive test with python! But see my description in post #21, for some reason the python MRE is the one with the lowest severity comparing to all the other cases. I can rarely produce a failure with it. Did someone in the team try to reproduce with the c++ code?

Thanks

Lincoln

jakaskerl · Jan 22, 2024

Hi lincolnxlw
Sorry if I've missed the code somewhere, but afaik the cpp code was for ROS only? The is no cpp code that runs standalone like python right? Please correct me if I've missed something.
The python API are just bindings for cpp so in theory, it should have the same fault occurrence frequency as the cpp examples, unless there is something else going on.

Thanks,
Jaka

Llincolnxlw · Jan 22, 2024

Hi jakaskerl

Sorry I meant the C++ code for ROS I provided above. Have someone from the team try to run them with IMX477 extensively like you do with the python example? I know we are waiting for IMX577 for further testing according to @erik. Just want to make sure so far we can confirm IMX477 do not have issue with all three ROS nodes I provided (all the modules I purchased turned out to be IMX577). If that is the case, for the time being I can request Arducam to ship us IMX477 specifically so we can release our product without further delay (looks like they have same PCB dimension so our existing plastic mount should still work). Does that make sense?

Thanks

Lincoln

jakaskerl · Jan 22, 2024

Hi lincolnxlw
@DaniloPejovic tested your MRE on ROS Noetic as well and encountered no issues. Looks like only the IMX577s are problematic.

Thanks,
Jaka

Llincolnxlw · Jan 22, 2024

Hi jakaskerl,

Got it. Thanks.

Hi @DaniloPejovic,

thanks again for testing the C++ MRE! Like I mentioned earlier in #27, I want to make sure the arguments I passed to docker run look good to you (the OAK-FFC-3P is connected to an Pi4 running precompiled OS OAK_CM4_POE_V10_64bit). what do you think

docker container run --net=host -it --rm \
    --privileged \
    -e DISPLAY=$DISPLAY \
    -e QT_X11_NO_MITSHM=1 \
    -v /dev/bus/usb:/dev/bus/usb \
    -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
    -v $HOME/.Xauthority:/root/.Xauthority:rw \
    -v $HOME/data/:/data:rw \
    --device /dev/i2c-1 \
    depthai-ros \
    bash

Thanks

Lincoln

Llincolnxlw · Jan 23, 2024

Hi @jakaskerl, @DaniloPejovic, @erik, @Luxonis-Adam,

Sorry for continuously spamming you guys with this issue.

Try to make everyone's life easier, I built a public docker image (amd64 and arm64) with depthai ros noetic and the MRE. And I created a github repo to launch the MRE with pipeline_graph output with one command. So to reproduce, simply three steps

Clone this repo by git clone https://github.com/lincolnxlw/depthai_dual_cam_ros_detector.git.
Download my prebuilt docker image by docker pull lincolnxlw/my-depthai-ros.
Run the main script in the repo's root folder ./run.sh.

I also provided details of my testing result in the README. When using IMX577, around 30% failure rate for two different hosts (my laptop and CM4). 0% failure rate when using IMX214.

@DaniloPejovic, could you reconfirm with the prebuilt docker image that there are no issues with using IMX477? If yes, do you have the model number for the IMX477 you used (should be something like B0346)? I will need to request the exact same model from Arducam and with the Chinese New Year around the corner, I may only have one shot to get a production batch before the New Year.

Thanks

Lincoln

jakaskerl · Jan 23, 2024

Hi lincolnxlw
By using your docker image, I was able to reproduce the issue on both IMX577 and IMX477. Tested with ov9782 and AR0234 and had no issues. I was unable to reproduce the issue in python, so I'm assuming the problem lies somewhere in ROS or docker.
Try running this on non-containerized ROS noetic.

Thanks,
Jaka

Llincolnxlw · Jan 23, 2024

Hi jakaskerl,

Thanks for the confirmation of the issue. The depthai precompiled 64 bit pi OS is in Debian bookworm and ROS noetic is not supported natively. I can try to build it from source but it can be messy. Do you guys have an 64bit pi OS with ROS noetic built-in I can use?

Our product's software is designed using a container-based architecture, so not using container will involve heavy system redesign. I will test without container for the MRE now of course. But any chance the team look into this ROS and container issue with IMX577/IMX477?

Thanks and let me know what you think,

Lincoln

jakaskerl · Jan 24, 2024

Hi @lincolnxlw

lincolnxlw But any chance the team look into this ROS and container issue with IMX577/IMX477?

Of course, we are just trying to pinpoint the issue to either ROS or Docker so we can solve it.

Thanks,
Jaka

Llincolnxlw · Jan 24, 2024

jakaskerl

Thanks, please keep me posted and let me know what else you need from my end.

Thanks

Lincoln

jakaskerl · Jan 29, 2024

Hi @lincolnxlw
I tested the dockerized version of the depthai-core library and the cameras work as expected. Do other pure depthai examples work for you in the ros envirnoment? The code you have sent that was "without ROS" still had includes and ROS functions so I could not run it. Can you run an example that doesn't have any reference to ROS inside the container?
I feel like this might be a ros specific issue.

Thanks,
Jaka

Llincolnxlw · Jan 29, 2024

Hi jakaskerl

I updated the repo with a program running the pipeline with pure depthai c++. Here is the CMakeList.txt detail (no linking to any ROS libraries) for this program. I also rebuilt the docker image, just pull the latest version and use this script to run the pipeline without ROS. And yes the issue still persist (around 1 out of 5 times). Feel free to use the source code and build it in other environment.

add_executable(image_publisher_node src/image_publisher.cpp)

target_link_libraries(image_publisher_node
  ${OpenCV_LIBRARIES}
  depthai::core
  yaml-cpp
)

But regardless, our application have downstream tasks that rely on ROS to receive image data and detection results from the depthai pipeline. So ROS is needed for us.

Thanks

Lincoln