DepthAI with isaac ROS NITROS Results in Crash

Igor · 13 Mar

I am writing a ros2 node which takes compressed images from depth AI, decodes them using NVIDIA NVJPEG and publishes them to isaac ROS NITROS topic. Whenever I try to boot up the node with a ROS NITROS managed publisher and depthai pipeline, the composed container crashes.

Here is a snippet of the pipeline creation:

dai::Pipeline pipeline;
    auto colorCam = pipeline.create<dai::node::ColorCamera>();
    colorCam->setBoardSocket(dai::CameraBoardSocket::CAM_A);
    colorCam->setResolution(dai::ColorCameraProperties::SensorResolution::THE_4_K); /// 3840 × 2160
    colorCam->setFps(frame_rate_);

    auto videoEnc = pipeline.create<dai::node::VideoEncoder>();
    videoEnc->setDefaultProfilePreset(frame_rate_, dai::VideoEncoderProperties::Profile::MJPEG);
    videoEnc->setQuality(compression_quality_);
    videoEnc->setLossless(false);
    colorCam->video.link(videoEnc->input);

    auto xout = pipeline.create<dai::node::XLinkOut>();
    xout->setStreamName("rgb");
    videoEnc->bitstream.link(xout->input);

    if(ip_addr_.empty()){
      RCLCPP_INFO(this->get_logger(), "Initialized device with no IP");
      device_ = std::make_shared<dai::Device>(pipeline);
    } else {
      RCLCPP_INFO(this->get_logger(), "Initialized device with IP address %s", ip_addr_.c_str());
      auto device_info = dai::DeviceInfo(ip_addr_);
      device_ = std::make_shared<dai::Device>(pipeline, device_info);
    }

I create the ROS NITROS topic like so:

  nitros_pub_ = std::make_shared<nvidia::isaac_ros::nitros::ManagedNitrosPublisher<
    nvidia::isaac_ros::nitros::NitrosImage>>(
    this, "/IPCams/" + camera_name_ + "/image_raw",
    nvidia::isaac_ros::nitros::nitros_image_bgr8_t::supported_type_name
  );

It seems to crash every whenever I try and initialize the device even without any processing of the frames downstream with the following message:

[ERROR] [component_container_mt-1]: process has died [pid 361008, exit code -11, cmd '/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args -r __node:=oak1_container -r __ns:=/'].

The node works fine without isaac ROS NITROS, and fails at runtime if I even just add the NITROS headers:

#include "isaac_ros_managed_nitros/managed_nitros_publisher.hpp"
#include "isaac_ros_nitros_image_type/nitros_image.hpp"

For reference I am using the OAK-1 W PoE.

jakaskerl · 14 Mar

Igor
cc @Luxonis-Adam not sure if even supported but perhaps you have some insight

Luxonis-Adam · 14 Mar

I think it would be the best if you could run the code with GDB and show inspect the backtrace. Is DepthAI built from source or installed from APT?

Igor · 14 Mar

Depthai core is built from source with version 2.29.0. I will look into running the code with GDB and get back to you.

Igor · 14 Mar

Here is the back trace,

   #0  0x00007cd9f85a3ad9 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >::basic_json(nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > > const&) ()
   from /home/dev/github/AIRCoM_main/install/depthai_bridge/lib/libdepthai_bridge.so
#1  0x00007cda041ed3c0 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >::basic_json(std::initializer_list<nlohmann::detail::json_ref<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > > > >, bool, nlohmann::detail::value_t) () from /opt/isaac_ros_depends/install/gxf_isaac_optimizer/share/gxf_isaac_optimizer/gxf/lib/libgxf_isaac_optimizer.so
#2  0x00007cd9f199dff2 in nlohmann::detail::to_json_tuple_impl<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool>, 0ul, 1ul, 2ul> (j=..., 
    t=std::tuple containing = {...}) at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4338
#3  0x00007cd9f1995b91 in nlohmann::detail::to_json<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool>, 0> (j=..., t=std::tuple containing = {...})
    at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4344
#4  0x00007cd9f198a3f6 in nlohmann::detail::to_json_fn::operator()<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool> const&> (
    this=0x7cd9f85d87b1 <nlohmann::detail::static_const<nlohmann::detail::to_json_fn>::value>, j=..., 
    val=std::tuple containing = {...}) at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4353
#5  0x00007cd9f197a934 in nlohmann::adl_serializer<std::tuple<long, int, bool>, void>::to_json<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool> const&> (j=..., val=std::tuple containing = {...})
--Type <RET> for more, q to quit, c to continue without paging--c
    at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4403
#6  0x00007cd9f196c687 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >::basic_json<std::tuple<long, int, bool> const&, std::tuple<long, int, bool>, 0> (this=0x7cda05ffce20, val=std::tuple containing = {...}) at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:17931
#7  0x00007cd9f195ac11 in nanorpc::packer::nlohmann_msgpack::serializer::pack_value<std::tuple<long, int, bool> > (this=0x7cda05ffcf80, value=std::tuple containing = {...}) at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/packer/nlohmann_msgpack.h:93
#8  0x00007cd9f19468bf in nanorpc::packer::nlohmann_msgpack::serializer::pack<std::tuple<long, int, bool> > (this=0x7cda05ffcf80, value=std::tuple containing = {...}) at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/packer/nlohmann_msgpack.h:64
#9  0x00007cd9f1930d68 in nanorpc::core::client<nanorpc::packer::nlohmann_msgpack>::call<long, int&, bool&> (this=0x7cd9c5287e00, id=10182484530226687077) at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/core/client.h:59
#10 0x00007cd9f191d2bf in nanorpc::core::client<nanorpc::packer::nlohmann_msgpack>::call<long, int&, bool&> (this=0x7cd9c5287e00, name="setTimesync") at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/core/client.h:50
#11 0x00007cd9f18fa9f8 in dai::DeviceBase::setTimesync (this=0x7cd9c52895c0, period=..., numSamples=10, random=true) at /opt/depthai-core/src/device/DeviceBase.cpp:1300
#12 0x00007cd9f18f672c in dai::DeviceBase::init2 (this=0x7cd9c52895c0, cfg=..., pathToMvcmd=..., pipeline=...) at /opt/depthai-core/src/device/DeviceBase.cpp:1018
#13 0x00007cd9f18f2be4 in dai::DeviceBase::DeviceBase (this=0x7cd9c52895c0, config=..., devInfo=...) at /opt/depthai-core/src/device/DeviceBase.cpp:509
#14 0x00007cd9f18d61fb in dai::Device::Device (this=0x7cd9c52895c0, pipeline=..., devInfo=...) at /opt/depthai-core/src/device/Device.cpp:44
#15 0x00007cda138c0633 in __gnu_cxx::new_allocator<dai::Device>::construct<dai::Device, dai::Pipeline&, dai::DeviceInfo&> (__p=0x7cd9c52895c0, this=<optimized out>) at /usr/include/c++/11/ext/new_allocator.h:162
#16 std::allocator_traits<std::allocator<dai::Device> >::construct<dai::Device, dai::Pipeline&, dai::DeviceInfo&> (__p=0x7cd9c52895c0, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:516
#17 std::_Sp_counted_ptr_inplace<dai::Device, std::allocator<dai::Device>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<dai::Pipeline&, dai::DeviceInfo&> (__a=..., this=0x7cd9c52895b0) at /usr/include/c++/11/bits/shared_ptr_base.h:519
#18 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<dai::Device, std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__a=..., __p=<optimized out>, this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:650
#19 std::__shared_ptr<dai::Device, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__tag=..., this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:1342
#20 std::shared_ptr<dai::Device>::shared_ptr<std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__tag=..., this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr.h:409
#21 std::allocate_shared<dai::Device, std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__a=...) at /usr/include/c++/11/bits/shared_ptr.h:863
#22 std::make_shared<dai::Device, dai::Pipeline&, dai::DeviceInfo&> () at /usr/include/c++/11/bits/shared_ptr.h:879
#23 RgbCameraPublisher::RgbCameraPublisher (this=<optimized out>, options=..., this=<optimized out>, options=...) at /home/dev/github/AIRCoM_main/src/sensor_pkg/src/oak_rgb_publisher.cpp:103
#24 0x00007cda13922908 in __gnu_cxx::new_allocator<RgbCameraPublisher>::construct<RgbCameraPublisher, rclcpp::NodeOptions const&> (__p=0x7cd9c45bd9c0, this=<optimized out>) at /usr/include/c++/11/ext/new_allocator.h:162
#25 std::allocator_traits<std::allocator<RgbCameraPublisher> >::construct<RgbCameraPublisher, rclcpp::NodeOptions const&> (__p=0x7cd9c45bd9c0, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:516
#26 std::_Sp_counted_ptr_inplace<RgbCameraPublisher, std::allocator<RgbCameraPublisher>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<rclcpp::NodeOptions const&> (__a=..., this=0x7cd9c45bd9b0) at /usr/include/c++/11/bits/shared_ptr_base.h:519
#27 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<RgbCameraPublisher, std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__a=..., __p=<synthetic pointer>: <optimized out>, this=<synthetic pointer>) at /usr/include/c++/11/bits/shared_ptr_base.h:650
#28 std::__shared_ptr<RgbCameraPublisher, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__tag=..., this=<synthetic pointer>) at /usr/include/c++/11/bits/shared_ptr_base.h:1342
#29 std::shared_ptr<RgbCameraPublisher>::shared_ptr<std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__tag=..., this=<synthetic pointer>) at /usr/include/c++/11/bits/shared_ptr.h:409
#30 std::allocate_shared<RgbCameraPublisher, std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__a=...) at /usr/include/c++/11/bits/shared_ptr.h:863
#31 std::make_shared<RgbCameraPublisher, rclcpp::NodeOptions const&> () at /usr/include/c++/11/bits/shared_ptr.h:879
#32 rclcpp_components::NodeFactoryTemplate<RgbCameraPublisher>::create_node_instance (this=<optimized out>, options=...) at /opt/ros/humble/include/rclcpp_components/rclcpp_components/node_factory_template.hpp:45
#33 0x00007cda164d9961 in rclcpp_components::ComponentManager::on_load_node(std::shared_ptr<rmw_request_id_s>, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >) () from /opt/ros/humble/lib/libcomponent_manager.so
#34 0x00007cda164da925 in std::_Function_handler<void (std::shared_ptr<rmw_request_id_s>, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >), std::_Bind<void (rclcpp_components::ComponentManager::*(rclcpp_components::ComponentManager*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(std::shared_ptr<rmw_request_id_s>, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >)> >::_M_invoke(std::_Any_data const&, std::shared_ptr<rmw_request_id_s>&&, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >&&, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >&&) () from /opt/ros/humble/lib/libcomponent_manager.so
#35 0x00007cda164e42ab in ?? () from /opt/ros/humble/lib/libcomponent_manager.so
#36 0x00007cda163af3d6 in ?? () from /opt/ros/humble/lib/librclcpp.so
#37 0x00007cda163acdba in rclcpp::Executor::execute_service(std::shared_ptr<rclcpp::ServiceBase>) () from /opt/ros/humble/lib/librclcpp.so
#38 0x00007cda163ad126 in rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) () from /opt/ros/humble/lib/librclcpp.so
#39 0x00007cda163b447a in rclcpp::executors::MultiThreadedExecutor::run(unsigned long) () from /opt/ros/humble/lib/librclcpp.so
#40 0x00007cda1613a253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#41 0x00007cda15ea7ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#42 0x00007cda15f38a04 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

Igor · 17 Mar

Fixed the runtime error by applying a patch in my dockerfile to the depthaiDependencies.cmake to use the system installed version of nlohmann_json. Would be nice if there was a cmake flag to turn off hunter for specific dependencies. Perhaps something like what is mentioned in: luxonis/depthai-coreissues/1021. Curious if you have any input on how to do this better.