• DepthAIROS

  • DepthAI with isaac ROS NITROS Results in Crash

I am writing a ros2 node which takes compressed images from depth AI, decodes them using NVIDIA NVJPEG and publishes them to isaac ROS NITROS topic. Whenever I try to boot up the node with a ROS NITROS managed publisher and depthai pipeline, the composed container crashes.

Here is a snippet of the pipeline creation:

dai::Pipeline pipeline;
    auto colorCam = pipeline.create<dai::node::ColorCamera>();
    colorCam->setBoardSocket(dai::CameraBoardSocket::CAM_A);
    colorCam->setResolution(dai::ColorCameraProperties::SensorResolution::THE_4_K); /// 3840 × 2160
    colorCam->setFps(frame_rate_);

    auto videoEnc = pipeline.create<dai::node::VideoEncoder>();
    videoEnc->setDefaultProfilePreset(frame_rate_, dai::VideoEncoderProperties::Profile::MJPEG);
    videoEnc->setQuality(compression_quality_);
    videoEnc->setLossless(false);
    colorCam->video.link(videoEnc->input);

    auto xout = pipeline.create<dai::node::XLinkOut>();
    xout->setStreamName("rgb");
    videoEnc->bitstream.link(xout->input);

    if(ip_addr_.empty()){
      RCLCPP_INFO(this->get_logger(), "Initialized device with no IP");
      device_ = std::make_shared<dai::Device>(pipeline);
    } else {
      RCLCPP_INFO(this->get_logger(), "Initialized device with IP address %s", ip_addr_.c_str());
      auto device_info = dai::DeviceInfo(ip_addr_);
      device_ = std::make_shared<dai::Device>(pipeline, device_info);
    }

I create the ROS NITROS topic like so:

  nitros_pub_ = std::make_shared<nvidia::isaac_ros::nitros::ManagedNitrosPublisher<
    nvidia::isaac_ros::nitros::NitrosImage>>(
    this, "/IPCams/" + camera_name_ + "/image_raw",
    nvidia::isaac_ros::nitros::nitros_image_bgr8_t::supported_type_name
  );

It seems to crash every whenever I try and initialize the device even without any processing of the frames downstream with the following message:

[ERROR] [component_container_mt-1]: process has died [pid 361008, exit code -11, cmd '/opt/ros/humble/lib/rclcpp_components/component_container_mt --ros-args -r __node:=oak1_container -r __ns:=/'].

The node works fine without isaac ROS NITROS, and fails at runtime if I even just add the NITROS headers:

#include "isaac_ros_managed_nitros/managed_nitros_publisher.hpp"
#include "isaac_ros_nitros_image_type/nitros_image.hpp"

For reference I am using the OAK-1 W PoE.

    I think it would be the best if you could run the code with GDB and show inspect the backtrace. Is DepthAI built from source or installed from APT?

    Depthai core is built from source with version 2.29.0. I will look into running the code with GDB and get back to you.

    Here is the back trace,

       #0  0x00007cd9f85a3ad9 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >::basic_json(nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > > const&) ()
   from /home/dev/github/AIRCoM_main/install/depthai_bridge/lib/libdepthai_bridge.so
#1  0x00007cda041ed3c0 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >::basic_json(std::initializer_list<nlohmann::detail::json_ref<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > > > >, bool, nlohmann::detail::value_t) () from /opt/isaac_ros_depends/install/gxf_isaac_optimizer/share/gxf_isaac_optimizer/gxf/lib/libgxf_isaac_optimizer.so
#2  0x00007cd9f199dff2 in nlohmann::detail::to_json_tuple_impl<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool>, 0ul, 1ul, 2ul> (j=..., 
    t=std::tuple containing = {...}) at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4338
#3  0x00007cd9f1995b91 in nlohmann::detail::to_json<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool>, 0> (j=..., t=std::tuple containing = {...})
    at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4344
#4  0x00007cd9f198a3f6 in nlohmann::detail::to_json_fn::operator()<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool> const&> (
    this=0x7cd9f85d87b1 <nlohmann::detail::static_const<nlohmann::detail::to_json_fn>::value>, j=..., 
    val=std::tuple containing = {...}) at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4353
#5  0x00007cd9f197a934 in nlohmann::adl_serializer<std::tuple<long, int, bool>, void>::to_json<nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >, std::tuple<long, int, bool> const&> (j=..., val=std::tuple containing = {...})
--Type <RET> for more, q to quit, c to continue without paging--c
    at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:4403
#6  0x00007cd9f196c687 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >::basic_json<std::tuple<long, int, bool> const&, std::tuple<long, int, bool>, 0> (this=0x7cda05ffce20, val=std::tuple containing = {...}) at /root/.hunter/_Base/16cc954/21498a3/624857f/Install/include/nlohmann/json.hpp:17931
#7  0x00007cd9f195ac11 in nanorpc::packer::nlohmann_msgpack::serializer::pack_value<std::tuple<long, int, bool> > (this=0x7cda05ffcf80, value=std::tuple containing = {...}) at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/packer/nlohmann_msgpack.h:93
#8  0x00007cd9f19468bf in nanorpc::packer::nlohmann_msgpack::serializer::pack<std::tuple<long, int, bool> > (this=0x7cda05ffcf80, value=std::tuple containing = {...}) at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/packer/nlohmann_msgpack.h:64
#9  0x00007cd9f1930d68 in nanorpc::core::client<nanorpc::packer::nlohmann_msgpack>::call<long, int&, bool&> (this=0x7cd9c5287e00, id=10182484530226687077) at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/core/client.h:59
#10 0x00007cd9f191d2bf in nanorpc::core::client<nanorpc::packer::nlohmann_msgpack>::call<long, int&, bool&> (this=0x7cd9c5287e00, name="setTimesync") at /opt/depthai-core/shared/depthai-shared/3rdparty/nanorpc/core/client.h:50
#11 0x00007cd9f18fa9f8 in dai::DeviceBase::setTimesync (this=0x7cd9c52895c0, period=..., numSamples=10, random=true) at /opt/depthai-core/src/device/DeviceBase.cpp:1300
#12 0x00007cd9f18f672c in dai::DeviceBase::init2 (this=0x7cd9c52895c0, cfg=..., pathToMvcmd=..., pipeline=...) at /opt/depthai-core/src/device/DeviceBase.cpp:1018
#13 0x00007cd9f18f2be4 in dai::DeviceBase::DeviceBase (this=0x7cd9c52895c0, config=..., devInfo=...) at /opt/depthai-core/src/device/DeviceBase.cpp:509
#14 0x00007cd9f18d61fb in dai::Device::Device (this=0x7cd9c52895c0, pipeline=..., devInfo=...) at /opt/depthai-core/src/device/Device.cpp:44
#15 0x00007cda138c0633 in __gnu_cxx::new_allocator<dai::Device>::construct<dai::Device, dai::Pipeline&, dai::DeviceInfo&> (__p=0x7cd9c52895c0, this=<optimized out>) at /usr/include/c++/11/ext/new_allocator.h:162
#16 std::allocator_traits<std::allocator<dai::Device> >::construct<dai::Device, dai::Pipeline&, dai::DeviceInfo&> (__p=0x7cd9c52895c0, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:516
#17 std::_Sp_counted_ptr_inplace<dai::Device, std::allocator<dai::Device>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<dai::Pipeline&, dai::DeviceInfo&> (__a=..., this=0x7cd9c52895b0) at /usr/include/c++/11/bits/shared_ptr_base.h:519
#18 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<dai::Device, std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__a=..., __p=<optimized out>, this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:650
#19 std::__shared_ptr<dai::Device, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__tag=..., this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:1342
#20 std::shared_ptr<dai::Device>::shared_ptr<std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__tag=..., this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr.h:409
#21 std::allocate_shared<dai::Device, std::allocator<dai::Device>, dai::Pipeline&, dai::DeviceInfo&> (__a=...) at /usr/include/c++/11/bits/shared_ptr.h:863
#22 std::make_shared<dai::Device, dai::Pipeline&, dai::DeviceInfo&> () at /usr/include/c++/11/bits/shared_ptr.h:879
#23 RgbCameraPublisher::RgbCameraPublisher (this=<optimized out>, options=..., this=<optimized out>, options=...) at /home/dev/github/AIRCoM_main/src/sensor_pkg/src/oak_rgb_publisher.cpp:103
#24 0x00007cda13922908 in __gnu_cxx::new_allocator<RgbCameraPublisher>::construct<RgbCameraPublisher, rclcpp::NodeOptions const&> (__p=0x7cd9c45bd9c0, this=<optimized out>) at /usr/include/c++/11/ext/new_allocator.h:162
#25 std::allocator_traits<std::allocator<RgbCameraPublisher> >::construct<RgbCameraPublisher, rclcpp::NodeOptions const&> (__p=0x7cd9c45bd9c0, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:516
#26 std::_Sp_counted_ptr_inplace<RgbCameraPublisher, std::allocator<RgbCameraPublisher>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<rclcpp::NodeOptions const&> (__a=..., this=0x7cd9c45bd9b0) at /usr/include/c++/11/bits/shared_ptr_base.h:519
#27 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<RgbCameraPublisher, std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__a=..., __p=<synthetic pointer>: <optimized out>, this=<synthetic pointer>) at /usr/include/c++/11/bits/shared_ptr_base.h:650
#28 std::__shared_ptr<RgbCameraPublisher, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__tag=..., this=<synthetic pointer>) at /usr/include/c++/11/bits/shared_ptr_base.h:1342
#29 std::shared_ptr<RgbCameraPublisher>::shared_ptr<std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__tag=..., this=<synthetic pointer>) at /usr/include/c++/11/bits/shared_ptr.h:409
#30 std::allocate_shared<RgbCameraPublisher, std::allocator<RgbCameraPublisher>, rclcpp::NodeOptions const&> (__a=...) at /usr/include/c++/11/bits/shared_ptr.h:863
#31 std::make_shared<RgbCameraPublisher, rclcpp::NodeOptions const&> () at /usr/include/c++/11/bits/shared_ptr.h:879
#32 rclcpp_components::NodeFactoryTemplate<RgbCameraPublisher>::create_node_instance (this=<optimized out>, options=...) at /opt/ros/humble/include/rclcpp_components/rclcpp_components/node_factory_template.hpp:45
#33 0x00007cda164d9961 in rclcpp_components::ComponentManager::on_load_node(std::shared_ptr<rmw_request_id_s>, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >) () from /opt/ros/humble/lib/libcomponent_manager.so
#34 0x00007cda164da925 in std::_Function_handler<void (std::shared_ptr<rmw_request_id_s>, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >), std::_Bind<void (rclcpp_components::ComponentManager::*(rclcpp_components::ComponentManager*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(std::shared_ptr<rmw_request_id_s>, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >)> >::_M_invoke(std::_Any_data const&, std::shared_ptr<rmw_request_id_s>&&, std::shared_ptr<composition_interfaces::srv::LoadNode_Request_<std::allocator<void> > >&&, std::shared_ptr<composition_interfaces::srv::LoadNode_Response_<std::allocator<void> > >&&) () from /opt/ros/humble/lib/libcomponent_manager.so
#35 0x00007cda164e42ab in ?? () from /opt/ros/humble/lib/libcomponent_manager.so
#36 0x00007cda163af3d6 in ?? () from /opt/ros/humble/lib/librclcpp.so
#37 0x00007cda163acdba in rclcpp::Executor::execute_service(std::shared_ptr<rclcpp::ServiceBase>) () from /opt/ros/humble/lib/librclcpp.so
#38 0x00007cda163ad126 in rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) () from /opt/ros/humble/lib/librclcpp.so
#39 0x00007cda163b447a in rclcpp::executors::MultiThreadedExecutor::run(unsigned long) () from /opt/ros/humble/lib/librclcpp.so
#40 0x00007cda1613a253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#41 0x00007cda15ea7ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#42 0x00007cda15f38a04 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

    Fixed the runtime error by applying a patch in my dockerfile to the depthaiDependencies.cmake to use the system installed version of nlohmann_json. Would be nice if there was a cmake flag to turn off hunter for specific dependencies. Perhaps something like what is mentioned in: luxonis/depthai-coreissues/1021. Curious if you have any input on how to do this better.