BradleyDillon

  • 25 minutes ago
  • Joined Feb 15, 2023
  • 0 best answers
  • Luxonis turns six years old this week! In this update we will share details of the recent progress that we’ve made along with outlining what you can expect in the future. 

    To start with, it’s an absolute privilege to get to work with some of the brightest and most driven engineers on the planet to help our customers win. The most rewarding part of the job is getting to see our customers reach scaled commercial deployments with our technology. Our customer’s use cases vary across a variety of robotics and automation applications. When we replace legacy technology or pioneer new ways of solving problems, we are helping to save lives and deliver productivity gains. The metric that we obsess about the most is our business conversion rate, which represents the portion of our customers reaching commercial scale versus those that are only running pilots. We have seen this metric increase 5x in the last four years! 

    Our team has continued to grow as we mature our complicated technological stack and use it to tackle the ever-changing complexities of the physical world. Half of global GDP is driven by physical human labor that requires the ability to see and perceive the world in real time. At Luxonis, our mission is to enable systems that can exceed humans at every possible task or project in the physical world. We are building essentially the best perceiving superhero on the planet; one that can see, sense, and process the physical world better than any human can. 

    This is why our devices provide sensor fusion on steroids with high resolution color cameras, stereo vision, Infrared, LiDAR, Thermal, IMUs, and microphones all synced together in unison dozens of times per second. And this is why our engineering team now consists of 80+ professionals who believe in our mission and drive rapid, iterative improvements across a variety of disciplines including electrical, mechanical, industrial design, firmware, cloud, applications, machine learning, synthetic data, and computer vision. We are always looking for great talent, and if you are interested in joining our team, please reach out to [careers@luxonis.com](mailto:careers@luxonis.com). 

    Recently, we’ve released a number of new and exciting products. In the new sensing category, we have introduced new products including our first Thermal device (here) and our first Time of Flight device (here). To enable the ability to see and perceive depth at far distances, we’ve launched our Long Range device (here) with a 15cm stereo baseline that allows for accurate depth at greater than 30 meters. And most exciting of all, we’ve continued to roll out and scale up manufacturing of our new OAK Series 4 of products. The OAK 4 has everything that you love with our current OAK lineup, along with some key new features: 

    • Power & Connectivity available with either USB or Ethernet 

    • Standard 48MP RGB camera with support available for up to 108MP 

    • An Octacore ARM processor running YOCTO Linux 

    • 48 TOPS for AI Inference 

    Series 4 is an edge AI sensor fusion powerhouse allowing you to do more on the edge in an all-in-one device than you’ve ever been able to do before. OAK 4 is your new go-to to option if you require a standalone device for automation use cases, along with if you need more capable sensors and processing in your larger robotics or automation system. A quick word on where this leaves our original OAK Series 2 lineup  

    Our Myriad X based based OAK Series 2 is here to stay and remains the best option for a variety of use cases. If you need a stereo camera as a peripheral device with industry leading stereo (here) then OAK Series 2 is an excellent option. Luxonis has a large amount of customers who have spec’d in the OAK Series 2 for their robotics or automation applications and plan to use it for years to come. This means that Luxonis customers can expect 1) Reliable supply of the OAK Series 2 for years to come (here), 2) Continued rapid pace of new DepthAI releases every few weeks, and 3) High touch support for helping OAK Series 2 solve your problem at scale. 

    If you want to give us input on what we should improve and what we should do next with our products, please reach out. 

    The last exciting thing that we’ll share is that Luxonis is soon to be introducing a patent-pending technology that allows our stereo depth to be self-healing. Available now upon request and planned to be integrated into DepthAI by early this Summer, this new technology gives our depth algorithm the ability to dynamically recalibrate in the field. This means that you can expect to have a vision system that always works: it works out of the box and it works even after it’s gone through mechanical & thermal stress.

    Here are some additional rapid-fire items that have happened in the last year at Luxonis: 

    • We have scaled up manufacturing at a new location in Taiwan 

    • We opened an office in Austin, TX bringing our global office total to six 

    • We revamped our website:  https://www.luxonis.com/ 

    • We provided only our second pricing refresh in the last six years, you can still place orders at current prices through April 20th, 2025:  here

    For those of you who have lived startup life, you’ll know that it is a roller coaster with lots of extreme ups and downs. Fortunately, we love it and know that it’s all a part of the journey in bringing our mission and engineering efficiency to our customers. We believe that our 7th year in business is going to be our best yet. Stay tuned! 

  • @ThomasA We will have OAK ToF units available again in approximately 6 weeks. Thanks for your patience! The sooner you place an order the higher up on the list for shipping you'll be.

  • As a follow up to our blog here on seeking to never discontinue our poducts (here), we’d like to share an update with our community. Two values that are core to all that we do at Luxonis are:  customer obsession and being engineering centric. It is these values that drive our relentless efforts to never end-of-life our product lines. We know that our customers and developers using our platform have often spent years designing, testing, and scaling up with our products and that makes the effort required to make a switch a massive undertaking. These factors are what drive our excitement in sharing that we will continue providing our OAK Series 1 and 2 of cameras with abundant supply for the next five years.

    Our Series 1 and 2 of products include an input component supplied by Intel named the Myriad X. Many of you have asked about Intel’s plans to discontinue manufacturing Myriad X chips in 2025. We will be placing end-of-life purchases with Intel for the Myriad X chip in 2025 and this supply will support all expected future demand, large incremental opportunities, and any warranty obligations over the next five years.

    Luxonis has emerged as the leading independent American provider that is solely focused on 3D stereo cameras and edge AI. We have pioneered on device AI, we have the most advanced sensor stack (48MP, IMUs, IR, microphones, thermal, ToF, etc.), and we have a robust software ecosystem including cloud, applications, and models. We have recently had breakthroughs in our depth quality with our recently produced units now leading the industry in depth accuracy (competitor benchmark here). Lastly, we have IP67 rated cameras including options with both USB and POE connections.

    Our team of 80 engineers exclusively works only on developing and improving our platform everyday.

    If you’d like to discuss this topic with us further please drop me an email at bradley@luxonis.com

    Thanks for reading!

  • Good question, you can expect RVC4 units to be priced at approximately +$200 versus RVC2.

  • In this update we are going to share the latest and greatest news across our hardware and software offering.

    Let’s jump right in!

    OAK Series 4

    The first update is with our Series 4 of hardware. We are incredibly excited about this new generation that will include:

    • Octa-core ARM CPU running Linux (Kernel 5.15)
    • AI: 48 INT8, 12 FP16 TOPS
    • Computer vision: Stereo depth, frame warp engine, optical flow, feature detection, description matching, template matching
    • ISP: 5 camera streams, HDR, EIS, 3A, up to 3x 8K @ 30FPS
    • Encoding: 4K @ 240FPS decoding, 4K @ 120FPS encoding for H264 and H265. Decoding also supported for VP9, AV1

    We are developing four new standard devices for Series 4 - Depth, Single Camera, Short Range, and Long Range - and with variations we will be introducing 24 new SKUs all together. You can learn more about all of the Series 4 devices here. Our primary focus has been on developing depth-sensing and single-camera devices. We’ve successfully completed the production of our first batch, exceeding 100 units, and are now shipping early Private Access units to select customers.

    If you are interested in getting your hands on one of the first Series 4 OAKs via Private Access please sign up here.

    Now that we’ve completed production of our initial Series 4 units, we have been preparing for larger scale mass production. We have completed procurement for all of the necessary components and plan to kick off mass production by October. Once we go into mass production we will open up Series 4 for sale to all customers and plan to begin shipping orders to customers by December.

    If you want to schedule a call with us to learn more about Series 4, please book a time that works best for you here.

    Cyber Security Compliance

    We have learned from our customers that keeping data secure is a critical element for trusting and scaling deployments with Luxonis. Based on this feedback, Luxonis has started the process with Vanta to obtain our compliance certification with the System and Organization Controls (SOC) framework. Specifically, Luxonis is pursuing a SOC2 Type 2 and will have it completed by early 2025.

    The process to obtain our SOC2 Type 2 involves a third party audit that evaluates our internal controls over a period of time. The audit will check the security, availability, and confidentiality of Luxonis systems including controls, policies, and external dependencies with vendors such as cloud providers.

    Luxonis looks forward to completing our audit and will share the results with you once they are completed.

    Luxonis Hub Applications

    Luxonis Hub is our cloud and application layer that allows customers to remotely manage fleets of OAKs helping you to develop, deploy, and monitor your perception applications. Applications in Hub are publicly available and they are open source so that you can make your own proprietary modifications to them. Luxonis is excited to share details with you on new applications that we plan to launch over the coming months.

    Here is a list of all of the applications that we are looking at developing:

    1. Counter (car, people, bicycle) over a line
    2. Crowd counting
    3. Checking for Personal Protective Equipment (PPE) such as hard hats, safety vests, and safety glasses
    4. Box measurement for determining the x, y, and z dimensions of cardboard boxes
    5. QR and Barcode scanner
    6. Fire and smoke detection
    7. VIO / SLAM
    8. Optical Character Recognition (OCR)
    9. Object Detection
    10. Machine Guarding
    11. Multi-person body pose estimation
    12. Multi-camera multi-person tracking with reidentification
    13. Gaze estimation for digital ads
    14. People analytics such as age, gender, and emotion
    15. Multi-camera point cloud fusion

    Please reach out to us if you want to learn more about any of these applications.

    OAK Thermal

    The OAK-T is our first device with a thermal sensor and a color image sensor. Using a Tiny1-C with 256x192 resolution, the OAK-T works well in a variety of applications in the agricultural, industrial, transportation, and security sectors. We are currently producing our next batch of 250pcs of the OAK-T and those units will be available to ship by October.

    The update that we want to share on the OAK-T is that due to export controls Luxonis is unable to offer this product for sale in all the countries where we normally do business. Over the last 4+ years of shipping hardware we have delivered units to more than 110 countries. However, sometimes there are restrictions that prevent us from selling into certain places. With the OAK-T we are limited to countries that are NATO plus which consists of all NATO countries along with Australia, New Zealand, Japan, Israel, and South Korea.

    Time of Flight (ToF)

    We are pleased to share that we have shipped out mass produced units of our OAK-D SR POE. This device is specifically designed for short range applications and includes a ToF sensor. We are now in stock with this product and look forward to hearing any feedback that you may have on the product.

    That’s it, thanks for reading our updates!

  • Luxonis recently celebrated our 5th birthday of being in business and we’ve never been more optimistic about what the next 5 years will bring. In this blog we’ll review our founding story, major milestones, growth, our product evolution, and our future plans. 

    Founding Story 

    Our journey started back in 2019 when we were inspired to save lives by making bike commuting safe. Sadly, the lives of countless cyclists are lost, and many more are injured due to distracted drivers colliding with them. We set out to solve this problem by leveraging computer vision and machine learning to be able to automatically detect when a driver is on a collision course with a cyclist. Check out the original rendering for our first product idea named the Commute Guardian.    

    When we set out to create the Commute Guardian, we discovered that creating it with off the shelf components was extremely difficult. We needed to combine together multiple disparate pieces of hardware such as a stereo camera, processor, and AI accelerator. Finding the hardware was actually the easy part, the hard part was figuring out how to get the firmware to work. 

    Milestones 

    Discovering these challenges, we figured that we weren’t the only ones that were finding it extremely time consuming and expensive to put together a single computer vision system. This key insight led to the launch of our first products which were the OAK-D and OAK-1 that we introduced with a Kickstarter campaign. The OAK lineup delivered on five key elements that our customers required:

    • Embedded:  Small size, low weight, low power, fast boot 

    • Performant:  High resolution, high frame rate, low latency 

    • Spatial:  Disparity Depth 

    • AI:  Neural inference, object detection, semantic segmentation

    • CV:  Feature extraction, motion estimation, edge detection, optical flow, WARP and de-WARP

    With $1.4M raised across more than six thousand backers, we broke Kickstarter records for our product category.

    We quickly followed the success of our OAK-D Kickstarter campaign with an even more affordable OAK-1 Lite and OAK-D Lite which Luxonis continues to offer at an entry price of $99 and $149, respectively. We launched our Lite versions with a second Kickstarter campaign and our customers were delighted with our efforts to democratize access to AI computer vision devices.  You can see a timeline of our key milestones below:  

    Growth 

    With the launch of OAK-D and OAK-D Lite, Luxonis began scaling up our manufacturing and global distribution efforts. We have shipped products into over 110 countries around the world and now hold inventory locally in our key markets which include North America, Europe, and Hong Kong for serving the APAC region. 

    To support our customers' growing needs, Luxonis also expanded our global team which now includes 80 engineers. Luxonis performs all engineering disciplines in house and these include: 

    • Electrical

    • Mechanical

    • Industrial Design 

    • Firmware

    • Software 

    • Computer Vision 

    • Application development 

    • Machine Learning 

    • Simulation 

    With over 1 million SDK downloads, Luxonis now serves more than 25,000 customers around the world. We pride ourselves with our customer support and that includes our extensive documentation and our technical forum. Drawing from these resources, Luxonis recently introduced a ChatGPT-4 powered technical support chatbot that is now fielding hundreds of queries each week. 

    You can learn more about examples of the various industrial applications that Luxonis works on here.  

    Product Evolution 

    We are focused on executing our product strategy which is highlighted by three key elements: 

    1. Create a new generation of hardware that is 10x better every 2 years. Luxonis will be achieving this with our upcoming launch of Series 4  later in 2024. The pace of progress in edge AI hardware is remarkable and Luxonis will continue to be on the bleeding edge with Series 5 planned in 2026. 

    2. Continually add new and improved sensing capabilities to our hardware. Luxonis recently introduced our first Time of Flight device that is available now for purchase and we are in the process of launching our first Thermal device that is available for pre-order now. We also continue to make improvements to our optics by offering a 32MP camera and we plan to introduce a 108MP option in our Series 4 lineup.  

    3. Constantly move up the technology stack to make Luxonis a one-stop shop for robotic perception and AI automation. We have been scaling up our cloud and application Hub with a growing list of open-source, publicly available, and free applications including a Counter, QR code reader, and Object Detection.   

    Our product philosophy is to both never end-of-life our products and to cannibalize our existing product with an always improving product offering. Our view is that if we don’t come out with new products that beat our existing ones then somebody else will. 

    Looking Ahead 

    At Luxonis, our ultimate goal is to enable systems that exceed humans at every possible task. Providing, at a minimum, parity with human-level perception means that robotic and automation systems are able to reliably assist, if not outright replace, various tasks that humans are performing today. We are inspired by achieving this goal and believe that the benefits to society will be massive. The robotic and automation systems that we enable help to save lives, eliminate grueling physical labor, and remove the need for humans to perform mundane repetitive tasks. 

    Thank you to all of our stakeholders who have been along for the amazing ride over the last 5 years. We are incredibly grateful for all of the support and are extremely excited about what the next 5 years will bring. 

    You can get in touch with me by emailing me at [bradley@luxonis.com](mailto:bradley@luxonis.com). 

  • It’s incredible to see the progress that Luxonis has made since starting to work on our cloud & application layer - named Luxonis Hub - two years ago. A team of 20+ dedicated developers have been making improvements each week as we race towards Luxonis Hub becoming a robust production ready solution for small, mid-market, and enterprise businesses. Luxonis Hub enables customers to manage fleets of devices with key functionality such as automatically saving detections, providing OTA updates, tracking performance analytics, and deploying updated applications. We have a number of exciting updates that we’d like to share with you.

    First, we have been working to expand our list of publicly available applications. These Luxonis Hub applications are open source which means that you are free to modify them as needed for your own production needs. If you navigate to device then apps then install app you’ll see existing apps such as Counter (https://discuss.luxonis.com/blog/2315-the-power-of-Luxonis) and new ones such as Youtube Streaming and Consumer Demographic & Sentiment Analysis (CDSA). The CDSA app, which we showcased recently at the Consumer Electronics Show in Las Vegas, lets users track the age, gender, and emotion of crowds of humans; this can be very useful in industries such as retail. Luxonis is working to add a new app every month and next up you will see us add a bar and QR code scanning app. If you would like the opportunity to contribute to the direction of our app ecosystem please head to our innovation lab https://luxonis.com/innovation

    Second, alongside a growing list of publicly available apps, Luxonis is also enabling its users to create their own proprietary apps in Luxonis Hub. Users can navigate to Apps and then Create App to develop their own from one of the available templates. And you can get the support that you need by accessing our new documentation page here:  https://docs-beta.luxonis.com/. And exciting news that starting next month Luxonis will be announcing an app development contest with a cash prize for the best new Luxonis Hub app. Stay tuned for more details!

    Third, we’ve updated the Luxonis Hub business model to be as customer obsessed as possible. Users will continue to be able to access Luxonis Hub on a free forever plan that includes up to 2 GB storage for detections and apps. We have also introduced three standard paid tiers that range from 50 GB and 50 devices up to 1 TB and 1,000 devices. Additionally, please contact us at sales@luxonis.com to discuss our Enterprise plan that will be customized for your scaled use case.

    A fantastic improvement is that all plans include an unlimited number of users that you can add to your account. This means that you won’t be forced into a higher priced plan just so that you can add another user. You can learn more about Luxonis Hub account options  here.

    Finally, we are pleased to share that we have a number of businesses using Luxonis Hub in production environments and are fast approaching 1,000 connected OAKs. These customers are finding significant business value from being able to set up automatic recordings (e.g. if a specified object is detected then record 30 seconds of video) and being able to remotely deploy new apps. The Luxonis team also offers custom app development services if you are in need of an app that you don’t see is available yet.

    If you’d like to schedule a Luxonis Hub demo with Luxonis then please request a demo here

    You can sign up for Luxonis Hub here:  https://hub.luxonis.com/

  • Evolution gave humans, and most all other mammals, two eyes. Human eyes are typically spaced 6 to 7 centimeters (2.4 to 2.8 inches) apart. Remarkably, a substantial portion of the human brain is dedicated to processing visual information, with some studies suggesting a significant investment of the brain's resources in this area (source: University of Rochester). Our brain is able to perceive both how close an object is from us and where that object is heading. When Luxonis set out to create an edge artificial intelligence (AI) depth camera, we used learnings from human eye sight to design our products. This is why the stereo baselines of our cameras are typically 7.5 centimeters (3 inches) apart and a sizable portion of our onboard processing is dedicated towards fusing the left and right cameras together to create a three dimensional map. 

    At Luxonis, we are on a long term mission of empowering companies to surpass human perception in every task. To achieve that mission, it requires the accuracy of the depth map to be superior to humans. Fortunately, we have already long since exceeded this because human beings are actually quite poor at estimating distances. If you look at your hands do you really know if they are currently 10, 12, or 14 inches away from your head? There is a reason we all have a tape measure (or laser or smartphone) at home to help us determine distances. 

    We have recently been working diligently to improve the accuracy of our depth maps. And we are now proud to announce that starting in March 2024, stereo cameras shipped from Luxonis will be implemented with a next generation calibration procedure that delivers much better accuracy than our old generation. Please note that fortunately the performance is not hardware limited so existing devices can be recalibrated. 

    Here are the real world (not theoretical) depth accuracy measurements that you’ll be experiencing with our products come March: 

    • 0.7m - 4m (2’4” to 13’1”): below 2% absolute depth error

    • 4m - 7m (13’1” to 23’): below 4% absolute depth error

    • 7m - 12m (23’ to 39’4”): below 6% absolute depth error

    You can learn more about our improved performance here:

    https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/depth_accuracy/#p-75mm-baseline-distance-oaks

    If you’d like to get your hands on an OAK with improved depth sooner please reach out to us at [support@luxonis.com](mailto:support@luxonis.com).

    We'll be releasing competitive benchmark data on depth performance by the end of March.

    • The error message you're seeing, RuntimeError: No available devices, indicates that the DepthAI library is unable to find your OAK D Lite device. Here are a few steps you can take to troubleshoot this issue:

      1. Check your USB connection: Make sure your OAK D Lite is properly connected to your computer via a USB3 cable. If you're using a USB2 cable and want USB2 bandwidth, you need to specify the USB2 protocol. A bad USB cable can often cause connection issues.

      2. Discover connected devices: You can use the DepthAI API to discover all connected OAK cameras. Here's a code snippet that finds all OAK cameras and prints their MxIDs (unique identifier) and their XLink state:

      $$
      import depthai
      for device in depthai.Device.getAllAvailableDevices():
      print(f"{device.getMxId()} {device.state}")
      $$

      1. Specify the device: If you have multiple OAK devices connected, you may need to specify which device you want to use in your code. You can do this by providing the MxID, IP address, or USB path to the depthai.Device constructor:

      $$
      device_info = depthai.DeviceInfo("14442C108144F1D000") # MXID
      with depthai.Device(pipeline, device_info) as device:

      ...

      $$

      1. Check for device recognition: If you still can't connect to the OAK camera, you can check if your system recognizes the device by executing lsusb | grep 03e7 in your terminal. You should see a line similar to Bus 001 Device 120: ID 03e7:2485 Intel Movidius MyriadX.

      If none of these steps resolve your issue, please provide additional information such as the name of your OAK camera, the version of the DepthAI library and bootloader you're using, and any crash logs or error messages you're seeing.

      Sources:

    • To deploy your custom grayscale Yolo8 model on the OAK device, you need to convert it into a MyriadX blob format. This process involves two steps:

      1. Convert the model into OpenVINO's Intermediate Representation (IR) format using the Model Optimizer.

      2. Compile the IR format into a blob file using the Model Compiler.

      However, since you're encountering an error while converting the model to ONNX, it's possible that there's an unsupported layer or connection between two layers in your model. You can use the Netron app to visualize your model and identify any unsupported layers or connections.

      If you're still encountering issues, you can try using the Blobconverter tool, which allows you to convert and compile the model from various formats, including TensorFlow, Caffe, ONNX, and OpenVINO IR.

      Once you have the blob file, you can deploy it onto the MYRIAD-X processor within an OAK device for inference.

      Sources:

    • To switch on/off the OAK-1 and OAK-D IoT 40 cameras to conserve power, you can use an external device like a microcontroller or a Single Board Computer (SBC) such as a Raspberry Pi. You can control the power supply to the cameras using the GPIO pins of the microcontroller or SBC. However, please note that the cameras do not have a built-in sleep mode or power-saving mode. They will start running as soon as they are powered on and will stop when the power is cut off.

      As for customizing the timer remotely, you can use a device with network capabilities like a Raspberry Pi or an ESP32 board. You can write a program that runs on this device to control the cameras based on a timer. This program can be updated remotely if the device is connected to the internet. However, please note that the OAK cameras themselves do not have built-in network capabilities. They can only communicate with the device they are directly connected to.

      For programming the cameras, you can use the DepthAI API which supports Python and C++. You can write a program that runs on the host device (like a Raspberry Pi or ESP32) to control the cameras. This program can be updated remotely if the host device is connected to the internet.

    • The IP65-rated OAK-D Pro W PoE camera is designed to withstand harsh environments, including dust and water jets from any direction. However, it's not rated for immersion in water or exposure to corrosive substances like seawater or harsh cleaning solutions.

      The IP code of a device tells you whether it's dustproof and/or waterproof. An IP65 rating means the device is dust-tight and protected against water projected from a nozzle. It does not guarantee protection against immersion or corrosive substances.

      If you need a camera that can withstand immersion in water, you might want to consider the IP67-rated PoE devices like the OAK-D-POE and OAK-1-POE. An IP67 rating means the device is dust-tight and can withstand water immersion between 15 centimeters and 1 meter in depth.

      However, please note that even with an IP67 rating, the device may not be fully protected against corrosive substances like seawater or harsh cleaning solutions.

      Also, when installing the camera, make sure that the GORE vent isn't blocked, as it can lead to moisture fogging up in front of the cameras (on the inside of the front cover).

      TLDR: The only way to know is to actually test it out which is what we'd recommend that you try doing and we'd love to hear feedback on how well it holds up.

    • Yes, the connectors are indeed M series connectors from Amphenol. The M8 aux connector is an 8pin female A-coded connector, and the M12 ethernet is an 8pin female X-coded connector. The M12 plug does indeed go to a PoE cable, making this camera similar to existing PoE models. The M8 socket is used for auxiliary connections. If the M8 connector is not used, the cameras include an M8 connector cap for waterproofing.

      To connect the camera to your computer, you will need a PoE switch or injector to power the device. After powering the device, you should connect your computer to the same Local Area Network (LAN) as the PoE device. If you are using a DHCP server, you can check the connected devices on the DHCP server dashboard to see if the camera is connected and its IP. If there is no DHCP, the camera will fallback to a static IP. Once you know the IP of the camera and can ping it, you can run any code sample or depthai experiment as you would when connecting an OAK device with a USB-C cable.

    • We are developing our Series 4 version of the Thermal camera and that new version will come with the option to connect via POE or USB.

      The connectors on the back are M12 and M8.