The Background
Luxonis is customer obsessed. So when customers come to us with problems that are important to them, we listen. And often we can’t let go. So when customers came to us with advice on how to make their robots work outdoors, and questions about why their depth cameras don’t work outdoors, we listened. Even though the cameras being used were not made by us. In fact, these cameras were made by the incumbent in the market, RealSense, and the customer was already phasing them out. As noted here and shown below, the flagship of the RealSense depth camera line, the D455, was incapable of producing useful imagery outdoors.
This is a HUGE problem, as nearly every robotic system makes use of color data. And in agriculture and many industries, this is a TOTAL showstopper. So what did we do?
Well, we’re customer obsessed, so we fixed it for the customer. We made a version of our OAK-D with the required OV9782 global shutter color, and we fixed the problem. How? We’ll get there. But below is a quick before/after:
The before ^. Look at that. Classic computer vision image sensor tuning problem. It’s called an improper “Lens Shading Correction” (LSC) that results in colors that aren’t in the scene, but a result of the image sensor, lens, and lens holder interacting and effectively “making up” colors that don’t exist.
With proper LSC the image then shows more correct colors:
Before we get into the details of the how, let’s take a quick step back: All of our OAK-D S2 and OAK-D PoE S2 models come standard with the IMX378 as the center camera, but because we want to offer customers the greatest degree of flexibility we give them the option to replace this camera with the OV9782 to increase FOV from 120° to 150°. The one drawback with making this switch–until now–is that discoloration could occur when capturing video outside.
Ultimately, this issue relates to an oversaturation problem caused by exposure of the OV9782 to bright light. In these conditions, pink or violet hues can creep into the image.
Intel RealSense D455 - Pink Color Issue
We’re not the only ones dealing with this problem, we’re just the only ones who solved it (more on that in a second). Intel is the closest thing we have to a competitor on the market, and their D455 camera–which uses the same OV9782 that our OAK Series does–struggles with outdoor coloration creep too. And Intel still hasn’t found a solution.
A few minutes of Googling reveals the depth of their customers’ concerns, both in the form of direct support forums and industry news. Here are a few examples:
Intel may have the luxury to just let this one slide, but that’s not how we operate at Luxonis.
We even won a fantastic new customer who got tired of waiting on Intel to act. Greenzie decided to move away from the D455 for many reasons, but this coloration issue being one of them. Be sure to check them out for all your commercial autonomous lawn moving needs!
Moving Forward - the Fix & the Future
Time to get down to it. How did we fix a problem that a beast like Intel couldn't (or wouldn’t)? To put it simply, we’re a team that doesn’t quit. We knew that proper color representation is essential for our customers, so worked the problem until discovering that a specialized implementation of Lens Shading Correction (LSC) made a huge improvement. Think of it like putting a pair of sunglasses on the camera lens that allows it to maintain a more consistent picture.
We’ve seen the before/after images, but now let’s take a look at a video of a test in action! Even with very short shutter rates in high light environments, we can see how the presence of invasive coloring is significantly reduced.
Want to know what’s even better? All of our wide FOV cameras–current and future models–now offer this enhanced functionality, while still allowing the option to choose between the IMX378 (120° DFOV 12MP Rolling Shutter Color) and OV9782 (150° DFOV 1MP Global Shutter Color) for the center camera.
Be sure to take a look at our full range of Wide FOV camera options to find the one that’s right for you:
OAK-D-W - Wide FOV, USB powered
OAK-D-Pro-W - Wide FOV with night-vision and active stereo vision, USB powered
OAK-D-W-PoE - Wide FOV, powered over ethernet
OAK-D-Pro-W-PoE - Wide FOV with night-vision and active stereo vision, powered over ethernet
Applying the fix
To fix the issue above, please follow the Camera tuning tutorial, and use the OV9782 Wide FOV color tuning for sunlight environments tuning blob.
