I have seen [this](https://github.com/luxonis/depthai/issues/245) really cool example. However, it only considers the RGB data from the camera. I've been trying to work out how to improve this using the depth data as well to mitigate lighting changes but I'm not sure where to start. I've seen [this](https://github.com/luxonis/depthai/issues/246) open issue that looks like it would do exactly what I want. I'm struggling to work out the logic required to know when to ignore anomalies in the depth data, and when to use depth data to discount lighting changes etc. Apologies if this is the wrong forum for this question.

Hi @"MatthewGale"#p6163 , Dot projector won't improve depth noise per se, more the fill-rate. For noise, you should look into postprocessing filters ([ondevice](https://docs.luxonis.com/projects/api/en/latest/samples/StereoDepth/depth_post_processing/#depth-post-processing), or perhaps on the host, [like WLS](https://github.com/luxonis/depthai-experiments/tree/master/gen2-wls-filter)). And for dot projector, it depends on the environment, but in bright env you want to have it at 800mA to have an affect at all (otherwise mono cameras can't see the dots). Thanks, Erik

Hi @"erik"#p6167, That is really helpful thank you.

How complicated would adding depth to motion estimation be?

MatthewGale

I have seen this really cool example. However, it only considers the RGB data from the camera. I've been trying to work out how to improve this using the depth data as well to mitigate lighting changes but I'm not sure where to start. I've seen this open issue that looks like it would do exactly what I want. I'm struggling to work out the logic required to know when to ignore anomalies in the depth data, and when to use depth data to discount lighting changes etc. Apologies if this is the wrong forum for this question.

erik

Hi MatthewGale ,
I think it would be best to also show your interest in that issue, just so we can track customer interest of features (to add). What would be the final goal of motion estimation background removal - to mask the background, or to eg. digitally zoom (crop out) to the moving object?
Thanks, Erik

MatthewGale

Hi erik,

I'm looking to see whether change detection is more accurate when using a 3D camera. I'm using change detection to track a moving object.

Thanks,

Mattie.

erik

Hi MatthewGale ,
Interesting, I am not sure what would be the best way to include the depth information to "change detection" model. You could check the change just like for RGB image, but due to noise, you'd need to add quite some filtering/thresholding, but it is possible - as there are companies doing eg. people counting with just depth information. Thoughts?
Thanks, Erik

MatthewGale

Hi erik,

Yes, it's quite tricky to get the thresholding correct as I'm getting a lot of false detections due to noise in the depth data. I've got the pro oak d camera, and have set the dot projector output to 200mA, is that just automatically used to improve the depth information or do I need to signify elsewhere in the code to use the dot projector?

Thanks,

Mattie.

erik

Hi MatthewGale ,
Dot projector won't improve depth noise per se, more the fill-rate. For noise, you should look into postprocessing filters (ondevice, or perhaps on the host, like WLS).
And for dot projector, it depends on the environment, but in bright env you want to have it at 800mA to have an affect at all (otherwise mono cameras can't see the dots).
Thanks, Erik

MatthewGale

Hi erik,

That is really helpful thank you.