Rrbn23
- Dec 18, 2024
- Joined Jun 26, 2024
- 2 best answers
Alright,
I will try that. Thank you
Hi Luxonis,
I have a system with 6 OAK-D Pro W PoE cameras. They are all connected to a PoE 1GBit switch and should run with 10 fps, 400P. I stream the point cloud as well as left image from the cameras. In this mode each camera requires ~ 200-300MBit/s and my network gets totally overloaded with latency of 2-3sec.
I only need two of the six cameras at a time and would like to use something like the .setStartStreaming() and .setStopStreaming() but as I read in the forum this is not functional. Can I change the fps or anything to bring down the network load of the 4 cameras not in use?
any progress so far?
I tested with manual exposure and your settings. here an example of (10, 300). There is no noise. When I enable autoExposure I get the same noisy results as before.
Do you have customer service? I'll need that camera for a project and it is not working right. From you there is no answer for days, so how to continue with that problem?
Suggestion from my side: I'll send you the camera and you can find the problems yourself. You will send me a new one.
- I use python package 2.26.0, that was the latest I could install using pip
- The problem happens right after camera boot. The camera is not warm at all. There is no NN or anything else running on the camera except grabbing the two images.
- I have 4 cameras of the same type. I cannot reproduce the problem with any of the other 3 cameras. When running the same script.
- Here is the calibration from the camera:
User calibration: {
"batchName": "",
"batchTime": 1708949643,
"boardConf": "IR-C00M00-00",
"boardCustom": "",
"boardName": "NG9097",
"boardOptions": 7,
"boardRev": "R4M2E4",
"cameraData": [
[ 1, { "cameraType": 0, "distortionCoeff": [ 3\.5571253299713135, 1\.0675731897354126, 0\.00011813067976618186, -4.179932602710323e-06, 0\.019934868440032005, 3\.9141414165496826, 2\.124300003051758, 0\.15198968350887299, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0 ], "extrinsics": { "rotationMatrix": [ [ 0\.9998213648796082, 0\.008929289877414703, -0.01665874384343624 ], [ -0.009426653385162354, 0\.9995048642158508, -0.03002036362886429 ], [ 0\.016382433474063873, 0\.030172036960721016, 0\.9994104504585266 ] ], "specTranslation": { "x": -7.5, "y": 0.0, "z": 0.0 }, "toCameraSocket": 2, "translation": { "x": -7.511865139007568, "y": 0.006338753271847963, "z": 0.008916680701076984 } }, "height": 800, "intrinsicMatrix": [ [ 563\.3818359375, 0\.0, 620\.244384765625 ], [ 0\.0, 563\.4485473632812, 401\.442138671875 ], [ 0\.0, 0\.0, 1\.0 ] ], "lensPosition": 0, "specHfovDeg": 127.0, "width": 1280 } ], [ 0, { "cameraType": 0, "distortionCoeff": [ 12\.214872360229492, 11\.166379928588867, 0\.0009387647733092308, -0.0005655140266753733, -1.9312169551849365, 12\.175280570983887, 14\.985671043395996, -1.223604440689087, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0 ], "extrinsics": { "rotationMatrix": [ [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ] ], "specTranslation": { "x": -0.0, "y": -0.0, "z": -0.0 }, "toCameraSocket": -1, "translation": { "x": 0.0, "y": 0.0, "z": 0.0 } }, "height": 2160, "intrinsicMatrix": [ [ 2311\.1796875, 0\.0, 1912\.238525390625 ], [ 0\.0, 2309\.768798828125, 1092\.1376953125 ], [ 0\.0, 0\.0, 1\.0 ] ], "lensPosition": 0, "specHfovDeg": 108.0, "width": 3840 } ], [ 2, { "cameraType": 0, "distortionCoeff": [ 5\.346960544586182, 1\.4969804286956787, -0.0008067575399763882, 6\.0376351029844955e-05, 0\.028692806139588356, 5\.729836940765381, 3\.11305832862854, 0\.20841671526432037, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0 ], "extrinsics": { "rotationMatrix": [ [ 0\.9998905658721924, -0.013739221729338169, 0\.005482451058924198 ], [ 0\.013782133348286152, 0\.9998740553855896, -0.007867513224482536 ], [ -0.005373666994273663, 0\.00794221181422472, 0\.9999540448188782 ] ], "specTranslation": { "x": 3.75, "y": 0.0, "z": 0.0 }, "toCameraSocket": 0, "translation": { "x": 3.768968343734741, "y": 0.059201680123806, "z": -0.07226227968931198 } }, "height": 800, "intrinsicMatrix": [ [ 574\.7843627929688, 0\.0, 623\.1576538085938 ], [ 0\.0, 574\.4819946289062, 429\.1021423339844 ], [ 0\.0, 0\.0, 1\.0 ] ], "lensPosition": 0, "specHfovDeg": 127.0, "width": 1280 } ]],
"deviceName": "",
"hardwareConf": "F1-FV01-BC001",
"housingExtrinsics": {
"rotationMatrix": [], "specTranslation": { "x": 0.0, "y": 0.0, "z": 0.0 }, "toCameraSocket": -1, "translation": { "x": 0.0, "y": 0.0, "z": 0.0 }},
"imuExtrinsics": {
"rotationMatrix": [ [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ] ], "specTranslation": { "x": 0.0, "y": 0.0, "z": 0.0 }, "toCameraSocket": -1, "translation": { "x": 0.0, "y": 0.0, "z": 0.0 }},
"miscellaneousData": [],
"productName": "OAK-D-PRO-W-POE",
"stereoEnableDistortionCorrection": true,
"stereoRectificationData": {
"leftCameraSocket": 1, "rectifiedRotationLeft": [ [ 0\.9998087882995605, 0\.00805005244910717, -0.017819706350564957 ], [ -0.008317464031279087, 0\.9998531341552734, -0.0149836465716362 ], [ 0\.017696470022201538, 0\.015128996223211288, 0\.9997289180755615 ] ], "rectifiedRotationRight": [ [ 0\.999998927116394, -0.0008438312215730548, -0.0011870115995407104 ], [ 0\.0008616084815002978, 0\.9998862743377686, 0\.015056533738970757 ], [ 0\.001174171338789165, -0.015057540498673916, 0\.9998859167098999 ] ], "rightCameraSocket": 2},
"stereoUseSpecTranslation": false,
"version": 7,
"verticalCameraSocket": -1
}
Factory calibration: {
"batchName": "",
"batchTime": 1708949643,
"boardConf": "IR-C00M00-00",
"boardCustom": "",
"boardName": "NG9097",
"boardOptions": 7,
"boardRev": "R4M2E4",
"cameraData": [
[ 1, { "cameraType": 0, "distortionCoeff": [ 3\.5571253299713135, 1\.0675731897354126, 0\.00011813067976618186, -4.179932602710323e-06, 0\.019934868440032005, 3\.9141414165496826, 2\.124300003051758, 0\.15198968350887299, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0 ], "extrinsics": { "rotationMatrix": [ [ 0\.9998213648796082, 0\.008929289877414703, -0.01665874384343624 ], [ -0.009426653385162354, 0\.9995048642158508, -0.03002036362886429 ], [ 0\.016382433474063873, 0\.030172036960721016, 0\.9994104504585266 ] ], "specTranslation": { "x": -7.5, "y": 0.0, "z": 0.0 }, "toCameraSocket": 2, "translation": { "x": -7.511865139007568, "y": 0.006338753271847963, "z": 0.008916680701076984 } }, "height": 800, "intrinsicMatrix": [ [ 563\.3818359375, 0\.0, 620\.244384765625 ], [ 0\.0, 563\.4485473632812, 401\.442138671875 ], [ 0\.0, 0\.0, 1\.0 ] ], "lensPosition": 0, "specHfovDeg": 127.0, "width": 1280 } ], [ 0, { "cameraType": 0, "distortionCoeff": [ 12\.214872360229492, 11\.166379928588867, 0\.0009387647733092308, -0.0005655140266753733, -1.9312169551849365, 12\.175280570983887, 14\.985671043395996, -1.223604440689087, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0 ], "extrinsics": { "rotationMatrix": [ [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ] ], "specTranslation": { "x": -0.0, "y": -0.0, "z": -0.0 }, "toCameraSocket": -1, "translation": { "x": 0.0, "y": 0.0, "z": 0.0 } }, "height": 2160, "intrinsicMatrix": [ [ 2311\.1796875, 0\.0, 1912\.238525390625 ], [ 0\.0, 2309\.768798828125, 1092\.1376953125 ], [ 0\.0, 0\.0, 1\.0 ] ], "lensPosition": 0, "specHfovDeg": 108.0, "width": 3840 } ], [ 2, { "cameraType": 0, "distortionCoeff": [ 5\.346960544586182, 1\.4969804286956787, -0.0008067575399763882, 6\.0376351029844955e-05, 0\.028692806139588356, 5\.729836940765381, 3\.11305832862854, 0\.20841671526432037, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0, 0\.0 ], "extrinsics": { "rotationMatrix": [ [ 0\.9998905658721924, -0.013739221729338169, 0\.005482451058924198 ], [ 0\.013782133348286152, 0\.9998740553855896, -0.007867513224482536 ], [ -0.005373666994273663, 0\.00794221181422472, 0\.9999540448188782 ] ], "specTranslation": { "x": 3.75, "y": 0.0, "z": 0.0 }, "toCameraSocket": 0, "translation": { "x": 3.768968343734741, "y": 0.059201680123806, "z": -0.07226227968931198 } }, "height": 800, "intrinsicMatrix": [ [ 574\.7843627929688, 0\.0, 623\.1576538085938 ], [ 0\.0, 574\.4819946289062, 429\.1021423339844 ], [ 0\.0, 0\.0, 1\.0 ] ], "lensPosition": 0, "specHfovDeg": 127.0, "width": 1280 } ]],
"deviceName": "",
"hardwareConf": "F1-FV01-BC001",
"housingExtrinsics": {
"rotationMatrix": [], "specTranslation": { "x": 0.0, "y": 0.0, "z": 0.0 }, "toCameraSocket": -1, "translation": { "x": 0.0, "y": 0.0, "z": 0.0 }},
"imuExtrinsics": {
"rotationMatrix": [ [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ], [ 0\.0, 0\.0, 0\.0 ] ], "specTranslation": { "x": 0.0, "y": 0.0, "z": 0.0 }, "toCameraSocket": -1, "translation": { "x": 0.0, "y": 0.0, "z": 0.0 }},
"miscellaneousData": [],
"productName": "OAK-D-PRO-W-POE",
"stereoEnableDistortionCorrection": true,
"stereoRectificationData": {
"leftCameraSocket": 1, "rectifiedRotationLeft": [ [ 0\.9998087882995605, 0\.00805005244910717, -0.017819706350564957 ], [ -0.008317464031279087, 0\.9998531341552734, -0.0149836465716362 ], [ 0\.017696470022201538, 0\.015128996223211288, 0\.9997289180755615 ] ], "rectifiedRotationRight": [ [ 0\.999998927116394, -0.0008438312215730548, -0.0011870115995407104 ], [ 0\.0008616084815002978, 0\.9998862743377686, 0\.015056533738970757 ], [ 0\.001174171338789165, -0.015057540498673916, 0\.9998859167098999 ] ], "rightCameraSocket": 2},
"stereoUseSpecTranslation": false,
"version": 7,
"verticalCameraSocket": -1
}
Is there any update from your side?
- Edited
I set the camera right underneath the ceiling lights and got these results with manual exposure of 100us and iso 100:
left
right
right image is noisy again
Right now it's cloudy outside, so I cannot test and check the unrectified images. I will test when I get full sunlight again
- Edited
I am using dai.node.StereoDepth node. The quality of the depth image was very bad what made me wonder. When I looked at the rectified images I saw that the right one was noisy.
I followed the depthai examples that set coltrols for the mono cameras the following way:
controlIn.out.link(monoLeft.inputControl)
controlIn.out.link(monoRight.inputControl)Is this wrong for unsing stereo node? When Left camera sets the right settings, why send the control also to the right camera?
This still wouldn't explain why it only happens with one OAK module. I have 3 OAK-D Pro W and only one has this problem. The others look fine even in full sunlight, all running with the same code. Maybe the image sensor is somehow broken?
- Edited
I take each frame as opencv image from the queue and save it with cv::imwrite()
I have a second OAK-D-Pro W camera and with this one (same code) the images are not noisy. To verify if unrectified image is not noisy I need to take a test with strong sunlight, since the problem with the noisy image only happes in very bright scenes.
When I record left and right rectified images, my right image is very noisy. Here is an example:
Left
Right
I followed the samples for camera settings:
controlIn.out.link(monoLeft.inputControl)
controlIn.out.link(monoRight.inputControl)
controlQueue = device.getInputQueue(controlIn.getStreamName())
ctrl = dai.CameraControl()
ctrl.setAutoExposureEnable()
ctrl.setAutoExposureLimit(10000)
controlQueue.send(ctrl)Did I miss something?
I need to ask again:
As I know: depth = baseline * focalLength / disparity
My camera intrinsics say fx = 566.391, baseline = 75mm
When I apply that on a point with disparity = 16: depth = 2654.95mm
But when I read the depth image from depthai-pipeline, depth for exacly that point is 2681mm
That means the depthai-pipeline uses fx = 2681*16/75 = 571.947
When I try another point I get e.g. fx = 933*47/75 = 584.68
That means in the pipeline happen some more steps. Is this open source and I can see whats happening there?
ok, thanks. I will try like that
- Edited
Yes of course, but what if I estimate the disparity image myself? Can I feed that into the point cloud node?
I also saw that, when I crop the image mysef, I can give that information into the "getCameraIntrinsics" and I will receive the intrinsics for the cropped image. Is that correct?
-> topLeftPixelId and bottomRightPixelId is what i need
std::vector<std::vector<float>> getCameraIntrinsics(CameraBoardSocket cameraId,int resizeWidth = -1, int resizeHeight = -1,Point2f topLeftPixelId = Point2f(),Point2f bottomRightPixelId = Point2f(), bool keepAspectRatio = true) const;Hi Devs,
I am testing disparity estimateion with another framework and for that I receive recitified images from my OAK-D Pro W camera. My own disparity calculation looks nice but when I use camera intrinsics to calculate a point cloud from that, the point cloud looks distorted. I assume this is because the rectified images are zoomed and shifted.
Stereo Depth Video (luxonis.com)
Is it possible to get the values of that zoom and shift to add to my point cloud calculation or do I need to continue with uncropped images with alpha = 1.0? But this will make it even harder for me, because I need to do the crop by myself, with taking care of alignment of the stereo pair. Is there an easy way to solve tis?
Thank you for your help
OK,
thank you!
Hi,
so with subpixel and LR check I get 96 with no other process on the camera.
My question is: will ge() with blocking=false in the queue disrupt anything in the pipeline on the camera or can I call it whenever I want and get the actual frame no matter when it is called?
Thanks