OAK-D Pro synchronisation with another camera system (not luxonis)

fbt_

Hello. I have the OAK-D Pro (https://shop.luxonis.com/products/oak-d-pro?variant=42455252369631) and im trying to synch it with another Camera System from a different Company. I tried to figure it out myself but i cant help to find the informations I need.

Im working with C++ and currently i set the cam up like this:

dai::Pipeline pipeline;
auto camRgb = pipeline.create<dai::node::ColorCamera>();
auto xoutRgb = pipeline.create<dai::node::XLinkOut>();
camRgb->setBoardSocket(dai::CameraBoardSocket::RGB);
camRgb->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1080_P);
camRgb->setFps(60);
xoutRgb->setStreamName("rgb");
camRgb->video.link(xoutRgb->input);
device.setTimesync(true);
auto qRgb = device.getOutputQueue("rgb", 60, false);

I'm getting the frames like this:

#pragma omp parallel for
                    for(int c_id = 0; c_id < N_CAMS+1; c_id++){
                        if (i < per_cam_images[c_id].size()) {
                            if(c_id < N_CAMS){
                                //capturing logic of the other cam system
                            } else {
                                while (auto inRgb = qRgb->tryGet<dai::ImgFrame>()){
                                    if(inRgb){
                                        webcamImages[webCamCount] = std::make_shared<dai::ImgFrame>(*inRgb);
                                    }
                                }
                            }
                            
                        }
                    }

Its a parallel call where i trigger the cameras from the other system which is synched (so i dont have to worry about the other system) and also get all the frames currently in the queue of the luxonis camera. I match the results via timestamps but the synchronization to the Luxonis Frames is offset by an average of 25ms (which is a problem because I have to achieve a synchronization <1ms ).
I was wondering if there is a way to trigger the Camera manually because as far as i got this right it currently just starts capturing after this call dai::Device device(pipeline); and then i can get the frames it has in the queue which are async to the other system.
Any Ideas?

jakaskerl

fbt_
For sub ms, your only bet is hardware synchronization which is impossible to achieve without opening up the device. The D-Pro does not have fsync or strobe signals easily exposed.

If you can, i'd suggest you set the other camera to follow the timestamps of OAK-D in order to sync them.

Thanks,
Jaka

fbt_

Hi, thanks for your reply, I would prefer to not open the device. Could you maybe explain the last part a bit further? I can set a trigger signal for the other system, is there a way to get the trigger of the luxonis device? I saw that you can add a callback on the queue for a received message and get the exposure start timestamp of the Message, but I’m not sure how I can use this timestamp for synchronisation. Is there a way to get a callback when the luxonis cam triggers?

Best regards

jakaskerl

fbt_
The queue message has a getTimestamp method which is the timedelta at which the frame exposure began. You can time that up with the other camera to trigger it as a slave.

Thanks,
Jaka