Sync 2 Oak Pro Cameras, set the iso/exposure and record.

Hutch07

I would like to synchronize my 2 Oak-D Pros and record the video (CAMB and CAM_C only) with the IMU information for each frame (15 FPS) exported to a CSV. This was going well until I added adjusting the exposure of each camera. I had to upgrade the depthai-SDK to 1.13 for the exposure and now the reworked example code for record.py needs a major overhaul. It was written for depthai-sdk 1.2.

Can you suggest psudo code (or actual) on how to record these synchronized 2 streams with IMU info?

jakaskerl

Hi Hutch07
Could you post the code you are using? What error are you receiving when adding the exposure adjusting?

You could combine the experiment:
https://github.com/luxonis/depthai-experiments/tree/master/gen2-syncing#imu--rgb--depth-timestamp-syncing
With API's way of recording. Basically you would need to write all frames to a file each iteration. It might be a bit difficult to achieve for 15FPS, but largely depends on the host machine.

Thanks,
Jaka

Hutch07

IMU Example from here: https://docs.luxonis.com/projects/sdk/en/latest/samples/recording/SDK_mcap_record_imu/

FYI - Pasting code seems impossible on this site.

My python file is here:

I also modified the camera_control.py in the SDK to add : def set_exposure_iso(self, exposure, iso):

Hutch07

Error here is "No module named mcap_ros1

jakaskerl

Hi Hutch07
But you were able to run the mcap record example before adding the exposure control?

As for syncing, I would recommend you follow this example https://github.com/luxonis/depthai-experiments/tree/master/gen2-syncing#sync-multiple-oak-cameras.

Thanks,
Jaka

Hutch07

"luxonis/depthai-experiments/blob/master/gen2-syncing/host-multiple-OAK-sync.py"

works fine. I will save each frame as a file with frame-count in the name. Then I can save a CSV file with frame-count, IMU data, timestamps. I think somehow saving video would be faster.

:No MCAP will not work. Only VIDEO

Hutch07

If anyone can give me a hint on how to set the exposure with a pipeline, I think the modifications to host-multiple-OAK-sync.py would work. The motion blur I am getting will not work with auto exposure.

jakaskerl

Hi Hutch07

oakL =OakCamera()

controls = {
    'exposure':{
                'manual': [10, 200]
    },
}

leftL = oakL.create_camera('left', resolution='800p', fps=20)
rightL = oakL.create_camera('right', resolution='800p', fps=20)

imuL = oakL.create_imu()
imuL.config_imu(report_rate=100, batch_report_threshold=5)
oakL.visualize([leftL, rightL])
oakL.start(blocking=False)
while oakL.running():
    leftL.control.send_controls(controls)
    oakL.poll()

Or you can do it by do it with leftL.node.initialControl.setManualExposure(expos_start, iso_start)

This sets both cameras to the same settings

Thanks,
Jaka

Hutch07

When I use the depthai_sdk, I do not have a problem setting the exposure and can save the video.

When I use depthai like in "luxonis/depthai-experiments/blob/master/gen2-syncing/host-multiple-OAK-sync.py"

I can sync the images but setting the exposure seems to do nothing.

The Mono Camera exposure control example works with a single camera. "https://docs.luxonis.com/projects/api/en/latest/samples/MonoCamera/mono_camera_control/#mono-camera-control"

Hutch07

This code starts 2 pipelines for 2 OAK-D cameras but sending the manual exposure has no effect on the cameras. They stay in auto-exposure mode.

jakaskerl

Hi Hutch07
You are missing the link between Input (XLINK) and the control.

controlLeft = pipeline.create(dai.node.XLinkIn) #controlIn
controlLeft.setStreamName('controlLeft')
controlRight = pipeline.create(dai.node.XLinkIn) #controlIn
controlRight.setStreamName('controlRight')

controlLeft.out.link(cam[c].inputControl)
controlRight.out.link(cam[c].inputControl)

Thanks,
Jaka

Hutch07

That absolutely worked. I updated my def create_pipeline(cam_list) in the file and took out the "cam" loop to make it a bit simpler.

Thank You Jaka!

Hutch