synchronizing RGB/DEPTH/IMU leadst to 1FPS

JanKanka

Hi.

I'm implementing synchronization between IMU/RGB/DEPTH to be able to reconstruct true position of object in time. I'm following this demo, but after runnning, the shown FPS drops to 1-2FPS. Separate modality works just fine in 30FPS, same as nonsync RGB and DEPTH.

Is there any solution, how to speed it up all?

Thank you, best wishes, Jan

erik

Hi JanKanka ,
Are you using PoE or USB? Perhaps you could try increasing the threshold for the timestmap syncing, which is currently 11ms (hardcoded).
Thanks, Erik

JanKanka

Hi @erik I!m using PoE version of thje camera.

I did try to increase the threshold up to 100 but it went to 10FPS, not even close the 30FPS.

Thank you.

Best wishes, Jan

erik

JanKanka what's the FPS you receive each stream separately?

JanKanka

Without synchronisation I tried RGB @ 1080p up to 40FPS without problem. In default I use rgb & depth @ 30FPS.

erik

JanKanka I very much doubt you're able to get 1080P at 40FPS RGB stream - as that's above the 1gbps link bandwidth;
https://docs.luxonis.com/projects/api/en/latest/tutorials/low-latency/#bandwidth
Perhaps each of these streams separately can run at 30FPS, but not together on the POE (without downscaling/encoding/lowering FPS).

JanKanka

You are right, I do downscale the isp by factor 2/3, my bad.

So what would be the best solution here? To downscale the Images or somehow crop the images itself?

Thank you

erik

JanKanka You'd want to reduce bandwidth usage to below 700mbps (max bandwidth), please see the docs above on how to do that

JanKanka

Let me get back to this thread once more again, I'm trying to crop images on camera, but while croping both depth and rgb image the dai rises Fatal error." Please report to developers. Log: 'PipelineBuilder' '2221'"

I follow this two tutorials: depth crop, color crop

Thank you

jakaskerl

Hi JanKanka
Please attach a MRE so we can debug this.

Thanks,
Jaka

JanKanka

After rebooting the camera, everything works. Thank you