Using Replay for 3D detection and estimation of human poses RGBD

karimkhater

Hello,
I have some recordings that i did using the OAK-D PRO, it includes 3 streams (Color,Left,Right), and i was wondering how can i use those data and the Replay Class to do pose estimations RGBD. I tried using the sdk but it's only for RGB and also tried BlazePoseDepthai and the implementation made by geaxgx but they only take into consideration the Depth when it's recorded in Real-Time. Is there a documentation online on how to do it ?

Thanks in advance.

jakaskerl

karimkhater
No documentation since it is an edge use-case. The idea is to follow luxonis/depthai-experimentsblob/master/gen2-record-replay/replay.py , using replay class and insert it into blazepose code where dai.Device() is called. You would need to delete/replace all parts where the cameras are created. I think this can be done in a single file.

geaxgx/depthai_blazeposeblob/main/BlazeposeDepthai.py

Thanks,
Jaka

karimkhater

thanks for the response, i will give it a try and keep you updated.

karimkhater

@jakaskerl

i have been trying to implement it for two days with depthai_sdk==1.15.0 and for some reason whenever i try to print replay.frames it always show that there is 0 frames but when i print replay.getstreams i get my color,left,right streams
therefore i tried to downgrade towards depthai_sdk==1.9.4 and i can find my frames, is this a problem with the newest version of the sdk?

if args.replay:
    pipeline = tracker.get_pipeline()
    replay = tracker.get_replay()
 
    with dai.Device(pipeline) as device:
        replay.createQueues(device)
        tracker.set_Device(device)
        tracker.get_Queues()
        
        while replay.sendFrames():
            print("HERE")
            print(replay.disabledStreams)
            print(replay.getStreams())
            print(len(replay.frames))
            
            rgbFrame = replay.frames['color_in']
            frame, body = tracker.next_frame(rgbFrame)
            print(frame,body)
            if frame is None: 
                break
            # Draw 2d skeleton
            frame = renderer.draw(frame, body)
            key = renderer.waitKey(delay=1)
            if key == 27 or key == ord('q'):
                break

i get this using the old version :

HERE
[]
['color', 'left', 'right']
3

and this with the newest one :

[]
['color', 'left', 'right']
0

UPDATE : i found out that starting from depthai_sdk==1.9.5 i get 0 frames and for before 1.9.5 i get 3 frames

jakaskerl

karimkhater
There were plenty of breaking changes from 1.9 to 1.15 so I would assume the functions don't work the same anymore.

Record/replay folder was last update 2 years ago..
We are currently working on depthai v3 that will have this functionality directly in the API.

Thanks,
Jaka

karimkhater

jakaskerl Can i know when is the expected release of depthai v3? or is it still in early development?

Thanks,
Karim

jakaskerl

karimkhater
Expected release is mid November.

Thanks,
Jaka