I am trying to do simple pose detection using mediapipe, and would like to pass the video footage from Oak-D like you would do with a webcam. I want to use the SDK code because API seems a bit too complex for me.
The code for webcam is:
import cv2
import mediapipe as mp
mpDraw = mp.solutions.drawing_utils
mpPose = mp.solutions.pose
pose = mpPose.Pose()
webcam = cv2.VideoCapture(0)
while True:
success, img = webcam.read()
results = pose.process(img)
if results.pose_landmarks:
mpDraw.draw_landmarks(img, results.pose_landmarks, mpPose.POSE_CONNECTIONS)
cv2.imshow("Image", img)
if cv2.waitKey(1) == ord('q'):
break
Basically I wish I can just access the Oak-D footage simply like with "cv2.VideoCapture(0)
However, in the SDK, I think the video frames is packet.frame. But the problem is that I can only access it in a callback function?
So I tried this
import cv2
import time
import mediapipe as mp
from depthai_sdk import OakCamera
from depthai_sdk.classes import DetectionPacket
with OakCamera() as oak:
color = oak.create_camera('color')
mpDraw = mp.solutions.drawing_utils
mpPose = mp.solutions.pose
pose = mpPose.Pose()
def cb(packet: DetectionPacket):
while True:
results = pose.process(packet.frame)
# print(results.pose_landmarks)
if results.pose_landmarks:
mpDraw.draw_landmarks(packet.frame, results.pose_landmarks, mpPose.POSE_CONNECTIONS)
cv2.imshow(packet.name, packet.frame)
if cv2.waitKey(1) == ord('q'):
break
oak.callback(color.out.main, callback=cb)
oak.start(blocking=True)
Somehow it sort of works, but it is stuck on the first frame. If I hold 'q' it will work play continuously like a normal webcam until it exits out with an error. How can I fix this error? Or is there an easier way to pass the Oak-D video feed into the pose.process without creating a callback function?
Thank you for your patience!