Modern GUI for depthai apps

ramkunchur

Hi @Brandon , @erik and depthai team....

I have one very basic, but probably a question every Computer Vision projects AI developer has (probably 🙂 )

We all know that using Python we can develop very good Computer Vision systems, performance, inference, they are all good.

But what I want to know is how do we make GUI for these OpenCV videostream/camera based inference projects?
Even though all apps work well and do what they need to do, without proper GUI, they look like academic projects and cannot be deployed in end user environments as standalone products.

I do know about GUI frameworks for Python such as Tkinter, Qt, Kivy etc, but want to know which one we use to integrate opencv camera frame produced by depthai app into a modern fullscreen GUI.

This way we can make actual products, may be camera frame on one side of screen and inference parameters value, buttons etc on other side of the screen.

Any help in right direction would be highly appreciated, even better with working example with depthai sample.

Thanks & Best Regards,
Ram

Mojo

Hi @ramkunchur,
great question 🙂. Here is a simple example using tkinter library that should hopefully point you in the right direction. It's not much, but it gets the job done.

#!/usr/bin/env python3

import cv2
import depthai as dai
import tkinter as tk
from threading import Thread


def createPipeline():
    pipeline = dai.Pipeline()

    # Define sources
    camRgb = pipeline.createColorCamera()
    monoLeft = pipeline.create(dai.node.MonoCamera)
    monoRight = pipeline.create(dai.node.MonoCamera)

    # Properties
    camRgb.setPreviewSize(640, 400)
    camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
    camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
    monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
    monoLeft.setBoardSocket(dai.CameraBoardSocket.LEFT)
    monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
    monoRight.setBoardSocket(dai.CameraBoardSocket.RIGHT)

    # Define outputs
    xoutRgb = pipeline.create(dai.node.XLinkOut)
    xoutLeft = pipeline.create(dai.node.XLinkOut)
    xoutRight = pipeline.create(dai.node.XLinkOut)
    xoutRgb.setStreamName("rgb")
    xoutLeft.setStreamName("left")
    xoutRight.setStreamName("right")

    streams = ("rgb", "left", "right")

    # Linking
    camRgb.preview.link(xoutRgb.input)
    monoLeft.out.link(xoutLeft.input)
    monoRight.out.link(xoutRight.input)

    return pipeline, streams


def run(pipeline):
    # Connect to device and start pipeline
    with dai.Device(pipeline) as device:
        queues = [device.getOutputQueue(s, 8, False) for s in streamNames]

        while True:
            for queue in queues:
                name = queue.getName()
                if name != currentStream:  # Display only the "selected" stream
                    continue
                image = queue.get()
                frame = image.getFrame() if name != "rgb" else image.getCvFrame()
                cv2.imshow(name, frame)

            if cv2.waitKey(1) == ord("q"):
                break


pipeline, streamNames = createPipeline()

tkWindow = tk.Tk()
tkWindow.geometry("400x150")
tkWindow.title("Switch streams example")


def callback(*args):
    global currentStream
    currentStream = tkWindow.getvar(args[0])
    cv2.destroyAllWindows()


currentStream = streamNames[0]

currentStreamVar = tk.StringVar(tkWindow)
currentStreamVar.set(currentStream)  # default value
currentStreamVar.trace_add("write", callback)

dropdown = tk.OptionMenu(tkWindow, currentStreamVar, *streamNames)
dropdown.pack()


thread = Thread(target=run, args=(pipeline,))
thread.setDaemon(True)
thread.start()

tkWindow.mainloop()

ramkunchur

Hi Mojo

Thanks a lot, I'll give it a try, I am looking for something more advanced such as Qt5 or something which gives lot of controls and gets the GUI designed quickly.

Thanks again for your valuable information 🙂

CodingArcher

ramkunchur Hi, i have also been working for quite some time on getting a GUI interface working for my project.
It definitly seems hard to get it working without having too much of a video lag happening. What i found was that @Mojo 's idea/concept with TKinter is the best way i have found so far. I do agree QT5 is very nice to use but makes it much more difficult to get it working

best of luck

ramkunchur

Hi CodingArcher

Thanks so much for your valuable information and help 🙂

BillS

I use pysimplegui. It is much better than Tk ( it is actually built on Tk)

GUI built with PySimpleGUIi https://pysimplegui.readthedocs.io/en/latest/

erik

Hello BillS , thanks for sharing! If you are interested, we are currently working on depthai demo GUI with QT, PR here.

BillS

I took a look. I'd be glad to try it it out. adding a GUI is a good way of learning an API. It is also a very effective vehicle for demonstrating the features and benefits of complex technology.