I'll be working on improving the performance of the demo, and from initial profiling shown that most of the time is spend on frames decoding, but there are also other improvements to be made as well (e.g. moving resize calls on device wherever possible)
To add to Brandon's answer, we're also investigating H264/H265 usage as it has HW acceleration, which can give more improvements than using turbojpeg for decoding speed (which I added today while testing how to speed up the decoding process)
Regarding web preview, this fits nicely into the efforts we've already made with the demo and we'll focus on adding it after performance optimizations - as not only will it allow to use regular browser to preview the demo, it will also be useful for Docker use-case (where we won't have to map the display to container as it is right now, we'll be able to simply expose a port with the web server).
In fact, while introducing QT GUI, we added a parameter named
--guiType, which now accepts just two values -
qt. So, having all of the demo communication already battle-tested with QT, we can add a
web GUI type and use current QT GUI implementation and one of the experiments we have as a starting points.
We could actually mimic what the QT GUI is doing currently, but with HTML elements instead of QML, so both of these experiences would be consistent.
I didn't think about which technology to use to create this web preview, so if you have any thoughts or suggestions on what we should use, you're more than welcome to share here.
For streaming process, we have the following ones ready-to-use:
And personally, I think MJPEG + REST API would be a go here due to stability reasons, with WebRTC being a great modern way to approach this project but at the same time being more unstable and setup-related (e.g. with HTTP access)
For webserver, I would choose Flask, but again, alternatives welcome.