Hi team, We're building a production monitoring system on an OAK-4D R9 deployed via Luxonis Hub as an OAK App. We have a custom browser dashboard (not the Luxonis Visualizer) that should show live video and JSON metrics from the camera in real time, no Foxglove UI, just our own canvas/HTML elements. We want to have video, (worst case images) and data on the UI side. We followed the architecture of custom UI -> custom server -> luxonis HUB API -> camera but how can we get the custom stream and metadata? More specifically, our architecture: * Camera app runs `dai.RemoteConnection` (httpPort=8082) * Hub relays WebRTC from the camera to the browser via `requestDeviceAppConnection` * Browser connects to `wss://signal.cloud.luxonis.com/session/` with the returned `clientId`/`authToken` * Browser implements the Foxglove WebSocket protocol over WebRTC data channels to subscribe to topics **What works:** * Hub WebRTC signaling works correctly (`AcknowledgeSession` → SDP offer/answer → connection state `connected`) * `system` channel opens, we receive `advertise` messages (binary, 2-byte header before JSON) * Topics added with `viz.addTopic("Color", color_out)` (node output overload) appear in `advertise` **What doesn't work / questions:** 1. **Topics added with the queue overload are never advertised.** We call `queue = viz.addTopic("Metrics", useVisualizationIfAvailable=False)` and push JSON bytes via `queue.send(buf)`. The topic never appears in the Foxglove `advertise` message. Only topics using the node-output overload show up. Is this expected? Is there a way to advertise a custom JSON topic via WebRTC without a `dai.Node.Output`? 2. **`Color` topic is advertised and subscribed but no data arrives on the `message` data channel.** The camera runs `viz.addTopic("Color", color_out)` where `color_out` is 1920×1080 NV12 at 20fps (\~3MB/frame). After subscribing via `{"op":"subscribe","subscriptions":[{"id":1,"channelId":}]}`, zero messages arrive. Possible causes: frames too large for SCTP data channels? Does `Color` come as a WebRTC media track instead? Is the SDP offer supposed to include `m=video`? 3. **Best practice for MJPEG via WebRTC data channel.** We plan to add a `VideoEncoder` (MJPEG profile, 640×360) and advertise via the node-output overload: `viz.addTopic("ColorJPEG", mjpeg_enc.bitstream)`. Will MJPEG bitstream frames arrive in the `message` data channel as raw JPEG bytes (payload starts at byte 13 after Foxglove header)? Are there any schema/encoding fields we need to include in the channel metadata on the `system` channel? 4. **Hub tunnel for HTTP.** We use `assign_frontend_port = true` in `oakapp.toml`. Does Hub set a `FRONTEND_PORT` environment variable that the app should read and bind to? We want to serve a plain HTTP `/metrics` JSON endpoint that the VPS server polls via the Hub-proxied `frontend.url`. Does `team { deviceApp(id:) { frontend { url } } }` return the HTTPS tunnel URL that proxies to `localhost:{FRONTEND_PORT}` on the camera? 5. **`useVisualizationIfAvailable` docs.** The docs say "If present will call `getVisualizationMessage` method on the output objects." Does `False` affect whether the topic is advertised via WebRTC at all, or only how the data is encoded before transmission? Our use case (custom dashboard on a Hub-deployed app, receiving both video and structured data via WebRTC) is probably common for production deployments where you don't want to embed the full Foxglove visualizer. Any guidance on the correct pattern would be very helpful. Thanks

**Hey @"OskarSonc"#3466**, one more question on the OAK-4D R9 networking side. We tried Tailscale to create a private tunnel between the camera and our VPS backend. The problem is that on OAK-4D R9, the OS uses overlay filesystems, so Tailscale's service doesnt automatically restart, making it unusable for a production deployment. Does Luxonis have a recommended VPN or secure tunnel solution that works reliably with the OAK-4D R9? Specifically: 1. Is there a persistent writable path in the OAK app container that survives reboots (like a Docker named volume) where VPN state could be stored? 2. Has anyone on the team validated Tailscale (or WireGuard directly) or OPEVPN working persistently on OAK-4D R9? 3. If no VPN is officially supported, is direct outbound HTTP from the camera to a public backend endpoint the expected pattern? Thanks I understand that we are going a little bit out of topic for this thread, I just wanted to ask, because I think it is relevant with the data pushing camera_*edgecustom*server.

Best way to stream custom topics a custom UI

nstavrou

Hi team,

We're building a production monitoring system on an OAK-4D R9 deployed via Luxonis Hub as an OAK App. We have a custom browser dashboard (not the Luxonis Visualizer) that should show live video and JSON metrics from the camera in real time, no Foxglove UI, just our own canvas/HTML elements.

We want to have video, (worst case images) and data on the UI side. We followed the architecture of custom UI -> custom server -> luxonis HUB API -> camera but how can we get the custom stream and metadata?

More specifically, our architecture:

Camera app runs dai.RemoteConnection (httpPort=8082)
Hub relays WebRTC from the camera to the browser via requestDeviceAppConnection
Browser connects to wss://signal.cloud.luxonis.com/session/ with the returned clientId/authToken
Browser implements the Foxglove WebSocket protocol over WebRTC data channels to subscribe to topics

What works:

Hub WebRTC signaling works correctly (AcknowledgeSession → SDP offer/answer → connection state connected)
system channel opens, we receive advertise messages (binary, 2-byte header before JSON)
Topics added with viz.addTopic("Color", color_out) (node output overload) appear in advertise

What doesn't work / questions:

Topics added with the queue overload are never advertised. We call queue = viz.addTopic("Metrics", useVisualizationIfAvailable=False) and push JSON bytes via queue.send(buf). The topic never appears in the Foxglove advertise message. Only topics using the node-output overload show up. Is this expected? Is there a way to advertise a custom JSON topic via WebRTC without a dai.Node.Output?
Color topic is advertised and subscribed but no data arrives on the message data channel. The camera runs viz.addTopic("Color", color_out) where color_out is 1920×1080 NV12 at 20fps (~3MB/frame). After subscribing via {"op":"subscribe","subscriptions":[{"id":1,"channelId":<colorId>}]}, zero messages arrive. Possible causes: frames too large for SCTP data channels? Does Color come as a WebRTC media track instead? Is the SDP offer supposed to include m=video?
Best practice for MJPEG via WebRTC data channel. We plan to add a VideoEncoder (MJPEG profile, 640×360) and advertise via the node-output overload: viz.addTopic("ColorJPEG", mjpeg_enc.bitstream). Will MJPEG bitstream frames arrive in the message data channel as raw JPEG bytes (payload starts at byte 13 after Foxglove header)? Are there any schema/encoding fields we need to include in the channel metadata on the system channel?
Hub tunnel for HTTP. We use assign_frontend_port = true in oakapp.toml. Does Hub set a FRONTEND_PORT environment variable that the app should read and bind to? We want to serve a plain HTTP /metrics JSON endpoint that the VPS server polls via the Hub-proxied frontend.url. Does team { deviceApp(id:) { frontend { url } } } return the HTTPS tunnel URL that proxies to localhost:{FRONTEND_PORT} on the camera?
useVisualizationIfAvailable docs. The docs say "If present will call getVisualizationMessage method on the output objects." Does False affect whether the topic is advertised via WebRTC at all, or only how the data is encoded before transmission?

Our use case (custom dashboard on a Hub-deployed app, receiving both video and structured data via WebRTC) is probably common for production deployments where you don't want to embed the full Foxglove visualizer. Any guidance on the correct pattern would be very helpful.

Thanks

OskarSonc

Hey nstavrou a couple of the things you’re trying are outside what RemoteConnection currently supports.

viz.addTopic(...queue...) is not a general “advertise arbitrary JSON bytes” path. A topic is only advertised after the first message can be serialized as a supported visualizer/Foxglove message, so a plain dai.Buffer containing JSON will not show up as a topic. For structured app data, the supported pattern today is to use registerService(...) on the backend and postToService(...) / fetchService(...) from the frontend, or expose a small HTTP endpoint from the app.

For video, I would strongly recommend using an encoded stream (VideoEncoder.out, typically H.264) rather than raw 1920x1080 NV12 @ 20fps. That remote path is intended for encoded streams; sending raw frames at that size/rate is likely too heavy. Also, encoded topics are not delivered as raw JPEG/H264 bytes directly after the Foxglove header, but as a DepthAI EncodedFrame protobuf payload.

For the frontend tunnel part, assign_frontend_port = true assigns OAKAPP_STATIC_FRONTEND_PORT inside the container, which is the port your app should bind to if you want to serve your own HTTP endpoint or frontend.

Useful references:

OAK App config / frontend port: docs.luxonis.com/software-v3/oak-apps/configuration
oakctl env vars: docs.luxonis.com/software-v3/oak-apps/oakctl
Custom frontend examples: github.com/luxonis/oak-examples/tree/main/custom-frontend

So the safest supported pattern for a custom Hub dashboard today is:

video via visualizer.addTopic("Video", encoder.out)
metrics/control via services or your own HTTP endpoint on OAKAPP_STATIC_FRONTEND_PORT

Thans,
Oskar

nstavrou

Hey @OskarSonc, thank you for your quick response. You've been very helpful.

We've read the Integration Architecture page which describes exactly our setup (we have a custom server VPS that we want to take the data and the server will serve the custom UI, sorry for not mentioning it in the beggining):

VPS backend calls Hub GraphQL for session bootstrap (requestDeviceAppConnection) ✅
VPS returns only the derived payload (authToken + clientId) to the browser ✅
Browser connects directly to the camera via WebRTC for live video ✅

This works perfectly. The gap is the data plane for metrics.

The integration architecture page describes the control plane (GraphQL) and the browser session transport (WebRTC), but doesn't cover how a backend receives live structured data from a running camera app 24/7 — even when no browser is connected.

We noticed the docs say "Webhooks are not yet a broad stable public integration surface" — which suggests this is a known gap.

Specific questions:

Is there a way for a VPS backend (with Hub API key) to access the camera's HTTP endpoint through the Hub tunnel? We get 401 with Bearer token on frontend.url.
Is camera-to-VPS outbound HTTP push (camera POSTs metrics directly to our server, outside Hub) the current recommended pattern for continuous backend data collection?
Are webhooks or a Hub data-plane API on the roadmap for this use case?

(We're already serving /metrics via NGINX, configured on backend-run.sh script) on OAKAPP_STATIC_FRONTEND_PORT (confirmed working from inside the container via wget). The 401 is specifically when our VPS backend tries to reach it through the Hub tunnel URL.)

Thank you.

OskarSonc

nstavrou - Thanks for the clarification. frontend.url is for the hosted app frontend/browser flow, not for server-to-server access with a Hub API key bearer token, so the 401 from your VPS is expected.

For continuous backend metrics collection, the recommended pattern today is to have the app push metrics outbound directly to your backend (HTTP/MQTT/etc.), independently of Hub/WebRTC. Hub’s public integration surface is currently focused on the control plane and browser session bootstrap, not a general backend data-plane for always-on app metrics. Webhooks are also not yet a broad stable public integration surface.

So the split for this use case is:

Hub API / requestDeviceAppConnection for live browser sessions
app -> your VPS outbound push for continuous metrics ingestion

Thanks,
Oskar

nstavrou

Hey @OskarSonc, one more question on the OAK-4D R9 networking side.

We tried Tailscale to create a private tunnel between the camera and our VPS backend. The problem is that on OAK-4D R9, the OS uses overlay filesystems, so Tailscale's service doesnt automatically restart, making it unusable for a production deployment.

Does Luxonis have a recommended VPN or secure tunnel solution that works reliably with the OAK-4D R9? Specifically:

Is there a persistent writable path in the OAK app container that survives reboots (like a Docker named volume) where VPN state could be stored?
Has anyone on the team validated Tailscale (or WireGuard directly) or OPEVPN working persistently on OAK-4D R9?
If no VPN is officially supported, is direct outbound HTTP from the camera to a public backend endpoint the expected pattern?

Thanks

I understand that we are going a little bit out of topic for this thread, I just wanted to ask, because I think it is relevant with the data pushing camera_edge<->customserver.

OskarSonc

Hey nstavrou - on OAK-4D R9, the app container root filesystem is not the right place for persistent VPN state. For persistent writable storage, use the app storage paths exposed by the platform such as OAKAGENT_STORAGE_APP / OAKAGENT_STORAGE_SHARED / OAKAGENT_STORAGE_GLOBAL.

At the moment, Luxonis does not ship an officially supported turnkey VPN/tunnel solution for OAK4. WireGuard kernel support is present and we have seen it work via nmcli, but it has not been validated by us as an official production deployment pattern. We do not currently have a validated Tailscale or OpenVPN recommendation for OAK-4D R9 either.

So if you need a production-safe path today, the expected pattern is still direct outbound HTTPS/MQTT/etc. from the app to your backend. If helpful, we can suggest a concrete outbound metrics pattern, or share the current WireGuard/Tailscale caveats in more detail.

Best,
Oskar