OAK-D Pro W PoE GPIO reference

cnolan

I've been wondering about using GPIOs on the OAK-D Pro W PoE for solving my frame syncing issues. There are a few different references for the M8 connector, the current version of the OAK-D Pro W PoE documentation page suggests that there is just a single 3V3 GPIO exposed on pin 1 via GPIO 52. There is also a VBUSIN at 5V to 'power accessories' but seemingly no associated 5V accessory lines. The new daisy-chaining FSYNC Y-Adapter wiring diagram also seems to refer to pin 3 as a GPIO3_5V line, which is also confusing, but would be useful if it could be used as such. Is there a a GPIO reference for the OAK-D Pro W PoE?

jakaskerl

cnolan
There is a single 3V3 GPIO pin at position 1. For the FSYNC adapter, I'd have to ask the EE team, but I think the connection is not named correctly (or is confusing). It is connected to ISO ground, not to the GPIO.

Thanks,
Jaka

cnolan

Thanks Jaka, is there any documentation about using that 3v3 GPIO line? We've tried triggering it using MX_PIN=52 along the lines of the example in the Script node documentation but we're getting no change on that M8 pin (the pin seems to be floating).

jakaskerl

cnolan
Set GPIO 6 to HIGH as well so it reverses level shifter direction. Then it should work.

#!/usr/bin/env python3
'''
NOTE: This should only be run on OAK-D-S2-POE (pro), as other OAK cameras might have different GPIO configuration!
'''
import depthai as dai
import time

# Start defining a pipeline
pipeline = dai.Pipeline()

script = pipeline.create(dai.node.Script)
script.setScript("""
    import time
    import GPIO

    MX_PIN_SET_DIR = 6 # set direction
    MX_PIN_SET_VAL = 52 # set value
    
    GPIO.setup(MX_PIN_SET_DIR, GPIO.OUT, GPIO.PULL_DOWN)
    GPIO.setup(MX_PIN_SET_VAL, GPIO.OUT, GPIO.PULL_DOWN)
    GPIO.write(MX_PIN_SET_DIR, True)  # Set direction to output
    
    toggleVal = True

    while True:
        toggleVal = not toggleVal
        ret = GPIO.write(MX_PIN_SET_VAL, toggleVal)
        node.warn(f"GPIO write: {toggleVal}")
        time.sleep(5)
        
""")
# Define script for output
script.setProcessor(dai.ProcessorType.LEON_CSS)


with dai.Device() as device:
    device.startPipeline(pipeline)
    print("Pipeline started")
    while True:
        time.sleep(1)

Thanks,
Jaka

cnolan

Thanks Jaka, is there any reference documentation for this kind of knowledge? It'd be nice to have a list of what is possible / what settings apply to what particular models etc.

jakaskerl

Hi @cnolan
We have schematics available for download on depthai-hardware, but unfortunately, the Pro Poe is a chip down design so we can't opensource it due to IP. So most I can do is tell you what is connected to what.

GPIO boot 1v8 is the GPIO 52 signal. The DIR signal connects to MXIO6, from schematics you could see that pin 6 on the MX must be HIGH in order for level shifter to be set as output.

Thanks,
Jaka

cnolan

Thanks Jaka. I've managed to get the that GPIO working now.

A follow up question: is there an internal line that I can use in the script node as a frame capture / FSYNC interrupt?

jakaskerl

cnolan
Try GPIO41. Then set all cameras as inputs and follow: https://docs.luxonis.com/hardware/platform/deploy/frame-sync/ .

Thanks,
Jaka

cnolan

Sorry Jaka, I mean I want to read the FSIN line, for example if I set left to output and rgb / right to input, then I want the script to read / interrupt on that line.

jakaskerl

Hi cnolan
Unfortunately, the FSYNC line is not accessible from inside the script node.

cnolan

Thanks jakaskerl, understood. I'm guessing the same goes for the STROBE line, that's also not accessible?

jakaskerl

cnolan I'm guessing the same goes for the STROBE line, that's also not accessible?

Correct.

cnolan Oh, and another question: given that the FSYNC line can't output, what kind of timing / duty cycle do we need to provide for the input (i.e. so we can sync between cameras)? In the documentation it says FSIN can only correct for a certain amount of drift - how does it get to a synchronised state with an input line at arbitrary phase relative to the camera start time?

cc @Luxonis-Alex

Thanks,
Jaka

cnolan

Oh, and another question: given that the FSYNC line can't output, what kind of timing / duty cycle do we need to provide for the input (i.e. so we can sync between cameras)? In the documentation it says FSIN can only correct for a certain amount of drift - how does it get to a synchronised state with an input line at arbitrary phase relative to the camera start time?

cnolan

So I think my external FSYNC might be working, but that's as far as I've got. As fredrik said over on this thread:

Reading the documentation made me believe that it ought to be really simple. Unfortunately, from what I understand, that is not the case.

All I'm trying to do is find a way to synchronise frames between both cameras and an external system. What I can't do:

I can't start and stop frame capture on demand (then use the strobe output to register frames) because the camera crashes.
I can't trigger frame capture on demand using FSIN because the cameras will run at their set FPS regardless of the FSIN. This was how I was hoping to control everything, just instruct the cameras to wait until they receive a trigger to take each video frame regardless of whether that was at 0 or 60 FPS. Using an external trigger is what would seem to be the thing to achieve this, but I can't seem to manage this at a steady / reasonable frame rate, and all the frames I do capture are not exposed correctly.
I can't use script nodes to trigger the GPIO pin on frame capture and route the frames as appropriate, because the timing of script nodes does not seem to be reliable (and on top of that, it seems to be fairly easy to slow the entire pipeline down just by conditionally copying frames from input to output).
I can't seem to enable and disable the STROBE output to signal which frames I'm recording, as setStrobeDisable does not appear to have any effect when I send it to the cameras.

All I want is, for every frame (with each frame synced between the two mono & rgb cameras and between multiple PoE camera sets), to have a trigger logged against the timing of an external ~ 1000Hz signal I'm capturing, either by triggering from the external system or by recording frame capture events from the cameras in the external system. How can I manage this?

jakaskerl

cnolan

cnolan I can't start and stop frame capture on demand (then use the strobe output to register frames) because the camera crashes.

Which camera is the culprit of this? I would guess that that either RGB does not want to stop the capture or that you have a more complex pipeline in which case stopping streams from one would crash the pipeline.

cnolan I can't trigger frame capture on demand using FSIN because the cameras will run at their set FPS regardless of the FSIN. This was how I was hoping to control everything, just instruct the cameras to wait until they receive a trigger to take each video frame regardless of whether that was at 0 or 60 FPS. Using an external trigger is what would seem to be the thing to achieve this, but I can't seem to manage this at a steady / reasonable frame rate, and all the frames I do capture are not exposed correctly.

Can you show me the MRE of the code you are using?

cnolan I can't seem to enable and disable the STROBE output to signal which frames I'm recording, as setStrobeDisable does not appear to have any effect when I send it to the cameras.

STROBE is driven by left camera, whenever this camera is active the strobe will signal so.

cnolan All I want is, for every frame (with each frame synced between the two mono & rgb cameras and between multiple PoE camera sets), to have a trigger logged against the timing of an external ~ 1000Hz signal I'm capturing, either by triggering from the external system or by recording frame capture events from the cameras in the external system. How can I manage this?

You need an external system that drives the FSYNC at 12-24V at exactly the same interval (and set the FPS of the cameras to match that) - cameras are synced internally. That should do it, but I fear it might not be possible to start/stop on demand - at least not when using IMX378.

Thanks
Jaka

cnolan

Sorry @jakaskerl, completely missed your reply here. Regarding the first question, even this simple example doesn't work (the cameras stop, but won't start up again). In addition, if I disable streaming via the initialControl, a few frames are captured when the pipeline starts anyway. Am I doing something wrong here?

#!/usr/bin/env python3

import cv2
import depthai as dai

# Create pipeline
pipeline = dai.Pipeline()

# Define sources and outputs
monoLeft = pipeline.create(dai.node.MonoCamera)
monoRight = pipeline.create(dai.node.MonoCamera)
xoutLeft = pipeline.create(dai.node.XLinkOut)
xoutRight = pipeline.create(dai.node.XLinkOut)

xoutLeft.setStreamName('left')
xoutRight.setStreamName('right')

# Properties
monoLeft.setBoardSocket(dai.CameraBoardSocket.CAM_B)
monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
monoLeft.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.OUTPUT)
monoRight.setBoardSocket(dai.CameraBoardSocket.CAM_C)
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
monoRight.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)

controlIn = pipeline.create(dai.node.XLinkIn)
controlIn.setStreamName('control')
controlIn.out.link(monoRight.inputControl)
controlIn.out.link(monoLeft.inputControl)

# Linking
monoRight.out.link(xoutRight.input)
monoLeft.out.link(xoutLeft.input)

# monoLeft.initialControl.setStopStreaming()
# monoRight.initialControl.setStopStreaming()

device_infos = dai.Device.getAllAvailableDevices()
print(device_infos)

# Connect to device and start pipeline
with dai.Device(device_infos[0]) as device:
    device.startPipeline(pipeline)
    controlQueue = device.getInputQueue('control')

    qLeft = device.getOutputQueue(name="left", maxSize=4, blocking=False)
    qRight = device.getOutputQueue(name="right", maxSize=4, blocking=False)

    while True:
        inLeft = qLeft.tryGet()
        inRight = qRight.tryGet()

        if inLeft is not None:
            cv2.imshow("left", inLeft.getCvFrame())

        if inRight is not None:
            cv2.imshow("right", inRight.getCvFrame())

        key = cv2.waitKey(1)
        if key == ord('q'):
            break
        elif key == ord('p'):
            ctrl = dai.CameraControl()
            ctrl.setStopStreaming()
            controlQueue.send(ctrl)
        elif key == ord('s'):
            ctrl = dai.CameraControl()
            ctrl.setStartStreaming()
            controlQueue.send(ctrl)

jakaskerl

cnolan
No, this has to be fixed, it's a depthai bug.

Thanks,
Jaka

cnolan

Ok that’s good to know. Is there any timeline for a fix?

Cheers,
Chris

jakaskerl

cnolan
No timeline, yet unfortunately. Hopefully somewhere down the line after v3 drops.

Thanks,
Jaka