TL/DR: I can turn my PyTorch model into a blob. But I cannot run
model(image), the forward function, if the model is a blob.
Long story: I managed to get the model to infer on the RGB camera node, so that is pretty neat. However, I am using the PyTorch model to actually run the inference, because I could not figure a way to do so for the blob version.
This tutorial stops exactly at generating the blob.
Also, this repo is unclear to me. I see that the function
get() is being called, but when I do that, it actually freezes my program. What is
get() supposed to return?
My (simplified) setup for this is the following:
Convert model to blob:
# Load model checkpoint
checkpoint = torch.load('checkpoints/checkpoint_ssd300.pt', map_location='cuda')
model = checkpoint['model']
# Export the model
dummy_input = torch.randn(1, 3, 300, 300, device='cuda')
torch.onnx.export(model, dummy_input, "ssd300.onnx", verbose=False)
# Simplify the model
model_simp, check = simplify('ssd300.onnx')
assert check, "Simplified ONNX model could not be validated"
# Convert the model to blob
shaves=6, use_cache=True, data_type='FP16')
Link neural net (along with RGB cam, skipped):
nn = pipeline.createNeuralNetwork()
xout_nn = pipeline.createXLinkOut()
# Load model
with depthai.Device(pipeline) as device:
q_rgb = device.getOutputQueue("rgb")
q_nn = device.getOutputQueue("nn")
frame = None
# Main loop
# Instead of get (blocking), we use tryGet (nonblocking)
# which will return the available data or None otherwise
in_rgb = q_rgb.tryGet()
in_nn = q_nn.tryGet() # this returns None
if in_rgb is not None:
# Retrieve 'bgr' (opencv format) frame
frame = in_rgb.getCvFrame()
if frame is not None:
# what could "model" be here, aside from a pytorch checkpoint?
box_location, text_location, text_box_location, det_labels = detect(model, frame, min_score=0.5,
detect function expects a PyTorch checkpoint, because somewhere in the code I call the
def detect(model, frame, ....):
.... = model(frame)
If anyone can help me with this, my question is how can I actually forward pass on the blob model itself. I have no idea how to handle this problem.
Thank you very much if you are reading this. For completeness, the whole code is here.
Edit: Even with the linking to the rgb cam (which I had forgotten)
in_nn (=q_nn.tryGet) is just a
depthai.NNData, which cannot run a forward pass.
in_nn.getData() does return a list, but its shape cannot be broken into the expected shapes of the outputs that would normally be optained via inference. Any ideea what this could mean?
The following getters do exist, but naturally they output empty lists: