Hello,

I am trying to write the same python app (gen2-deeplabv3_person) in a C++ application. When I get the vector by calling getLayerInt32, I want to generate the color cv::Mat. I can't figure out how to write the following python code into C++:

def decode_deeplabv3p(output_tensor):
class_colors = [[0,0,0], [0,255,0]]
class_colors = np.asarray(class_colors, dtype=np.uint8)

output = output_tensor.reshape(nn_shape,nn_shape)
output_colors = np.take(class_colors, output, axis=0)
return output_colors

Does anybody have an idea how this can be written in C++? I will appreciate any hints anybody can provide.

Thanks,

James

  • erik replied to this.

    Hi Tusk,
    Unfortunately, I am not familiar with C++ / OpenCV, so I would suggest asking this question on OpenCV community (discord/stack overflow/etc.) as it's not specific to DepthAI/OAK cameras.
    Thanks, Erik

    • Tusk replied to this.

        erik

        Erick,

        Thank you for your reply. My apologies if my question wasn't clear. I am asking about the 32 bit vector from the tensor layer, which is retrieved from the DataOutputQueue via the Depth AI SDK that I am using to perform segmentation with an oak-d lite camera. I believe my question is in the appropriate discussion forum, as it isn't about OpenCV itself, but about how to interpret what I am getting out of the layer and use it in a cv::Mat, just like it is used in the gen2-deeplabv3_person example in the Depth AI experiments git. It may not be you, but I believe there may be somebody in the forum that can help me.

        Thanks,

        James

          So, you are asking what is the output of the model (the Int32 layer that you get with DepthAI)?
          The output layer (before reshaping) is a [256x256 vector,], which after being reshaped to 256x256 represents a binary mask, where 0=background and 1=person. In the deeplabv3_person experiment, the output is actually being reshaped to [256, 256, 1, 1], so that np.take() can be used while decoding the output (generating color mask). An alternative way of doing this is by first reshaping the array to [256, 256] only, scaling the binary mask to range [0-255], and then calling cv2.applyColorMap(). See decoding in deeplabv3_multiclass. Here the output layer is the same, with the difference that mask is not binary, but contains integers corresponding to each class. If you are using only a person classifier, this could be modified by simply multiplying all 1s corresponding to the person class by 255, and then applying the color map.

          Is this what you are interested in @Tusk ?

          • Tusk replied to this.

            Matija thank you very much for the reply. I have a better understanding of the 32 bit vector. In python, it is very easy to perform reshaping of np arrays. Since I am doing this in cpp, my hope was to get some hints on how to do this with cv::Mat. Do you have any experience with this sort of array manipulation in cpp?

            Thanks,

            James

                Hey, I personally don't unfortunately. I'll forward this internally and someone should get back to you.