Newbie problems, MacOS, get data failing

petercorke

Just got my OAK-D today, made it's way to Australia. Very impressive. Install went fine (MacOS 10.15.7), demo runs fine. Then I tried the minimal working code sample. Maybe this is a doco gotcha, but my blob file has a suffix mobilenet-ssd.blob.sh14cmx14NCE1. Fixed that, but I get the diagnostics below.

My machine is a pretty new and grunty Intel iMac, OAK-D is plugged directly with no hub, tried the USB2 option, no better.

XLink initialized.
Sending internal device firmware
Successfully connected to device.
Usb speed : High/480Mbps
Mx serial id : 14442C10114B9CD000
Loading config file
Attempting to open stream config_d2h
watchdog started 
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: valid (v5)
  Board name     : BW1098OBC
  Board rev      : R0M0E0
  HFOV L/R       : 71.86 deg
  HFOV RGB       : 68.7938 deg
  L-R   distance : 7.5 cm
  L-RGB distance : 3.75 cm
  L/R swapped    : yes
  L/R crop region: center
  Rectification Rotation R1 (left):
    0.999948,    0.005908,    0.008364,
   -0.005910,    0.999983,    0.000217,
   -0.008363,   -0.000266,    0.999965,
  Rectification Rotation R2 (right):
    0.999776,    0.020927,   -0.003168,
   -0.020928,    0.999781,   -0.000208,
    0.003162,    0.000275,    0.999995,
  Calibration intrinsic matrix M1 (left):
  854.910889,    0.000000,  638.519897,
    0.000000,  855.479980,  406.928772,
    0.000000,    0.000000,    1.000000,
  Calibration intrinsic matrix M2 (right):
  860.884644,    0.000000,  637.492920,
    0.000000,  861.494568,  413.291748,
    0.000000,    0.000000,    1.000000,
  Calibration rotation matrix R:
    0.999821,   -0.015021,    0.011520,
    0.015015,    0.999887,    0.000666,
   -0.011529,   -0.000493,    0.999933,
  Calibration translation matrix T:
   -7.451624,
   -0.155977,
    0.023609,
  Calibration Distortion Coeff d1 (Left):
   -5.793810,   20.928015,    0.000307,    0.001104,  -25.365406,   -5.846439,   21.146641,
  -25.590557,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,
  Calibration Distortion Coeff d2 (Right):
   -5.310071,   17.243616,    0.000766,   -0.000059,  -19.681963,   -5.364985,   17.475134,
  -19.921881,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,    0.000000,
/Users/corkep/code/depthai/resources/nn/mobilenet-ssd/mobilenet-ssd.json
depthai: Calibration file is not specified, will use default setting;
config_h2d json:
{"_board":{"calib_data":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"mesh_left":[0.0],"mesh_right":[0.0]},"_load_inBlob":true,"_pipeline":{"_streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"NCEs":1,"NN_config":{"NN_family":"mobilenet","confidence_threshold":0.5,"output_format":"detection"},"blob0_size":14477056,"blob1_size":0,"calc_dist_to_bb":false,"camera_input":"rgb","cmx_slices":7,"keep_aspect_ratio":true,"num_stages":1,"shaves":7},"app":{"enable_reconfig":true,"sync_sequence_numbers":false,"sync_video_meta_streams":false,"usb_chunk_KiB":64},"board":{"clear-eeprom":false,"left_fov_deg":69.0,"left_to_rgb_distance_m":0.0,"left_to_right_distance_m":0.03500000014901161,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":69.0,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":false},"camera":{"mono":{"fps":30.0,"resolution_h":720,"resolution_w":1280},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"depth_limit_mm":10000,"lr_check":false,"median_kernel_size":7,"padding_factor":0.30000001192092896,"warp_rectify":{"edge_fill_color":-1,"mirror_frame":true,"use_mesh":false}},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
size of input string json_config_obj to config_h2d is ->1613
size of json_config_obj that is expected to be sent to config_h2d is ->1048576
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1048576 bytes to config_h2d
!!! XLink write successful: config_h2d (1048576)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 14477056
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #1!
Writing 14477056 bytes to inBlob
!!! XLink write successful: inBlob (14477056)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /Users/corkep/code/depthai/resources/nn/mobilenet-ssd/mobilenet-ssd.blob.sh14cmx14NCE1
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #2!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
Input layer : 
Name: data
Index: 0
Element type: uint8
Element size:  1byte
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 3, Height : 300, Width : 300]

Output layer : 
Name: detection_out
Index: 0
Element type: float16
Element size:  2 bytes
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 1, Height : 100, Width : 7]

CNN to depth bounding-box mapping: start(0, 0), max_size(0, 0)
Host stream start:metaout
Opening stream for read: metaout
Attempting to open stream metaout
Successfully opened stream metaout with ID #1!
Starting thread for stream: metaout
Host stream start:previewout
Opening stream for read: previewout
Attempting to open stream previewout
Started thread for stream: metaout
Successfully opened stream previewout with ID #2!
Starting thread for stream: previewout
depthai: INIT OK!
Started thread for stream: previewout
E: [global] [    119836] [] addEvent:264	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    119836] [] addEventWithPerf:276	 addEvent(event) method call failed with an error: 3
E: [global] [    119836] [] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
Closing stream metaout: ...
E: [global] [    119836] [] addEvent:264	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    119836] [] addEventWithPerf:276	 addEvent(event) method call failed with an error: 3
E: [global] [    119836] [] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
Closing stream previewout: ...
E: [global] [    119836] [] dispatcherEventSend:53	Write failed (header) (err -99) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [    119836] [] sendEvents:1027	Event sending failed
Closing stream metaout: DONE.
Thread for metaout finished.
E: [global] [    119836] [] dispatcherEventSend:53	Write failed (header) (err -99) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [    119836] [] sendEvents:1027	Event sending failed
Closing stream previewout: DONE.
Thread for previewout finished.
watchdog triggered 
=== New data in observer stream host_capture, size: 4
Writing 4 bytes to host_capture
E: [global] [    126501] [] dispatcherEventSend:53	Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    126501] [] sendEvents:1027	Event sending failed
!!! XLink write successful: host_capture (4)
Stopping threads: ...
Stopping threads: DONE 0.000s.
Closing all observer streams: ...
Closing stream host_capture: ...
Closing stream host_capture: DONE.
Closing all observer streams: DONE.
Reseting device: 0.
E: [global] [    126501] [] dispatcherEventSend:53	Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [    126501] [] sendEvents:1027	Event sending failed
E: [global] [    126501] [] XLinkResetRemote:266	can't wait dispatcherClosedSem

Reseting: DONE.
XLink already initialized.
Sending internal device firmware
Successfully connected to device.
Usb speed : High/480Mbps
Mx serial id : 14442C10114B9CD000
Loading config file
Attempting to open stream config_d2h
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: valid (v5)
  Board name     : BW1098OBC
  Board rev      : R0M0E0
  HFOV L/R       : 71.86 deg
  HFOV RGB       : 68.7938 deg
  L-R   distance : 7.5 cm
  L-RGB distance : 3.75 cm
  L/R swapped    : yes
  L/R crop region: center
  Rectification Rotation R1 (left):

GergelySzabolcs

Hello. petercorke
In order to use neural network model with 14 shaves change the config file to :

"ai": {
    "blob_file": "/path/to/depthai/resources/nn/mobilenet-ssd/mobilenet-ssd.blob.sh14cmx14NCE1",
    "blob_file_config": "/path/to/depthai/resources/nn/mobilenet-ssd/mobilenet-ssd.json",
    "shaves" : 14,
    "cmx_slices" : 14,
    "NN_engines" : 1,
},

By default it's set to 7, that's why it's not working.

Brandon

Hi @petercorke ,

Sorry about the trouble here. I think I know what the issue is. In Gen2 (as a result of us supporting an arbitrary number of neural networks), the max number a SHAVES a neural network can currently use is 7.

So the neural blob that you have there is compiled for 14 SHAVES. We need to document this more clearly. And this is why the demo (Gen1 based) works but the Gen2 example does not, for this blob. And eventually we will allow Gen2 to use 14 SHAVES as well.

To recompile a given network to use 7 SHAVES instead, you can actually just use the python3 depthai_demo.py -cnn <cnn_name> -sh 7 -nce 1 and this will use our model compiler to generate the neural blob with these settings.

Then you can just snag the generated blobs from depthai/resources/nn/<cnn_name>/

Example below:

So then you can snag the 7-shave version and use it in Gen2.

Thanks and sorry for the confusion here. I didn't think about this incompatibility until your post.
-Brandon

Brandon

Correction. I misread. I was thinking you were doing Gen2 here, whereas you are actually having an issue with Gen1. We are in the process of updating all the getting started docs to use Gen2 instead, as it is way more flexible.

And I was wrong about the shave limitation on Gen2. It now automatically allocates resources and can use SHAVES for neural inference dynamically and automatically.

Could you try giving these examples a run instead? They show how to do a slew of things with the Gen2 pipeline builder:
https://github.com/luxonis/depthai-experiments/tree/master/gen2_examples

Thanks again,
Brandon

petercorke

Brandon thanks for the quick response, you must be deluged with questions. I ran the first demo just fine. Looks like I should have set that up in a fresh python environment, because after that setup, I can't run the gen1 demo that my question was about. But that demo did run following the tip from GergelySzabolcs .

petercorke

GergelySzabolcs thanks a lot, works just fine now. What's the suffix on the blob file? I though it was a hash, but if it is I don't know which one. Would be good to update the web page for minimal working code sample and explain that.

Brandon

petercorke Yes, good idea. We will do so. Thanks for circling back.