Images not coming from the Oak camera

Nithin

Hi,
While running the 01_rgb_preview.py, i get no response. There is no error and at the same time there is no pop up window showing up either.
When i end the run with CTRL+C, i can see that the code is at in_rgb = q_rgb.get() # blocking call, will wait until a new data has arrived.
I am using the Oak 1 camera
Your help will be much appreciated.

Brandon

Hi Nithin ,

Which OS are you on? We have seen this before on macOS with specific versions of OpenCV:
https://discuss.luxonis.com/d/95-video-preview-window-fails-to-appear-on-macos

I'm wondering if that is what is going wrong here.

Thoughts?

Thanks,
Brandon

Nithin

Brandon
Thanks for your prompt response.
I am on Ubuntu 18.04.5
open cv version is 4.5.1

I checked the earlier link you shared. That does not seem to be my issue. I am able to read an image from disk and display in a window in the same virtual environment. It is stuck on “in_rgb = q_rgb.get() #blocking call, will wait until a new data has arrived”

Brandon

Strange. Could you try running the script in USB2 mode to check if there is some USB3 communication issue?

To do so could change line 24 from with dai.Device(pipeline) as device: to with Device(pipeline, usb2Mode=True) as device:

This will force USB2 connectivity, which should eliminate (most) chance(s) that this is a USB3 communication issue. A caveat: Some hosts still are convinced about using USB3 if a USB3 cable is plugged in, and still run into issues even in this mode. But it's very rare.

Thoughts?

Thanks,
Brandon

Nithin

Hi Brandon,
Tried the change. Unfortunately the same issue persists. Could it be a device issue ?
" in_rgb = q_rgb.get() # blocking call, will wait until a new data has arrived"
KeyboardInterrupt

Nithin

Tried a clean install in another system. While running cmake, i get the following error:
"CMake Error at cmake/pybind11-mkdoc.cmake:75 (file):
file does not recognize sub-command TOUCH_NOCREATE"
Tried running cmake in the old system, similar error there also.

Luxonis-Alex

Could you give a try running with Gen1:

git clone https://github.com/luxonis/depthai
cd depthai
./install_requirements.py
./depthai_demo.py

Also try here:

./depthai_demo.py -fusb2

If it still fails, the logs from running this script may help to pinpoint the problem.

Nithin

Installation was all successful. No errors or warnings.
While running demo, no pop up came up. Copy pasting the output here:

No calibration file. Using Calibration Defaults.
Using depthai module from: /home/game1/oakvenv/venv/lib/python3.8/site-packages/depthai.cpython-38-x86_64-linux-gnu.so
Depthai version installed: 1.0.0.0
Compiling model for 14 shaves, 14 cmx_slices and 1 NN_engines
model_compilation_target: cloud
################|| Downloading models ||################

========== Retrieving /home/game1/oakvenv/depthai/model_compiler/downloads/mobilenet-ssd/FP16/mobilenet-ssd.xml from the cache

========== Retrieving /home/game1/oakvenv/depthai/model_compiler/downloads/mobilenet-ssd/FP16/mobilenet-ssd.bin from the cache

################|| Post-processing ||################

It stays here and nothing happens further. When i give keyboard interrupt, the below happens:

################|| Post-processing ||################

^CTraceback (most recent call last):
File "./depthai_demo.py", line 418, in <module>
dai.startLoop()
File "./depthai_demo.py", line 47, in startLoop
configMan = DepthConfigManager(args)
File "/home/game1/oakvenv/depthai/depthai_helpers/config_manager.py", line 26, in init
self.generateJsonConfig()
File "/home/game1/oakvenv/depthai/depthai_helpers/config_manager.py", line 192, in generateJsonConfig
blobMan = BlobManager(self.args, self.calc_dist_to_bb, shave_nr, cmx_slices, NCE_nr)
File "/home/game1/oakvenv/depthai/depthai_helpers/config_manager.py", line 377, in init
self.blob_file = self.compileBlob(self.args['cnn_model'], self.args['model_compilation_target'])
File "/home/game1/oakvenv/depthai/depthai_helpers/config_manager.py", line 431, in compileBlob
ret = download_and_compile_NN_model(nn_model, model_zoo_folder, shave_nr_opt, cmx_slices_opt, NCE_nr, outblob_file, model_compilation_target)
File "/home/game1/oakvenv/depthai/model_compiler/model_compiler.py", line 180, in download_and_compile_NN_model
result = myriad_compile_model_cloud(xml=xml_path, bin=bin_path, shaves = shaves, cmx_slices=cmx_slices, nces=nces, output_file=output_location)
File "/home/game1/oakvenv/depthai/model_compiler/model_compiler.py", line 109, in myriad_compile_model_cloud
response = requests.post(url, data=payload, files=files, params=params)
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/home/game1/oakvenv/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.8/http/client.py", line 1230, in request
self.send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1276, in send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1225, in endheaders
self.send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1043, in _send_output
self.send(chunk)
File "/usr/lib/python3.8/http/client.py", line 965, in send
self.sock.sendall(data)
KeyboardInterrupt

Nithin

Same thing happens when i use -fusb2 also

Nithin

On the same usb port i am able to use an Intel realsense camera and that works. That also requires usb 3.0

Luxonis-Alex

Could you let the command run for more time, it could take 1..2 minutes, or even more (depending on internet connection speed), to prepare the NN model blob initially.

You could try adding an option that will process a smaller blob: -cnn face-detection-retail-0004
So:
./depthai_demo.py -cnn face-detection-retail-0004

./depthai_demo.py -cnn face-detection-retail-0004 -fusb2

Nithin

changing it to a smaller blob helped in progressing further.
Now i am getting the following output:
No calibration file. Using Calibration Defaults.
Using depthai module from: /home/game1/oakvenv/venv/lib/python3.8/site-packages/depthai.cpython-38-x86_64-linux-gnu.so
Depthai version installed: 1.0.0.0
Compiled mode found: compiled for 14 shaves, 14 cmx_slices and 1 NN_engines
FORCE USB2 MODE
XLink initialized.
Sending internal device firmware
Successfully connected to device.
Usb speed : High/480Mbps
Mx serial id : 14442C1031A492CD00
Loading config file
Attempting to open stream config_d2h
watchdog started
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: invalid / unprogrammed
['metaout', 'previewout']
Available streams: ['meta_d2h', 'left', 'right', 'disparity', 'depth', 'metaout', 'previewout', 'jpegout', 'video', 'object_tracker', 'rectified_left', 'rectified_right', 'color']
/home/game1/oakvenv/depthai/resources/nn/face-detection-retail-0004/face-detection-retail-0004.json
depthai: Using calibration from stored in EEPROM
RGB camera (IMX378) is not detected on board!
WARNING: NN inference was requested on RGB camera (IMX378), defaulting to right stereo camera (OV9282)!
config_h2d json:
{"board":{"calib_data":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"mesh_left":[0.0],"mesh_right":[0.0]},"load_inBlob":true,"pipeline":{"streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"NCEs":1,"NN_config":{"NN_family":"mobilenet","confidence_threshold":0.5,"output_format":"detection"},"blob0_size":1332032,"blob1_size":0,"calc_dist_to_bb":true,"camera_input":"right","cmx_slices":14,"keep_aspect_ratio":true,"num_stages":1,"shaves":14},"app":{"enable_reconfig":true,"sync_sequence_numbers":false,"sync_video_meta_streams":false,"usb_chunk_KiB":64},"board":{"clear-eeprom":false,"left_fov_deg":71.86000061035156,"left_to_rgb_distance_m":0.03750000149011612,"left_to_right_distance_m":0.07500000298023224,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":68.7938003540039,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":true},"camera":{"mono":{"fps":30.0,"resolution_h":400,"resolution_w":640},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"depth_limit_mm":10000,"lr_check":false,"median_kernel_size":7,"padding_factor":0.30000001192092896,"warp_rectify":{"edge_fill_color":0,"mirror_frame":true,"use_mesh":false}},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
size of input string json_config_obj to config_h2d is ->1653
size of json_config_obj that is expected to be sent to config_h2d is ->1048576
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1048576 bytes to config_h2d
!!! XLink write successful: config_h2d (1048576)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 1332032
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #2!
Writing 1332032 bytes to inBlob
!!! XLink write successful: inBlob (1332032)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /home/game1/oakvenv/depthai/resources/nn/face-detection-retail-0004/face-detection-retail-0004.blob.sh14cmx14NCE1
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #3!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
Input layer :
Name: data
Index: 0
Element type: uint8
Element size: 1byte
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 3, Height : 300, Width : 300]

Output layer :
Name: detection_out
Index: 0
Element type: float16
Element size: 2 bytes
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 1, Height : 200, Width : 7]

This keeps getting repeated.
Same output i get with and without -fusb2

(venv) game1@TAILWIND1:_{/oakvenv/depthai$} ./depthai_demo.py -cnn face-detection-retail-0004 -fusb2
No calibration file. Using Calibration Defaults.
Using depthai module from: /home/game1/oakvenv/venv/lib/python3.8/site-packages/depthai.cpython-38-x86_64-linux-gnu.so
Depthai version installed: 1.0.0.0
Compiled mode found: compiled for 14 shaves, 14 cmx_slices and 1 NN_engines
FORCE USB2 MODE
XLink initialized.
Sending internal device firmware
Successfully connected to device.
Usb speed : High/480Mbps
Mx serial id : 14442C1031A492CD00
Loading config file
Attempting to open stream config_d2h
watchdog started
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: invalid / unprogrammed
['metaout', 'previewout']
Available streams: ['meta_d2h', 'left', 'right', 'disparity', 'depth', 'metaout', 'previewout', 'jpegout', 'video', 'object_tracker', 'rectified_left', 'rectified_right', 'color']
/home/game1/oakvenv/depthai/resources/nn/face-detection-retail-0004/face-detection-retail-0004.json
depthai: Using calibration from stored in EEPROM
RGB camera (IMX378) is not detected on board!
WARNING: NN inference was requested on RGB camera (IMX378), defaulting to right stereo camera (OV9282)!
config_h2d json:
{"board":{"calib_data":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"mesh_left":[0.0],"mesh_right":[0.0]},"load_inBlob":true,"pipeline":{"streams":[{"name":"metaout"},{"name":"previewout"}]},"ai":{"NCEs":1,"NN_config":{"NN_family":"mobilenet","confidence_threshold":0.5,"output_format":"detection"},"blob0_size":1332032,"blob1_size":0,"calc_dist_to_bb":true,"camera_input":"right","cmx_slices":14,"keep_aspect_ratio":true,"num_stages":1,"shaves":14},"app":{"enable_reconfig":true,"sync_sequence_numbers":false,"sync_video_meta_streams":false,"usb_chunk_KiB":64},"board":{"clear-eeprom":false,"left_fov_deg":71.86000061035156,"left_to_rgb_distance_m":0.03750000149011612,"left_to_right_distance_m":0.07500000298023224,"name":"","override-eeprom":false,"revision":"","rgb_fov_deg":68.7938003540039,"stereo_center_crop":false,"store-to-eeprom":false,"swap-left-and-right-cameras":true},"camera":{"mono":{"fps":30.0,"resolution_h":400,"resolution_w":640},"rgb":{"fps":30.0,"resolution_h":1080,"resolution_w":1920}},"depth":{"depth_limit_mm":10000,"lr_check":false,"median_kernel_size":7,"padding_factor":0.30000001192092896,"warp_rectify":{"edge_fill_color":0,"mirror_frame":true,"use_mesh":false}},"ot":{"confidence_threshold":0.5,"max_tracklets":20}}
size of input string json_config_obj to config_h2d is ->1653
size of json_config_obj that is expected to be sent to config_h2d is ->1048576
Attempting to open stream config_h2d
Successfully opened stream config_h2d with ID #1!
Writing 1048576 bytes to config_h2d
!!! XLink write successful: config_h2d (1048576)
Closing stream config_h2d: ...
Closing stream config_h2d: DONE.
Creating observer stream host_capture: ...
Attempting to open stream host_capture
Successfully opened stream host_capture with ID #0!
Creating observer stream host_capture: DONE.
Read: 1332032
Attempting to open stream inBlob
Successfully opened stream inBlob with ID #2!
Writing 1332032 bytes to inBlob
!!! XLink write successful: inBlob (1332032)
Closing stream inBlob: ...
Closing stream inBlob: DONE.
depthai: done sending Blob file /home/game1/oakvenv/depthai/resources/nn/face-detection-retail-0004/face-detection-retail-0004.blob.sh14cmx14NCE1
Attempting to open stream outBlob
Successfully opened stream outBlob with ID #3!
Closing stream outBlob: ...
Closing stream outBlob: DONE.
Input layer :
Name: data
Index: 0
Element type: uint8
Element size: 1byte
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 3, Height : 300, Width : 300]

Output layer :
Name: detection_out
Index: 0
Element type: float16
Element size: 2 bytes
Offset: 0 byte
Dimensions: [Batch : 1, Channel : 1, Height : 200, Width : 7]

CNN to depth bounding-box mapping: start(0, 0), max_size(0, 0)
Host stream start:metaout
Opening stream for read: metaout
Attempting to open stream metaout
Successfully opened stream metaout with ID #1!
Starting thread for stream: metaout
Host stream start:previewout
Opening stream for read: previewout
Attempting to open stream previewout
Started thread for stream: metaout
Successfully opened stream previewout with ID #3!
Starting thread for stream: previewout
depthai: INIT OK!
Started thread for stream: previewout
E: [global] [ 681763] [python3] addEvent:262 Condition failed: event->header.flags.bitField.ack != 1
E: [global] [ 681763] [python3] addEventWithPerf:276 addEvent(event) method call failed with an error: 3
E: [global] [ 681763] [python3] XLinkReadData:156 Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
Closing stream metaout: ...
E: [global] [ 681763] [python3] addEvent:262 Condition failed: event->header.flags.bitField.ack != 1
E: [global] [ 681763] [python3] addEventWithPerf:276 addEvent(event) method call failed with an error: 3
E: [global] [ 681763] [python3] XLinkReadData:156 Condition failed: (addEventWithPerf(&event, &opTime))
Device get data failed: 7
E: [global] [ 681763] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ
Closing stream metaout: DONE.
Closing stream previewout: ...
Thread for metaout finished.

E: [xLink] [ 681763] [Scheduler00Thr] sendEvents:1027 Event sending failed
E: [global] [ 681763] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [ 681764] [Scheduler00Thr] sendEvents:1027 Event sending failed
Closing stream previewout: DONE.
Thread for previewout finished.
watchdog triggered
=== New data in observer stream host_capture, size: 4
Writing 4 bytes to host_capture
E: [global] [ 690043] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [ 690043] [Scheduler00Thr] sendEvents:1027 Event sending failed
!!! XLink write successful: host_capture (4)
Stopping threads: ...
Stopping threads: DONE 0.000s.
Closing all observer streams: ...
Closing stream host_capture: ...
Closing stream host_capture: DONE.
Closing all observer streams: DONE.
Reseting device: 0.
E: [global] [ 690043] [Scheduler00Thr] dispatcherEventSend:53 Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [ 690043] [Scheduler00Thr] sendEvents:1027 Event sending failed
Reseting: DONE.
XLink already initialized.
Sending internal device firmware
Successfully connected to device.
Usb speed : High/480Mbps
Mx serial id : 14442C1031A492CD00
Loading config file
Attempting to open stream config_d2h
Successfully opened stream config_d2h with ID #0!
Closing stream config_d2h: ...
Closing stream config_d2h: DONE.
EEPROM data: invalid / unprogrammed
/home/game1/oakvenv/depthai/resources/nn/face-detection-retail-0004/face-detection-retail-0004.json
depthai: Using calibration from stored in EEPROM
RGB camera (IMX378) is not detected on board!
WARNING: NN inference was requested on RGB camera (IMX378), defaulting to right stereo camera (OV9282)!

Nithin

Device gets heated up so i am guessing something is working !

Luxonis-Alex

Thanks!

So it makes sense, this is the useful log:

RGB camera (IMX378) is not detected on board!
WARNING: NN inference was requested on RGB camera (IMX378), defaulting to right stereo camera (OV9282)!

And since this is an OAK-1, there are no other cameras than the main RGB (IMX378).

Wondering what could have happened, there is a chance that the RGB camera connector snapped off place, due to a potential mechanical shock, could have been during shipping...

Would you want to try open the metal enclosure, and give it a check?

Luxonis-Alex

Nithin

Opened the camera up. The connector was fixed properly only. Just to be sure, I removed it and fixed it back also. When i plug in the camera, there are 2 LED lights that come on too.
But the same error exists after re-fixing the connector too. So i am guessing it is a hardware issue ?
I had ordered another camera too, but that is stuck in customs for nearly a month now and Fedex is not able to get it through for some some reason.

Brandon

Hi Nithin we can send you a replacement for the one with the issue and also for the one that's stuck. Could you shoot us an email at support@luxonis.com?

We have seen this sort of issue once before, and we think it is a result of a thermal interface material (TIM) in the design that is too rigid, causing the PCBA to flex during assembly.

We have since revised the design to use a softer TIM.

Thanks and sorry about the trouble here!
-Brandon

Nithin

Hi Brandon,
I received the one that was stuck in customs today. And it took all less than 2 minutes to get going !!! It works !
So your suspicion is most likely correct - a hardware issue on the other camera. I will shoot an email to support as suggested by you for the replacement.
Thanks a ton for the support 🙂

Brandon

Nithin Thank you.

And as an update, your replacement should have now already shipped. Please ping us if you have not received a tracking number yet. :-)