CUSTOM OCR TRAINING

Thanks @demoacct01 . To your question demoacct01 , we do have an in-progress example but it is crashing on some of the warp/re-shape when taking the EAST bounding boxes and feeding them through WARP/deWARP on the Myriad X.

We just fixed the crash (got the message that it was fixed just now, actually). So the last step is the OCR output decoding. But the whole pipeline is actually now running.

Here's the Github issue:
https://github.com/luxonis/depthai/issues/124

And here's the WIP PR for the whole demo flow:
https://github.com/luxonis/depthai-experiments/pull/26

We will likely update the PR shortly with the whole flow working, excepting the host-side decoding of the OCR. We're discussing internally the timing on that.

Thoughts?

Thanks again,
Brandon

Hi demoacct01 ,

Sorry about the trouble. The OCR is likely the slow part. The detection itself is a lot faster:

This example is still a work in progress. Luxonis-Alex is finishing it up now. We actually lost the engineer who was working on this (another one to Amazon!) so it got delayed as a result.

This PR is what will allow for the flow to be on DepthAI/OAK directly:
https://github.com/luxonis/depthai-shared/pull/16

It's what took a lot of time/debugging here. So now that this is out of the way, hopefully we will have the whole flow working well soon.

Thanks and sorry about the delay.
-Brandon

Hi @demoacct01 ,
The demo in the Youtube video is only running the text detection (bounding boxes) network, whereas the Gen2 one https://github.com/luxonis/depthai-experiments/pull/26 is running both text detection and text recognition, passing each cropped bounding box to the recognition model. So depending on the number of detections, it may be slower.

I'm working now on this Gen2 example, moving the rotated cropping from host (OpenCV warp) to run on device (with the recent rotated cropping/rescale added in ImageManip node), and will add a decoder for the recognition model outputs as well: https://docs.openvinotoolkit.org/2020.1/_models_intel_text_recognition_0012_description_text_recognition_0012.html

Hello! I tried to use gen2-ocr project from GitHub
https://github.com/luxonis/depthai-experiments/tree/master/gen2-ocr
and got several errors. Using depthai version from requirements caused a warning
"DeprecationWarning: setCamId() is deprecated, use setBoardSocket() instead."

Switching between depthai versions made other warnings. There are some of them:

• Creating depthai.Pipeline() has no Constructor
• nn doesn't have function nn.passthrough (line 25)
• silent crashes while compiling
  I use OAK-D with Windows-10. What am I doing wrong?

Hi asidsunrise ,

Sorry about the delay (been inundated with Brexit/pandemic shipping/logistics problems in Europe). This is quite odd. So just to confirm, when trying out that example, you ran:

python -m pip install -r requirements.txt

And then:

python3 main.py

And it is still giving those errors?

If so, is the installation of requirements throwing errors? As from what you shared, it is seeming that the API being used is not compatible with the example.

Thoughts?

Thanks,
Brandon

Brandon If so, is the installation of requirements throwing errors? As from what you shared, it is seeming that the API being used is not compatible with the example.

You are right, I've got errors with the requirements installed. I think the problem relates to using wrong API or the right API was modified. But I tried another versions of depthai too and it didn't work. There are screenshots applied

Strange. Let me see if I can reproduce what you are seeing here. Will circle back shortly.

I can't seem to replicate the version error here. For me it is just running:

So here are the commands I did to get it to run:

cd ~/depthai-experiments/gen2-ocr
git checkout master
git pull
python3 -m pip install -r requirements.txt
python3 main.py 

Could you try doing these in sequence? And actually, to make sure that the other version of depthai installed, could you do this sequence:

python3 -m pip uninstall depthai
cd ~/depthai-experiments/gen2-ocr
git checkout master
git pull
python3 -m pip install -r requirements.txt
python3 main.py 

I just repeated that whole sequence to be sure, and it works. I'm wondering if something has locked an incorrect version of depthai on the system. If this doesn't work, would it be OK to do a TeamViewer session to debug?

If so, please shoot us an email at support@luxonis.com to coordinate the session.

Thanks - and sorry about the trouble.
-Brandon

asidsunrise Thanks. Very strange. Perhaps our CI/CD didn't properly build for Windows for this example. Will be curious to find out.

Brandon Well, I tried to run it on my Macbook and got the same error. The re-installing depthai version didn't help. If it could help to find a problem it will be OK to use TeamViewer, but I'm not sure it is possible in the next 2-3 days.

Hi, same issue on my iMac under Catalina (macOS 10.15.7)...

Thanks and sorry about the delay, both. So could you please send an email to support@luxonis.com to facilitate a teamviewer to see what is happening here?

I'll try this again now on my iMac as well.

Thanks and sorry again about the delay (so far behind on shipping issues in Europe),
Brandon

Hi @Will ,

So I was able to replicate this with the version that was checked out on my iMac. Steps:

cd ~/depthai-experiments/gen2-ocr 
python3 -m pip install -r requirements.txt

Which produced:

Looking in indexes: https://pypi.org/simple, https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local
Requirement already satisfied: numpy==1.19.5 in /Users/leeroy/Library/Python/3.9/lib/python/site-packages (from -r requirements.txt (line 1)) (1.19.5)
Requirement already satisfied: opencv-python==4.5.1.48 in /Users/leeroy/Library/Python/3.9/lib/python/site-packages (from -r requirements.txt (line 2)) (4.5.1.48)
Collecting depthai==0.0.2.1+e88c98c8a7681dd16cd754a832df1165380ac305
  Using cached https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local/depthai/depthai-0.0.2.1%2Be88c98c8a7681dd16cd754a832df1165380ac305-cp39-cp39-macosx_10_9_x86_64.whl (4.3 MB)
Installing collected packages: depthai
  Attempting uninstall: depthai
    Found existing installation: depthai 0.4.1.1
    Uninstalling depthai-0.4.1.1:
      Successfully uninstalled depthai-0.4.1.1
Successfully installed depthai-0.0.2.1+e88c98c8a7681dd16cd754a832df1165380ac305

And then I ran:

python3 main.py

Which resulted in:

/Users/leeroy/depthai-experiments/gen2-ocr/main.py:15: DeprecationWarning: setCamId() is deprecated, use setBoardSocket() instead.
  colorCam.setCamId(0)
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: XLink write error, error message: X_LINK_ERROR
zsh: abort      python3 main.py

I am now updating the repo with git pull on the master branch to check if this improves it.

Same error:

python3 main.py                           
/Users/leeroy/depthai-experiments/gen2-ocr/main.py:15: DeprecationWarning: setCamId() is deprecated, use setBoardSocket() instead.
  colorCam.setCamId(0)
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: XLink write error, error message: X_LINK_ERROR
zsh: abort      python3 main.py

So I am checking with the team now. The fact that I can reproduce this is good as I think we will now be able to fix it quickly.

Thanks and sorry about the issues here.

-Brandon

Dear Brandon:

we are using OAK-1 with EAST with text_recognition_0012 as demo video shown:


and source code below :

So far we are satisfy the

we have a request is we want to add ROI function and combine with source code， then it could judge OCR ok or ng only inside of ROI , of course we can adjust ROI by execute CMD .
Please assist us to realize the above function ,

all the best~~

Hello king , your google drive video/source code can't be accessed (it says that we have to request the access). Regarding the ROI, from my understanding you would like to specify the ROI and run text detection/OCR in that region?

Dear erik:

Yes~_you are right
as attached picture content is using text_recognition_0012 results that you couldn't seen by google video:

As we know EAST OCR is 1080p resolution and getting all of text of whole paper.
but our customer want only setting their own ROI and could adjust size of ROI ,finally running OCR inside ROI.

for example like COGNEX OCR (shown red ROI):

We have three ideas:

1. We would like to use IMREAD_UNCHANGE to cutoff picture ,and imshow
   

2. Change the aspect ratio (stretch the image)
   Apply letterboxing to the image,
   as below:
   https://docs.luxonis.com/projects/api/en/latest/tutorials/maximize_fov/

3. SpatialLocationCalculator
   to create ROI and combine with text_recognition
   https://docs.luxonis.com/projects/api/en/latest/components/nodes/spatial_location_calculator/

Please assist us which method could be match our goal??

all the best~~