SceneTextRecognition

Done for GeoAI Internship at GeoWorks, Singapore Land Authority.

Three different ways to Detect, Recognize or Spot Scene Text.

Three Models used:

1. OpenCV OCR
2. EAST (Tensorflow Implementation)
3. FOTS - Fast Oriented Text Spotting with a Unified Network

Please check the links for original repository/code.
I have made little modifications to all to suit my needs, especially for the FOTS code since it's used for my final deliverable

Please also check the links for training purposes, as the instructions below only cover testing. ☺️☺️☺️

Visual Studio Code and WSL Ubuntu were used for this Proof of Concept. 💻

💻1️⃣💻

Usage of OpenCV OCR (Test)

1. Clone GitHub Repository / Download & unzip ZIP file named ‘OpenCV OCR’
2. Activate Virtual Environment and go to the directory where the OpenCV OCR folder is
3. Install the required dependencies (requirements.txt)

pip install -r requirements.txt

4. Download the EAST text detection pb file and put it in the same folder as the py file.

Download the pb file from this link

5. Initiate Xming (to enable SSH X11 Forwarding), if you are using WSL Ubuntu also.
6. Run the command

python3 text_recognition_test.py --east frozen_east_text_detection.pb --image images/img6.jpg

💻2️⃣💻

Usage of EAST (Test)

1. Clone GitHub Repository / Download & unzip ZIP file named ‘EAST’
2. Activate Virtual Environment and go to the directory where the EAST is
3. Install the required dependencies (requirements_venv.txt)

pip install -r requirements_venv.txt

4. Download the pre-trained models and put it into a folder with the name of the checkpoint_path

• Models trained on ICDAR 2013 + ICDAR 2015 training data
• Resnet V1 50 provided by tensorflow slim

5. Run the command

python eval.py --test_data_path=images_test/ --gpu_list=0 --checkpoint_path=east_icdar2015_resnet_v1_50_rbox/ \--output_dir=result/ 

💻3️⃣💻

Usage of FOTS (Test)

1. Clone GitHub Repository / Download & unzip ZIP file named ‘FOTS’
2. Activate Virtual Environment and go to the directory where the FOTS is
3. Install the required dependencies (requirements_FOTS.txt)

pip install -r requirements_FOTS.txt

4. Download the pre-trained model and put it into a folder with the name of the checkpoint_path

• SynthText 6-epochs pre-trained model

5. Run the command

python3 main_test.py --gpu_list='0' --test_data_path=images_test/ --checkpoint_path=checkpoints/

Final Deliverable: Flask Application

Features of Flask Application:

• FOTS: Spot Scene Text in the picture
• Extract GeoTAGS (GPS Info/Coordinates) of the picture
• OneMap API: Reverse Geocoding to get Address of the picture
• OneMap API: Static Map of the Address of the picture

In the process of choosing the scene text recognition/spotting model (FOTS), I have chosen some criteria such as:

• Implementation uses Tensorflow and Python (familiarity)
• Able to produce rotated bounding boxes
• Able to SPOT scene text, meaning both detect and recognize scene text
• Model able to be trained using ICDAR Datasets

Usage

1. Usage is the same as Usage of FOTS (Test)
2. Run the command

python3 app_geo.py

Output

Thank you to the team at GeoWorks, Singapore Land Authority!