Skip to content

Official Pytorch implementations of TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition (IJCAI 2023)

License

Notifications You must be signed in to change notification settings

simplify23/TPS_PP

Repository files navigation

TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition

1682662695807

The official code of TPS_PP (IJCAI 2023) Paper Link

TPS++, an attention-enhanced TPS transformation that incorporates the attention mechanism to text rectification for the first time. TPS++ builds a more flexible content-aware rectifier, generating a natural text correction that is easier to read by the subsequent recognizer. This code is based on MMOCR 0.4.0 ( Documentation ) with PyTorch 1.6+.

Code List

  • NRTR + TPS_PP
  • CRNN + TPS_PP
  • ABINet-LV + TPS_PP

Installation

Please refer to Install Guide.

Get Started

Please see Getting Started for the basic usage of MMOCR 0.4.0.

Datasets

The specific configuration of the dataset for training and testing can be found here Dataset Document

testing 
├── mixture
│   ├── icdar_2013
│   ├── icdar_2015
│   ├── III5K
│   ├── ct80
│   ├── svt
│   ├── svtp

training
├── mixture
│   ├── Syn90k
│   ├── SynthText

Pretrained Models

Get the pretrained models from BaiduNetdisk(passwd:cd9r), GoogleDrive. checkpoint model in model/xxx/latest.pth, pre-train model in pre_train/xxx/latest.pth

Methods IIIT5K SVT IC13 IC15 SVTP CUTE AVG
NRTR + TPS_PP 96.3 94.6 96.6 85.7 89.0 92.4 92.4
NRTR + TPS_PP * 95.6 95.1 97.2 85.9 89.8 90.3 92.3

First, the model needs to be pre-trained using without TPS_PP (pre-train), and then trained end-to-end with a network that incorporates TPS_PP (checkpoint). * denotes the performance of the implemented code. checkpoint model in model/xxx/latest.pth, pre-train model in pre_train/xxx/latest.pth.

Train

Please refer to the training configuration Training Doc

NRTR+TPS++

Setp 1 : Download NRTR pre_train/nrtr/latest.pth in mmocr_ijcai/nrtr/latest.pth

#Step 2
PORT=1234 ./tools/dist_train.sh configs/textrecog/nrtr/nrtr_tps++.py ./ckpt/ijcai_nrtr_tps_pp 4 
          --seed=123456 --load-from=mmocr_ijcai/nrtr/nrtr_latest.pth

Testing

Please refer to the testing configuration Testing Doc

Acknowledgement

This code is based on MMOCR

Citation

If you find our method useful for your reserach, please cite

@article{zheng2023tps++,
  title={TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition},
  author={Zheng, Tianlun and Chen, Zhineng and Bai, Jinfeng and Xie, Hongtao and Jiang, Yu-Gang},
  journal={IJCAI},
  year={2023}
}

License

This project is released under the Apache 2.0 license.

About

Official Pytorch implementations of TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition (IJCAI 2023)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages