Classification of a scanned document type using the dynamic time warping method

Cover Page

Cite item

Full Text

Abstract

The work addresses an important problem in the field of automatic document image recognition: determining the type of a scanned document from a predefined set of possible types. The proposed document classification method compares parallel projections of the input image with reference projections of templates from the target set, which can be generated using just a few document image samples. The matching is performed using a dynamic time warping algorithm. The classification method requires neither prior binarization of the sample, nor keyword extraction or recognition, nor detection of geometric primitives. However, it does require preliminary image deskewing. Experiments were conducted on a manually normalized dataset of business documents comprising eight distinct types, achieving a classification accuracy of 99.79%. For the same images normalized automatically, the accuracy reached 99.76%. For the document type with the largest average image size (2479х3589 pxs), the average processing time is 12.31±1.53 ms on a PC with an AMD Ryzen 5 5600X CPU, 64GB RAM.

About the authors

T. R. Maximova

Smart Engines Service LLC

Email: t.maksimova@smartengines.com
Topics of interest: document type classification in images, document recognition systems. Moscow, Russia

P. V. Bezmaternykh

Smart Engines LLC; Federal Research Center «Computer Science and Control» of Russian Academy of Sciences

Email: bezmaternyh@isa.ru
Topics of interest: document image analysis. Moscow, Russia; Moscow, Russia

References

  1. Arlazarov V.V., Andreeva E.I., Bulatov K.B., Nikolaev D.P., Petrova O.O., Savelev B.I., Slavin O.A. Document image analysis and recognition: A survey. Computer Optics. 2022; 46(4):567–589. doi: 10.18287/2412-6179-CO-1020.
  2. Skoryukina N.S., Arlazarov V.V., Nikolaev D.P. and Faradjev I.A. "Efficient Location and Identification of Documents in Images," United States Patent. Patent № US11574492В2, 02.09.2020. Р. 1-25, 2023.
  3. Awal A.M., Ghanmi N., Sicre R., Furon T. Complex document classification and localization application on identity document images. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). 2017; 426–431. IEEE. doi: 10.1109/ICDAR.2017.77.
  4. Gayer A.V., Arlazarov V.V. Muldt: Multilingual ultra-lightweight document text detection for embedded devices. IEEE Access. 2024; 12:170530–170540. doi: 10.1109/ACCESS.2024.3474616.
  5. Bahi H.E., Zatni A. Text recognition in document images obtained by a smartphone based on deep convolutional and recurrent neural network. Multimedia Tools and Applications. 2019; 78(18):26453–26481. doi: 10.1007/s11042-019-07855-z.
  6. Slavin O.A. and Fedorov G.O. "Ob ispolzovanii shtrikh-kodirovaniya i spetsializirovannykh ustroystv v korporativnom elektronnom dokumentooborote," Trudy ISA RAN (Proceedings of ISA RAS), vol. 4, pp. 185-197, 2003.
  7. Shengnan Z., Shanlei Y., Lianqiang N. Automatic recognition method for checkbox in data form image. In 2014 Sixth International Conference on Measuring Technology and Mechatronics Automation. 2014; page 159–162. IEEE. doi: 10.1109/ICMTMA.2014.42.
  8. ConsultantPlus. What documents may be required when applying for a job? 2024. (In Russ.) URL: https://www.consultant.ru/edu/student/consultation/dokumenty_ustroystvo_na_rabotu/.
  9. Seifollahi S., Piccardi M., Jolfaei A. An embedding-based topic model for document classification. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 20(3):1–13. doi: 10.1145/3431728.
  10. Xiao Y., Cho K. Efficient character-level document classification by combining convolution and recurrent layers. arXiv. 2016. doi: 10.48550/arXiv.1602.00367.
  11. Postnikov V.V. "Avtomaticheskaya identifikatsiya i raspoznavanie strukturirovannykh dokumentov". Dissertatsiya, pp. 1-126, 2001.
  12. Skoryukina N., Arlazarov V., Nikolaev D. Fast method of ID documents location and type identification for mobile and server application. In 2019 International Conference on Document Analysis and Recognition (ICDAR). 2019; pages 850–857. IEEE. doi: 10.1109/ICDAR.2019.00141.
  13. Open Source Computer Vision. Feature detection and description. URL:https://docs.opencv.org/4.x/db/d27/tutorial_py_table_of_contents_feature2d.html.
  14. Skoryukina N.S., Tropin D.V., Shemiakina Y.A., Arlazarov V.V. Document localization and classification as stages of a document recognition system. Pattern Recognit. Image Anal. 2023; 33(4):699–716. doi: 10.1134/S1054661823040430.
  15. Hu J., Kashi R., Wilfong G. Document classification using layout analysis. In Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99, pages 556–560. IEEE, 1999. doi: 10.1109/DEXA.1999.795245.
  16. Postnikov V.V. Formalnyy podkhod k zadache identifikatsii graficheskikh obrazov strukturirovannykh dokumentov. ITiVS, (4):280–299, 1999.
  17. Rusin˜ol M., Frinken V., Karatzas D., Bagdanov A. D., Llado´s J. Multimodal page classification in administrative document image streams. International Journal on Document Analysis and Recognition (IJDAR). 2014; 17(4):331–341. doi: 10.1007/s10032-014-0225-8.
  18. Bezmaternykh P., Nikolaev D., Postnikov V. Metod identifikatsii tipa dokumenta po strukture proektsiy ego izobrazheniya na koordinatnye osi. ITaS. 2008; с. 498–501, 2008. ISBN 978-59-01158-08-0.
  19. Povolotskiy М.А., Kuznetsova Е.G., Utkin N.V., Nikolaev D.P. Segmentation of vehicle registration plates based on dynamic time warping. Sensory systems. 2018; 32(1):50–59. doi: 10.7868/S0235009218010080.
  20. Sakoe H., Chiba S. Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing. 1(26):43–49. doi: 10.1109/TASSP.1978.1163055.
  21. Hull J.J. Document image skew detection: survey and annotated bibliography. World Scientific. 1998. Р. 40–64. doi: 10.1142/9789812797704_0003.
  22. Bezmaternykh P.V. Text Image Normalization Using Fast Hough Transform. ITiVS. 2024; (4):3–16. doi: 10.14357/20718632240401.
  23. Bezmaternykh P.V., Nikolaev D.P., Arlazarov V.L. High-performance digital image processing. Pattern Recognit. Image Anal. 2023; 33(4):743–755. doi: 10.1134/S1054661823040090.
  24. Brady M.L., Yong W. Fast parallel discrete approximation algorithms for the radon transform. In Proceedings of the fourth annual ACM symposium on Parallel algorithms and architectures. pages 91–99. ACM. doi: 10.1145/140901.140911.
  25. Gonzalez R.C., Woods R.E., Masters B.R. Digital image processing, third edition. 14(2):029901.
  26. Aliev М.А., Kunina I.А., Nikolaev D.P., Polevoy D.V. On the practical aspects of computing the Hough image by the Brady-Yong algorithm. Informatsionnye protsessy. 2023; 23(2):250–273. doi: 10.53921/18195822_2023_23_2_250.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).