FCN and LSTM Based Computer Vision System for Recognition of Vehicle Type, License Plate Number, and Registration Country

Аннотация

We propose an advanced Automatic number-plate recognition (ANPR) system, which not only recognizes the number and the issuing state, but also the type and location of the vehicle in the input image. The system is based on a combination of existing methods, modifications to neural network architectures and improvements in the training process. The proposed system uses machine-learning approach and consists of three main parts: segmentation of input image by Fully Convolutional Network for localization of license plate and determination of vehicle type; recognition of the characters of the localized plate by a Maxout CNN and LSTM; determination of the state that has issued the license plate by a CNN. The training of these neural network models is accomplished using a manually labeled custom dataset, which is expanded with data augmented techniques. The resulting system is capable of localizing and classifying multiple types of vehicles (including motorcycles and emergency vehicles) as well as their license plates. The achieved precision of the localization is 99.5%. The whole number recognition accuracy is 96.7% and character level recognition accuracy is 98.8%. The determination of issuing state is precise in 92.8% cases.