A semantic hybrid approach based on grouping adjacent regions and a combination of multiple descriptors and classifiers for automatic image annotation

Abstract

A large percentage of photos on the Internet cannot be reached by search engines because of the semantic gap due to the absence of textual meta-data. Despite of decades of research, neither model based approaches can provide quality annotation to images. Many segmentation algorithms use a low-level predicates to control the homogeneity of the regions. So, the resulting regions are not always being semantically compact. The first proposed approach to resolve this problem is to regroup the adjacent region of image. Many features extraction method and classifiers are also used singly, with modest results, for automatic image annotation. The second proposed approach is to select and combine together some efficient descriptors and classifiers. This document provides a hybrid semantic annotation system that combines both approaches in hopes of increasing the accuracy of the resulting annotations. The color histograms, Texture, GIST and invariant moments, used as features extraction methods, are combined together with multi-class support vector machine, Bayesian networks, Neural networks and nearest neighbor classifiers, in order to annotate the image content with the appropriate keywords. The accuracy of the proposed approach is supported by the good experimental results obtained from two image databases (ETH-80 and coil-100 databases).