A probabilistically entropic mechanism of topical clusterisation along with thematic annotation for evolution analysis of meaningful social information of internet sources

Abstract

An approach to monitoring temporal evolution of thematic clusters with evaluating their relations on base of probability and entropy methods is presented. It allows to get a temporary map of nested topics with their short annotations, concerning a predetermined main theme. The methods of semantic analysis of texts to generate topics and to find the most emotive of them to reflect a social significance are used. The technology word2vec was implemented to determine the relation of topics and evaluate their proximity to the main theme.

To increase the usability the visualization of nested topics is realized on base of a WEB interface. The proposed approach complements well the popular software for analyzing big volumes of data such as Elasticsearch (search for thematically similar documents). Results of case study of analyzing the theme “AEROFLOT” on base of news corpus which consists of 3 million messages is presented.