Information Meaning of Entropy of Nonergodic Measures

Abstract

The limit frequency properties of trajectories of the simplest dynamical system generated by the left shift on the space of sequences of letters from a finite alphabet are studied. The following modification of the Shannon-McMillan-Breiman theorem is proved: for any invariant (not necessarily ergodic) probability measure μ on the sequence space, the logarithm of the cardinality of the set of all μ-typical sequences of length n is equivalent to nh(μ), where h(μ) is the entropy of the measure μ. Here a typical finite sequence of letters is understood as a sequence such that the empirical measure generated by it is close to μ (in the weak topology).