Convergence of a multilayer perceptron to histogram Bayesian regression

The problem of enhancing the interpretability and consistency of Baysesian classifier solutions in approximating the empirical data by means of a multilayer perceptron is under consideration. Histogram regression preserves transparency and statistical interpretation but is limited by memory requirements ($O(n)$) and weak scalability, while a multilayer perceptron provides a memory efficient representation ($O(1)$)and high computational efficiency in combination with limited interpretability. The focus is on a unary learning scheme, when the training sample consists of examples in the same target class and additional background points which are uniformly distributed over a compact subset of the feature space. This approach enables one to treat each class separately and implement the failure mechanism outside the data support, which enhances the model reliability. It is proposed to consider the perceptron output as a consistent analogue of the histogram class interval induced by the linearity cells of the perceptron. It is proved that under the natural assumptions of regularity and controlled growth of architecture the output function of a multilayer perseptron is consistent and equivalent to a histogram estimator. Theoretical consistency is rigorously ðroved in the case of a fixed first layer, while numerical experiments confirm the applicability of the results to models all of whose layers are trained. Thus histogram interpretation ensures the statistical verification of the consistency of perceptron approximation and addscredibility to classification solutions in the framework of a unary model.

Multilayer perceptron, histogram regressions, piecewise linear activation functions, Bayesian classifier, consistency, asymptotic equivalence, VC-dimension, random hyperplanes, unary classification