Modifying Texture of a Photograph Object, with the Use of Neural Networks Ensemble

Abstract

In this paper, we consider a problem-solving technique of the texture changing of an object in a photograph. The task in hand is relevant in the field of intelligent image processing and has a number of practical applications. The solutions to this problem have been proposed in a number of works devoted to the neural network style transfer approach, but they have a number of limitations. Examples of the limitations are as follows: the texture transfer selectivity absence (the image is changed entirely), the target texture distortion with the heterogeneity of the original one, the initial illumination distortion of the object, and the absence of the photographic realism of the resulting image.

To solve the problems mentioned above, in this paper we propose a sequential image processing with the use of several neural networks types: segmental, stylizing and generative-adversarial (GAN) ones. The reliable transfer problem of an object illumination is proposed to be solved by the joint work of GAN and methods that do not use the neural network approach.

In the context of this paper we developed an algorithm that allows solving the texture transferring task with completely or partially elimination of the listed problems of classical methods. Its high quality of work is shown with the maintaining productivity acceptable for common use. Demonstration of the algorithm is performed on the task of a virtual furniture dust covers fitting (sofas, armchairs). In addition to the algorithm itself, this work includes an enumeration of some heuristics and limitations stated during its implementation and application.