Learning the Regularization Operator for the Optical Flow Problem

Abstract

The task of displacement estimation for frames of a video sequence is considered. A new convolutional neural network architecture for the optical flow problem is proposed. The method is based on learning the regularization operator for a fast optimization method. The proposed method has low computational complexity and memory footprint at test time. The neural network architecture is based on unrolling iterations of a fast primal-dual method as layers of a convolutional neural network. Iterations of the optimization method are represented as convolutions with filters that are trained on ground truth data by backpropagation. A real-time implementation using graphics processing units is proposed. Experimental results demonstrate an improved quality of the optical flow field as compared to the optimization method based on a fixed regularization operator.