Quadratic Programming Optimization with Feature Selection for Nonlinear Models
- Authors: Isachenko R.V.1,2, Strijov V.V.1,3
-
Affiliations:
- Moscow Institute of Physics and Technology (State University)
- Skolkovo Institute of Science and Technology
- A.A. Dorodnicyn Computing Centre
- Issue: Vol 39, No 9 (2018)
- Pages: 1179-1187
- Section: Part 1. Special issue “High Performance Data Intensive Computing” Editors: V. V. Voevodin, A. S. Simonov, and A. V. Lapin
- URL: https://journals.rcsi.science/1995-0802/article/view/203085
- DOI: https://doi.org/10.1134/S199508021809010X
- ID: 203085
Cite item
Abstract
The paper is devoted to the problem of constructing a predictive model in the high-dimensional feature space. The space is redundant, there is multicollinearity in the design matrix columns. In this case the model is unstable to changes in data or in parameter values. To build a stable model, the authors solve the dimensionality reduction problem for the feature space. It is proposed to use feature selection methods during parameter optimization process. The idea is to select the active set of model parameters which have to be optimized in the current optimization step. Quadratic programming feature selection is used to find the active set of parameters. The algorithm maximizes the relevance of model parameters to the residuals and makes them pairwise independent. Nonlinear regression and logistic regression models are investigated. We carried out the experiment to show how the proposed method works and compare it with other methods. The proposed algorithm achieves the less error and greater stability with comparison to the other methods.
About the authors
R. V. Isachenko
Moscow Institute of Physics and Technology (State University); Skolkovo Institute of Science and Technology
Author for correspondence.
Email: roman.isachenko@phystech.edu
Russian Federation, Institutskii per. 9, Dolgoprudnyi, Moscow oblast, 141700; ul. Nobelya 3, Moscow, 143026
V. V. Strijov
Moscow Institute of Physics and Technology (State University); A.A. Dorodnicyn Computing Centre
Email: roman.isachenko@phystech.edu
Russian Federation, Institutskii per. 9, Dolgoprudnyi, Moscow oblast, 141700; ul. Vavilova 40, Moscow, 119333