Hyperparameter-free NN algorithm for large-scale regression problems

Abstract

<p>In this paper, a new nonsmooth optimization based algorithm for solving large-scale regression problems is introduced. The regression problem is modeled using fullyconnected feedforward neural networks with one hidden layer, the piecewise linear activation, and the L1-loss functions. A novel constructive approach is developed for an automated determination of the proper number of hidden nodes. The limited memory bundle method [Haarala et.al., 2004, 2007] is applied to minimize the nonsmooth objective of the new regression problem. The proposed algorithm is evaluated using real-world data sets with both large number of input features and large number of samples. It is also compared with the well-known backpropagation neural network for regression using TensorFlow. The results demonstrate the superiority of the proposed algorithm as a predictive tool in most data sets used in our numerical experiments.<br /></p>