Hyperparameter-free NN algorithm for large-scale regression problems
Marko M. Mäkelä; Napsu Karmitsa; Pauliina Mäkinen; Kaisa Joki; Adil M. Bagirov; Sona Taheri
https://urn.fi/URN:NBN:fi-fe2021042825731
Tiivistelmä
In this paper, a new nonsmooth optimization based algorithm for solving large-scale regression problems is introduced. The regression problem is modeled using fullyconnected feedforward neural networks with one hidden layer, the piecewise linear activation, and the L1-loss functions. A novel constructive approach is developed for an automated determination of the proper number of hidden nodes. The limited memory bundle method [Haarala et.al., 2004, 2007] is applied to minimize the nonsmooth objective of the new regression problem. The proposed algorithm is evaluated using real-world data sets with both large number of input features and large number of samples. It is also compared with the well-known backpropagation neural network for regression using TensorFlow. The results demonstrate the superiority of the proposed algorithm as a predictive tool in most data sets used in our numerical experiments.
Kokoelmat
- Rinnakkaistallenteet [19207]