- Title
- Nonsmooth optimization-based hyperparameter-free neural networks for large-scale regression
- Creator
- Karmitsa, Napsu; Taheri, Sona; Joki, Kaisa; Paasivirta, Pauliina; Defterdarovic, J.; Bagirov, Adil; Mäkelä, Marko
- Date
- 2023
- Type
- Text; Journal article
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/197680
- Identifier
- vital:18914
- Identifier
-
https://doi.org/10.3390/a16090444
- Identifier
- ISSN:1999-4893 (ISSN)
- Abstract
- In this paper, a new nonsmooth optimization-based algorithm for solving large-scale regression problems is introduced. The regression problem is modeled as fully-connected feedforward neural networks with one hidden layer, piecewise linear activation, and the (Formula presented.) -loss functions. A modified version of the limited memory bundle method is applied to minimize this nonsmooth objective. In addition, a novel constructive approach for automated determination of the proper number of hidden nodes is developed. Finally, large real-world data sets are used to evaluate the proposed algorithm and to compare it with some state-of-the-art neural network algorithms for regression. The results demonstrate the superiority of the proposed algorithm as a predictive tool in most data sets used in numerical experiments. © 2023 by the authors.
- Publisher
- Multidisciplinary Digital Publishing Institute (MDPI)
- Relation
- Algorithms Vol. 16, no. 9 (2023), p.; http://purl.org/au-research/grants/arc/DP190100580
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- https://creativecommons.org/licenses/by/4.0/
- Rights
- Copyright © 2023 by the authors
- Rights
- Open Access
- Subject
- 40 Engineering; 46 Information and computing sciences; 49 Mathematical sciences; L1-loss function; Machine learning; Neural networks; Nonsmooth optimization; Regression analysis
- Full Text
- Reviewed
- Funder
- This work was financially supported by Research Council of Finland grants #289500, #319274, #345804, and #345805, and by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (Project no. DP190100580).
- Hits: 498
- Visitors: 506
- Downloads: 16
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Published version | 362 KB | Adobe Acrobat PDF | View Details Download |