- Title
- From convex to nonconvex: A loss function analysis for binary classification
- Creator
- Zhao, Lei; Mammadov, Musa; Yearwood, John
- Date
- 2010
- Type
- Text; Conference paper
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/36995
- Identifier
- vital:3906
- Identifier
- ISBN:1550-4786
- Abstract
- Problems of data classification can be studied in the framework of regularization theory as ill-posed problems. In this framework, loss functions play an important role in the application of regularization theory to classification. In this paper, we review some important convex loss functions, including hinge loss, square loss, modified square loss, exponential loss, logistic regression loss, as well as some non-convex loss functions, such as sigmoid loss, ø-loss, ramp loss, normalized sigmoid loss, and the loss function of 2 layer neural network. Based on the analysis of these loss functions, we propose a new differentiable non-convex loss function, called smoothed 0-1 loss function, which is a natural approximation of the 0-1 loss function. To compare the performance of different loss functions, we propose two binary classification algorithms for binary classification, one for convex loss functions, the other for non-convex loss functions. A set of experiments are launched on several binary data sets from the UCI repository. The results show that the proposed smoothed 0-1 loss function is robust, especially for those noisy data sets with many outliers. © 2010 IEEE.
- Publisher
- Sydney, NSW IEEE
- Relation
- Paper presented at10th IEEE International Conference on Data Mining Workshops, ICDMW 2010 p. 1281-1288
- Rights
- Open Access
- Rights
- This metadata is freely available under a CCO license
- Subject
- Classification; Loss function; Non-convex; Optimization; Regularization; 2 layer; Binary classification; Binary data; Data classification; Ill posed problem; Logistic regressions; Loss functions; Noisy data; Non-convex loss function; Regularization theory; UCI repository; Convex optimization; Data mining; Network layers; Neural networks; Exponential functions
- Full Text
- Reviewed
- Hits: 11109
- Visitors: 11219
- Downloads: 573
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Published | 578 KB | Adobe Acrobat PDF | View Details Download |