- Title
- Determining regularization parameters for derivative free neural learning
- Creator
- Ghosh, Ranadhir; Ghosh, Moumita; Yearwood, John; Bagirov, Adil
- Date
- 2005
- Type
- Text; Conference paper
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/39150
- Identifier
- vital:2721
- Identifier
-
https://doi.org/10.1007/11510888_8
- Identifier
- ISBN:9783540269236
- Abstract
- Derivative free optimization methods have recently gained a lot of attractions for neural learning. The curse of dimensionality for the neural learning problem makes local optimization methods very attractive; however the error surface contains many local minima. Discrete gradient method is a special case of derivative free methods based on bundle methods and has the ability to jump over many local minima. There are two types of problems that are associated with this when local optimization methods are used for neural learning. The first type of problems is initial sensitivity dependence problem- that is commonly solved by using a hybrid model. Our early research has shown that discrete gradient method combining with other global methods such as evolutionary algorithm makes them even more attractive. These types of hybrid models have been studied by other researchers also. Another less mentioned problem is the problem of large weight values for the synaptic connections of the network. Large synaptic weight values often lead to the problem of paralysis and convergence problem especially when a hybrid model is used for fine tuning the learning task. In this paper we study and analyse the effect of different regularization parameters for our objective function to restrict the weight values without compromising the classification accuracy.
- Publisher
- Leipzig, Germany : Springer
- Relation
- Paper presented at 4th International Conference, MLDM 2005: Machine Learning and Data Mining in Pattern Recognition, Leipzig, Germany : 9th-11th July 2005 p. 71-79
- Rights
- Copyright Springer
- Rights
- This metadata is freely available under a CCO license
- Subject
- Neural Methods
- Hits: 1645
- Visitors: 1592
- Downloads: 1
| Thumbnail | File | Description | Size | Format |
|---|