Journal article
Optimized approximation algorithm in neural networks without overfitting
School of Electrical Engineering and Computer Science, Ohio University, Athens, OH 45701, USA. yliu@bobcat.ent.ohiou.edu1
In this paper, an optimized approximation algorithm (OAA) is proposed to address the overfitting problem in function approximation using neural networks (NNs). The optimized approximation algorithm avoids overfitting by means of a novel and effective stopping criterion based on the estimation of the signal-to-noise-ratio figure (SNRF).
Using SNRF, which checks the goodness-of-fit in the approximation, overfitting can be automatically detected from the training error only without use of a separate validation set. The algorithm has been applied to problems of optimizing the number of hidden neurons in a multilayer perceptron (MLP) and optimizing the number of learning epochs in MLP's backpropagation training using both synthetic and benchmark data sets.
The OAA algorithm can also be utilized in the optimization of other parameters of NNs. In addition, it can be applied to the problem of function approximation using any kind of basis functions, or to the problem of learning model selection when overfitting needs to be considered.
Language: | English |
---|---|
Publisher: | IEEE |
Year: | 2008 |
Pages: | 983-95 |
ISSN: | 19410093 and 10459227 |
Types: | Journal article |
DOI: | 10.1109/TNN.2007.915114 |
Algorithms Approximation algorithms Approximation methods Backpropagation algorithms Computer Simulation Function approximation Least squares approximation Multilayer perceptrons Neural Networks, Computer Neural networks Neurons Pattern Recognition, Automated Signal Processing, Computer-Assisted Signal to noise ratio Training data backpropagation backpropagation training function approximation goodness-of-fit multilayer perceptron multilayer perceptrons neural network neural network (NN) learning optimisation optimized approximation algorithm overfitting overfitting problem signal-to-noise-ratio figure estimation stopping criterion