Conference paper
Empirical generalization assessment of neural network models
This paper addresses the assessment of generalization performance of neural network models by use of empirical techniques. We suggest to use the cross-validation scheme combined with a resampling technique to obtain an estimate of the generalization performance distribution of a specific model. This enables the formulation of a bulk of new generalization performance measures.
Numerical results demonstrate the viability of the approach compared to the standard technique of using algebraic estimates like the FPE. Moreover, we consider the problem of comparing the generalization performance of different competing models. Since all models are trained on the same data, a key issue is to take this dependency into account.
The optimal split of the data set of size N into a cross-validation set of size Nγ and a training set of size N(1-γ) is discussed. Asymptotically (large data sees), γopt→1 such that a relatively larger amount is left for validation
| Language: | English |
|---|---|
| Publisher: | IEEE |
| Year: | 1995 |
| Pages: | 30-39 |
| Proceedings: | 1995 IEEE Workshop on Neural Networks for Signal Processing |
| ISBN: | 078032739X , 078032739x and 9780780327399 |
| Types: | Conference paper |
| DOI: | 10.1109/NNSP.1995.514876 |
| ORCIDs: | Larsen, Jan and Hansen, Lars Kai |
Computer networks Cost function Mean square error methods Neural networks Predictive models Probability distribution Stochastic processes Testing Training data Uncertainty cross-validation data set error estimation error probability distribution error statistics estimation theory generalisation (artificial intelligence) generalization learning learning (artificial intelligence) neural nets neural network models performance assessment performance evaluation training set
