Journal article
The Quantitative Evaluation of Functional Neuroimaging Experiments: Mutual Information Learning Curves
Learning curves are presented as an unbiased means for evaluating the performance of models for neuroimaging data analysis. The learning curve measures the predictive performance in terms of the generalization or prediction error as a function of the number of independent examples (e.g., subjects) used to determine the parameters in the model.
Cross-validation resampling is used to obtain unbiased estimates of a generic multivariate Gaussian classifier, for training set sizes from 2 to 16 subjects. We apply the framework to four different activation experiments, in this case \$\backslash\$[/sup 15/ O]water data sets, although the framework is equally valid for multisubject fMRI studies.
We demonstrate how the prediction error can be expressed as the mutual information between the scan and the scan label, measured in units of bits. The mutual information learning curve can be used to evaluate the impact of different methodological choices, e.g., classification label schemes, preprocessing choices.
Another application for the learning curve is to examine the model performance using bias/variance considerations enabling the researcher to determine if the model performance is limited by statistical bias or variance. We furthermore present the sensitivity map as a general method for extracting activation maps from statistical models within the probabilistic framework and illustrate relationships between mutual information and pattern reproducibility as derived in the NPAIRS framework described in a companion paper.
Language: | English |
---|---|
Year: | 2002 |
Pages: | 772-786 |
ISSN: | 10959572 and 10538119 |
Types: | Journal article |
DOI: | 10.1006/nimg.2001.1033 |
ORCIDs: | Hansen, Lars Kai |
Adult Artifacts Brain Mapping Cerebral Cortex Data Interpretation, Statistical Female Humans Image Processing, Computer-Assisted Imaging, Three-Dimensional Magnetic Resonance Imaging Male Mathematical Computing Middle Aged Models, Statistical Psychomotor Performance Reference Values Tomography, Emission-Computed cross-validation generalization error learning curve macroscopic and microscopic models multisubject PET and fMRI studies mutual information prediction error sensitivity map