Information criteria for non-normalized models

Show full item record



Permalink

http://hdl.handle.net/10138/333909

Citation

Matsuda , T , Uehara , M & Hyvärinen , A 2021 , ' Information criteria for non-normalized models ' , Journal of Machine Learning Research , vol. 22 , pp. 1-33 . < https://www.jmlr.org/papers/v22/20-1366.html >

Title: Information criteria for non-normalized models
Author: Matsuda, Takeru; Uehara, Masatoshi; Hyvärinen, Aapo
Contributor: University of Helsinki, Department of Computer Science
Date: 2021
Language: eng
Number of pages: 33
Belongs to series: Journal of Machine Learning Research
ISSN: 1532-4435
URI: http://hdl.handle.net/10138/333909
Abstract: Many statistical models are given in the form of non-normalized densities with an intractable normalization constant. Since maximum likelihood estimation is computationally intensive for these models, several estimation methods have been developed which do not require explicit computation of the normalization constant, such as noise contrastive estimation (NCE) and score matching. However, model selection methods for general non normalized models have not been proposed so far. In this study, we develop information criteria for non-normalized models estimated by NCE or score matching. They are approximately unbiased estimators of discrepancy measures for non-normalized models. Simulation results and applications to real data demonstrate that the proposed criteria enable selection of the appropriate non-normalized model in a data-driven manner.
Subject: energy-based model
model selection
noise contrastive estimation
score matching
NOISE-CONTRASTIVE ESTIMATION
ASYMPTOTIC EQUIVALENCE
STATISTICAL-MODELS
CROSS-VALIDATION
SELECTION
CONSISTENCY
INFERENCE
NUMBER
113 Computer and information sciences
Rights:


Files in this item

Total number of downloads: Loading...

Files Size Format View
20_1366.pdf 578.7Kb PDF View/Open

This item appears in the following Collection(s)

Show full item record