On Model Selection, Bayesian Networks, and the Fisher Information Integral

Show full item record




Zou , Y & Roos , T 2017 , ' On Model Selection, Bayesian Networks, and the Fisher Information Integral ' , New Generation Computing , vol. 35 , no. 1 , pp. 5-27 . https://doi.org/10.1007/s00354-016-0002-y

Title: On Model Selection, Bayesian Networks, and the Fisher Information Integral
Author: Zou, Yuan; Roos, Teemu
Contributor organization: Department of Computer Science
Helsinki Institute for Information Technology
The Finnish Center of Excellence in Computational Inference Research (COIN)
Information, Complexity and Learning research group / Teemu Roos
Complex Systems Computation Group
Date: 2017-01
Language: eng
Number of pages: 23
Belongs to series: New Generation Computing
ISSN: 0288-3635
DOI: https://doi.org/10.1007/s00354-016-0002-y
URI: http://hdl.handle.net/10138/175523
Abstract: We study BIC-like model selection criteria and in particular, their refinements that include a constant term involving the Fisher information matrix. We perform numerical simulations that enable increasingly accurate approximation of this constant in the case of Bayesian networks. We observe that for complex Bayesian network models, the constant term is a negative number with a very large absolute value that dominates the other terms for small and moderate sample sizes. For networks with a fixed number of parameters, d, the leading term in the complexity penalty, which is proportional to d, is the same. However, as we show, the constant term can vary significantly depending on the network structure even if the number of parameters is fixed. Based on our experiments, we conjecture that the distribution of the nodes’ outdegree is a key factor. Furthermore, we demonstrate that the constant term can have a dramatic effect on model selection performance for small sample sizes.
Subject: 112 Statistics and probability
Fisher information integral
Bayesian networks
normalized maximum likelihood
Model selection
113 Computer and information sciences
Peer reviewed: Yes
Usage restriction: openAccess
Self-archived version: acceptedVersion

Files in this item

Total number of downloads: Loading...

Files Size Format View
new_generation_computing_v2.pdf 505.8Kb PDF View/Open

This item appears in the following Collection(s)

Show full item record