Calibrating expert assessments using hierarchical Gaussian process models

Show full item record



Permalink

http://hdl.handle.net/10138/322697

Citation

Perälä , T , Vanhatalo , J & Chrysafi , A 2020 , ' Calibrating expert assessments using hierarchical Gaussian process models ' , Bayesian analysis , vol. 15 , no. 4 , pp. 1251–1280 . https://doi.org/10.1214/19-BA1180

Title: Calibrating expert assessments using hierarchical Gaussian process models
Author: Perälä, Tommi; Vanhatalo, Jarno; Chrysafi, Anna
Contributor organization: Department of Mathematics and Statistics
Organismal and Evolutionary Biology Research Programme
Research Centre for Ecological Change
Environmental and Ecological Statistics Group
Biostatistics Helsinki
Date: 2020-12
Language: eng
Number of pages: 30
Belongs to series: Bayesian analysis
ISSN: 1931-6690
DOI: https://doi.org/10.1214/19-BA1180
URI: http://hdl.handle.net/10138/322697
Abstract: Expert assessments are routinely used to inform management and other decision making. However, often these assessments contain considerable biases and uncertainties for which reason they should be calibrated if possible. Moreover, coherently combining multiple expert assessments into one estimate poses a long-standing problem in statistics since modeling expert knowledge is often difficult. Here, we present a hierarchical Bayesian model for expert calibration in a task of estimating a continuous univariate parameter. The model allows experts' biases to vary as a function of the true value of the parameter and according to the expert's background. We follow the fully Bayesian approach (the so-called supra-Bayesian approach) and model experts' bias functions explicitly using hierarchical Gaussian processes. We show how to use calibration data to infer the experts' observation models with the use of bias functions and to calculate the bias corrected posterior distributions for an unknown system parameter of interest. We demonstrate and test our model and methods with simulated data and a real case study on data-limited fisheries stock assessment. The case study results show that experts' biases vary with respect to the true system parameter value and that the calibration of the expert assessments improves the inference compared to using uncalibrated expert assessments or a vague uniform guess. Moreover, the bias functions in the real case study show important differences between the reliability of alternative experts. The model and methods presented here can be also straightforwardly applied to other applications than our case study.
Subject: 111 Mathematics
expert elicitation
bias correction
Gaussian process
Supra Bayes
fisheries science
environmental management
STOCK ASSESSMENT
CATCH LIMITS
FISHERIES
PROBABILITY
ELICITATION
MANAGEMENT
JUDGMENTS
OPINION
RECONCILIATION
HEURISTICS
Peer reviewed: Yes
Rights: cc_by
Usage restriction: openAccess
Self-archived version: acceptedVersion
Funder: Valtion perusrahoitus/hankkeet
Valtion perusrahoitus/hankkeet
Unknown funder
Valtion perusrahoitus/hankkeet
SUOMEN AKATEMIA
SUOMEN AKATEMIA
Grant number:






Files in this item

Total number of downloads: Loading...

Files Size Format View
manuscript.pdf 690.5Kb PDF View/Open
euclid.ba.1573117289_1.pdf 516.4Kb PDF View/Open

This item appears in the following Collection(s)

Show full item record