Measures of Model Performance Based On the Log Accuracy Ratio

Show full item record



Permalink

http://hdl.handle.net/10138/312261

Citation

Morley , S K , Brito , T V & Welling , D T 2018 , ' Measures of Model Performance Based On the Log Accuracy Ratio ' , Space Weather , vol. 16 , no. 1 , pp. 69-88 . https://doi.org/10.1002/2017SW001669

Title: Measures of Model Performance Based On the Log Accuracy Ratio
Author: Morley, S. K.; Brito, T. V.; Welling, D. T.
Contributor: University of Helsinki, Department of Physics
Date: 2018-01
Language: eng
Number of pages: 20
Belongs to series: Space Weather
ISSN: 1542-7390
URI: http://hdl.handle.net/10138/312261
Abstract: Quantitative assessment of modeling and forecasting of continuous quantities uses a variety of approaches. We review existing literature describing metrics for forecast accuracy and bias, concentrating on those based on relative errors and percentage errors. Of these accuracy metrics, the mean absolute percentage error (MAPE) is one of the most common across many fields and has been widely applied in recent space science literature and we highlight the benefits and drawbacks of MAPE and proposed alternatives. We then introduce the log accuracy ratio and derive from it two metrics: the median symmetric accuracy and the symmetric signed percentage bias. Robust methods for estimating the spread of a multiplicative linear model using the log accuracy ratio are also presented. The developed metrics are shown to be easy to interpret, robust, and to mitigate the key drawbacks of their more widely used counterparts based on relative errors and percentage errors. Their use is illustrated with radiation belt electron flux modeling examples.
Subject: model validation
forecasting
POSITIONING SYSTEM CONSTELLATION
RADIATION BELT
GEOSYNCHRONOUS ORBIT
SPACE WEATHER
ENERGETIC ELECTRONS
NEURAL-NETWORK
TIME-SERIES
FIELD MODEL
PREDICTION
FORECAST
115 Astronomy, Space science
114 Physical sciences
Rights:


Files in this item

Total number of downloads: Loading...

Files Size Format View
2017SW001669.pdf 1.088Mb PDF View/Open

This item appears in the following Collection(s)

Show full item record