Daniel Kaszyński https://orcid.org/0000-0002-0865-0732 , Bogumił Kamiński https://orcid.org/0000-0002-0678-282X , Bartosz Pankratz https://orcid.org/0000-0001-7618-9119

(English) PDF


The market risk management process includes the quantification of the risk connected with defined portfolios of assets and the diagnostics of the risk model. Value at Risk (VaR) is one of the most common market risk measures. Since the distributions of the daily P&L of financial instruments are unobservable, literature presents a broad range of backtests for VaR diagnostics. In this paper, we propose a new methodological approach to the assessment of the size of VaR backtests, and use it to evaluate the size of the most distinctive and popular backtests. The focus of the paper is directed towards the evaluation of the size of the backtests for small-sample cases – a typical situation faced during VaR backtesting in banking practice. The results indicate significant differences between tests in terms of the p-value distribution. In particular, frequency-based tests exhibit significantly greater discretisation effects than duration-based tests. This difference is especially apparent in the case of small samples. Our findings prove that from among the considered tests, the Kupiec TUFF and the Haas Discrete Weibull have the best properties. On the other hand, backtests which are very popular in banking practice, that is the Kupiec POF and Christoffersen’s Conditional Coverage, show significant discretisation, hence deviations from the theoretical size.


Value at Risk, market risk management, backtesting, empirical size assessment


C00, C12, C15, D81, G32


Altman, D. G. (1991). Practical Statistics for Medical Research. London: Chapman & Hall/CRC. https://www.scribd.com/doc/273959883/Douglas-G-Altman-Practical-Statistics-for-Medical -Research-Chapman-Hall-CRC-1991

BCBS. (1996). Supervisory framework for the use of “backtesting” in conjunction with the internal models approach to market risk capital requirements. Basel: Basle Committee on Banking Supervision. https://www.bis.org/publ/bcbs22.pdf

BCBS. (2009). Revisions to the Basel II market risk framework. Basel: Bank for International Settlements. https://www.bis.org/publ/bcbs158.pdf

Berkowitz, J. (2001). Testing Density Forecasts, with Applications to Risk Management. Journal of Business & Economic Statistics, 19(4), 465–474. https://doi.org/10.1198/07350010152596718

Berkowitz, J., Christoffersen, P., Pelletier, D. (2011). Evaluating Value-at-Risk Models with Desk- -Level Data. Management Science, 57(12), 2213–2227. https://doi.org/10.1287/mnsc.1080.0964

Bontemps, C. (2014). Moment-based tests for discrete distributions. (IDEI Working Paper, n. 772). http://idei.fr/sites/default/files/medias/doc/by/bontemps/discrete-15oct2014.pdf

Campbell, S. D. (2006). A review of backtesting and backtesting procedures. Journal of Risk, 9(2), 1–17. http://dx.doi.org/10.21314/JOR.2007.146

Candelon, B., Colletaz, G., Hurlin, C., Tokpavi, S. (2011). Backtesting Value-at-Risk: a GMM Duration-Based Test. Journal of Financial Econometrics, 9(2), 314–343. https://doi.org/10.1093 /jjfinec/nbq025

Christoffersen, P. F. (1998). Evaluating Interval Forecasts. International economic review, 39(4), 841–862. https://doi.org/10.2307/2527341

Christoffersen, P., Pelletier, D. (2004). Backtesting Value-at-Risk: A Duration-Based Approach. Journal of Financial Econometrics, 2(1), 84–108. https://doi.org/10.1093/jjfinec/nbh004

Clements, M. P., Taylor, N. (2003). Evaluating interval forecasts of high-frequency financial data. Journal of Applied Econometrics, 18(4), 445–456. https://doi.org/10.1002/jae.703

Dowd, K. (1998). Beyond Value at Risk: The New Science of Risk Management. Chichester: John Wiley & Sons.

Dumitrescu, E. I., Hurlin, C., Pham, V. (2012). Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests. Finance, 33(1), 79–112. https://www.cairn-int.info/journal-finance -2012-1-page-79.htm?WT.tsrc=cairnPdf#

Engle, R. F., Manganelli, S. (2004). CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles. Journal of Business & Economic Statistics, 22(4), 367–381. https://doi.org /10.1198/073500104000000370

Escanciano, J. C., Olmo, J. (2011). Robust Backtesting Tests for Value-at-risk Models. Journal of Financial Econometrics, 9(1), 132–161. https://doi.org/10.1093/jjfinec/nbq021

Everitt, B. S. (Ed.). (2006). The Cambridge Dictionary of Statistics (3rd edition). Cambridge: Cambridge University Press.

Evers, C., Rohde, J. (2014). Model Risk in Backtesting Risk Measures (HEP Discussion Paper No. 529). http://diskussionspapiere.wiwi.uni-hannover.de/pdf_bib/dp-529.pdf

Haas, M. (2001). New Methods in Backtesting. https://www.ime.usp.br/~rvicente/risco/haas.pdf

Haas, M. (2005). Improved duration-based backtesting of value-at-risk. Journal of Risk, 8(2), 17–38. http://dx.doi.org/10.21314/JOR.2006.128

Hurlin, C. (29.04.2013). Backtesting Value-at-Risk Models. Séminaire Validation des Modeles Financiers, University of Orléans. https://www.univ-orleans.fr/deg/masters/ESA/CH/Slides _Seminaire_ Validation.pdf

Hurlin, C., Tokpavi, S. (2006). Backtesting Value-at-Risk Accuracy: A Simple New Test. Journal of Risk, 9(2), 19–37. http://dx.doi.org/10.21314/JOR.2007.148

Jorion, P. (2007). Value at Risk: The New Benchmark for Managing Financial Risk (3rd edition). New York: The McGraw-Hill Companies. https://www.academia.edu/8519246/Philippe_Jorion _Value_at_Risk_The_New_Benchmark_for_Managing_Financial_Risk_3rd_Ed_2007

Jorion, P. (2010). Financial Risk Manager Handbook: FRM Part I/Part II. Hoboken: John Wiley & Sons.

Krämer, W., Wied, D. (2015). A simple and focused backtest of value at risk. Economics Letters, 137, 29–31. https://doi.org/10.1016/j.econlet.2015.10.028

Kupiec, P. H. (1995). Techniques For Verifying the Accuracy of Risk Measurement Models. The Journal of Derivatives, 3(2), 73–84. https://doi.org/10.3905/jod.1995.407942

Lopez, J. A. (1998). Methods for Evaluating Value-at-Risk Estimates. Economic Policy Review, 4(3), 119–124. https://www.newyorkfed.org/medialibrary/media/research/epr/1998/EPRvol4no3.pdf

Małecka, M. (2014). Duration-Based Approach to VaR Independence Backtesting. Statistics in Transition new series, 15(4), 627–636. http://yadda.icm.edu.pl/yadda/element/bwmeta1.element .ekon-element-000171338797

Murdoch, D. J., Tsai, Y. L., Adcock, J. (2008). P-Values are Random Variables. The American Statistician, 62(3), 242–245. https://doi.org/10.1198/000313008X332421

Nieto, M. R., Ruiz, E. (2016). Frontiers in VaR forecasting and backtesting. International Journal of Forecasting, 32(2), 475–501. https://doi.org/10.1016/j.ijforecast.2015.08.003

Pajhede, T. (2015). Backtesting Value-at-Risk: A Generalized Markov Framework (Univ. of Copenhagen Dept. of Economics Discussion Paper No. 15–18). http://dx.doi.org/10.2139/ssrn .2693504

Pelletier, D., Wei, W. (2016). The Geometric-VaR Backtesting Method. Journal of financial econometrics, 14(4), 725–745. https://pdfs.semanticscholar.org/644b/ced159cafb17e48a24ffa36bbaf2ac776f18 .pdf?_ga=2.260873924.817706028.1606139341-1947910505.1605732686

Zhang, Y., Nadarajah, S. (2017). A review of backtesting for value at risk. Communications in Statistics – Theory and Methods, 47(15), 3616–3639. https://doi.org/10.1080/03610926.2017 .1361984

Ziggel, D., Berens, T., Weiß, G. N. F., Wied, D. (2014). A new set of improved Value-at-Risk backtests. Journal of Banking & Finance, 48, 29–41. https://doi.org/10.1016/j.jbankfin.2014 .07.005

Back to top
Copyright © 2019 Statistics Poland