ARTICLE
ABSTRACT
The market risk management process includes the quantification of the risk connected with defined portfolios of assets and the diagnostics of the risk model. Value at Risk (VaR) is one of the most common market risk measures. Since the distributions of the daily P&L of financial instruments are unobservable, literature presents a broad range of backtests for VaR diagnostics. In this paper, we propose a new methodological approach to the assessment of the size of VaR backtests, and use it to evaluate the size of the most distinctive and popular backtests. The focus of the paper is directed towards the evaluation of the size of the backtests for small-sample cases – a typical situation faced during VaR backtesting in banking practice. The results indicate significant differences between tests in terms of the p-value distribution. In particular, frequency-based tests exhibit significantly greater discretisation effects than duration-based tests. This difference is especially apparent in the case of small samples. Our findings prove that from among the considered tests, the Kupiec TUFF and the Haas Discrete Weibull have the best properties. On the other hand, backtests which are very popular in banking practice, that is the Kupiec POF and Christoffersen’s Conditional Coverage, show significant discretisation, hence deviations from the theoretical size.
KEYWORDS
Value at Risk, market risk management, backtesting, empirical size assessment
JEL
C00, C12, C15, D81, G32
REFERENCES
Altman, D. G. (1991). Practical Statistics for Medical Research. London: Chapman & Hall/CRC. https://www.scribd.com/doc/273959883/Douglas-G-Altman-Practical-Statistics-for-Medical -Research-Chapman-Hall-CRC-1991
BCBS. (1996). Supervisory framework for the use of “backtesting” in conjunction with the internal models approach to market risk capital requirements. Basel: Basle Committee on Banking Supervision. https://www.bis.org/publ/bcbs22.pdf
BCBS. (2009). Revisions to the Basel II market risk framework. Basel: Bank for International Settlements. https://www.bis.org/publ/bcbs158.pdf
Berkowitz, J. (2001). Testing Density Forecasts, with Applications to Risk Management. Journal of Business & Economic Statistics, 19(4), 465–474. https://doi.org/10.1198/07350010152596718
Berkowitz, J., Christoffersen, P., Pelletier, D. (2011). Evaluating Value-at-Risk Models with Desk- -Level Data. Management Science, 57(12), 2213–2227. https://doi.org/10.1287/mnsc.1080.0964
Bontemps, C. (2014). Moment-based tests for discrete distributions. (IDEI Working Paper, n. 772). http://idei.fr/sites/default/files/medias/doc/by/bontemps/discrete-15oct2014.pdf
Campbell, S. D. (2006). A review of backtesting and backtesting procedures. Journal of Risk, 9(2), 1–17. http://dx.doi.org/10.21314/JOR.2007.146
Candelon, B., Colletaz, G., Hurlin, C., Tokpavi, S. (2011). Backtesting Value-at-Risk: a GMM Duration-Based Test. Journal of Financial Econometrics, 9(2), 314–343. https://doi.org/10.1093 /jjfinec/nbq025
Christoffersen, P. F. (1998). Evaluating Interval Forecasts. International economic review, 39(4), 841–862. https://doi.org/10.2307/2527341
Christoffersen, P., Pelletier, D. (2004). Backtesting Value-at-Risk: A Duration-Based Approach. Journal of Financial Econometrics, 2(1), 84–108. https://doi.org/10.1093/jjfinec/nbh004
Clements, M. P., Taylor, N. (2003). Evaluating interval forecasts of high-frequency financial data. Journal of Applied Econometrics, 18(4), 445–456. https://doi.org/10.1002/jae.703
Dowd, K. (1998). Beyond Value at Risk: The New Science of Risk Management. Chichester: John Wiley & Sons.
Dumitrescu, E. I., Hurlin, C., Pham, V. (2012). Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests. Finance, 33(1), 79–112. https://www.cairn-int.info/journal-finance -2012-1-page-79.htm?WT.tsrc=cairnPdf#
Engle, R. F., Manganelli, S. (2004). CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles. Journal of Business & Economic Statistics, 22(4), 367–381. https://doi.org /10.1198/073500104000000370
Escanciano, J. C., Olmo, J. (2011). Robust Backtesting Tests for Value-at-risk Models. Journal of Financial Econometrics, 9(1), 132–161. https://doi.org/10.1093/jjfinec/nbq021
Everitt, B. S. (Ed.). (2006). The Cambridge Dictionary of Statistics (3rd edition). Cambridge: Cambridge University Press.
Evers, C., Rohde, J. (2014). Model Risk in Backtesting Risk Measures (HEP Discussion Paper No. 529). http://diskussionspapiere.wiwi.uni-hannover.de/pdf_bib/dp-529.pdf
Haas, M. (2001). New Methods in Backtesting. https://www.ime.usp.br/~rvicente/risco/haas.pdf
Haas, M. (2005). Improved duration-based backtesting of value-at-risk. Journal of Risk, 8(2), 17–38. http://dx.doi.org/10.21314/JOR.2006.128
Hurlin, C. (29.04.2013). Backtesting Value-at-Risk Models. Séminaire Validation des Modeles Financiers, University of Orléans. https://www.univ-orleans.fr/deg/masters/ESA/CH/Slides _Seminaire_ Validation.pdf
Hurlin, C., Tokpavi, S. (2006). Backtesting Value-at-Risk Accuracy: A Simple New Test. Journal of Risk, 9(2), 19–37. http://dx.doi.org/10.21314/JOR.2007.148
Jorion, P. (2007). Value at Risk: The New Benchmark for Managing Financial Risk (3rd edition). New York: The McGraw-Hill Companies. https://www.academia.edu/8519246/Philippe_Jorion _Value_at_Risk_The_New_Benchmark_for_Managing_Financial_Risk_3rd_Ed_2007
Jorion, P. (2010). Financial Risk Manager Handbook: FRM Part I/Part II. Hoboken: John Wiley & Sons.
Krämer, W., Wied, D. (2015). A simple and focused backtest of value at risk. Economics Letters, 137, 29–31. https://doi.org/10.1016/j.econlet.2015.10.028
Kupiec, P. H. (1995). Techniques For Verifying the Accuracy of Risk Measurement Models. The Journal of Derivatives, 3(2), 73–84. https://doi.org/10.3905/jod.1995.407942
Lopez, J. A. (1998). Methods for Evaluating Value-at-Risk Estimates. Economic Policy Review, 4(3), 119–124. https://www.newyorkfed.org/medialibrary/media/research/epr/1998/EPRvol4no3.pdf
Małecka, M. (2014). Duration-Based Approach to VaR Independence Backtesting. Statistics in Transition new series, 15(4), 627–636. http://yadda.icm.edu.pl/yadda/element/bwmeta1.element .ekon-element-000171338797
Murdoch, D. J., Tsai, Y. L., Adcock, J. (2008). P-Values are Random Variables. The American Statistician, 62(3), 242–245. https://doi.org/10.1198/000313008X332421
Nieto, M. R., Ruiz, E. (2016). Frontiers in VaR forecasting and backtesting. International Journal of Forecasting, 32(2), 475–501. https://doi.org/10.1016/j.ijforecast.2015.08.003
Pajhede, T. (2015). Backtesting Value-at-Risk: A Generalized Markov Framework (Univ. of Copenhagen Dept. of Economics Discussion Paper No. 15–18). http://dx.doi.org/10.2139/ssrn .2693504
Pelletier, D., Wei, W. (2016). The Geometric-VaR Backtesting Method. Journal of financial econometrics, 14(4), 725–745. https://pdfs.semanticscholar.org/644b/ced159cafb17e48a24ffa36bbaf2ac776f18 .pdf?_ga=2.260873924.817706028.1606139341-1947910505.1605732686
Zhang, Y., Nadarajah, S. (2017). A review of backtesting for value at risk. Communications in Statistics – Theory and Methods, 47(15), 3616–3639. https://doi.org/10.1080/03610926.2017 .1361984
Ziggel, D., Berens, T., Weiß, G. N. F., Wied, D. (2014). A new set of improved Value-at-Risk backtests. Journal of Banking & Finance, 48, 29–41. https://doi.org/10.1016/j.jbankfin.2014 .07.005
