Backtesting

Peter Christoersen
Desautels Faculty of Management, McGill University,
Copenhagen Business School and CREATES
1001 Sherbrooke Street West
Montreal, Canada H3A 1G5
peter.christoersen@mcgill.ca
Tel: (514) 398-2869
Fax: (514) 398-3876
November 19, 2008

Abstract
This chapter surveys methods for backtesting risk models using the ex ante risk measure forecasts
from the model and the ex post realized portfolio profit or loss. The risk measure forecast can
take the form of a V aR, an Expected Shortfall, or a distribution forecast. The backtesting
surveyed in this chapter can be seen as a final diagnostic check on the aggregate risk model
carried out by the risk management team that constructed the risk model, or they can be used
by external model-evaluators such as bank supervisors. Common for the approaches suggested
is that they only require information on the daily ex ante risk model forecast and the daily ex
post corresponding profit and loss. In particular, knowledge about the assumptions behind the
risk model and its construction is not required.
Keywords: Value-at-Risk, expected shortfall, distribution, forecasting, model evaluation, testing, historical simulation.

Prepared for the Encyclopedia of Quantitative Finace edited by Rama Cont and published by John Wiley &

Sons, Ltd. I am also aliated with CIRANO and CIREQ. I am grateful for financial support from FQRSC, IFM2
and SSHRC and from the Center for Research in Econometric Analysis of Time Series, CREATES, funded by the
Danish National Research Foundation.

1
Electronic copy available at: http://ssrn.com/abstract=2044825

Introduction

The term backtesting is used in several dierent ways in finance. Most commonly backtesting
denotes either 1) an assessment of the hypothetical historical performance of a suggested trading
strategy, or 2) the evaluation of financial risk models using historical data on risk forecasts and
profit and loss realizations. This chapter is about the evaluation of risk models.
The objective is to consider the daily ex ante risk measure forecasts from a model and test it
against the daily ex post realized portfolio loss. The risk measure forecast could take the form
of a V aR, an Expected Shortfall, or a distributional forecast. The goal is to be able to backtest
any of these risk measures of interest. The backtesting procedures developed in this chapter can
be seen as a final diagnostic check on the risk model carried out by the risk management team
that constructed the risk model, or they can be used by external model-evaluators such as bank
supervisors.
Evidence on actual bank V aRs and their backtesting performance can be found in Berkowitz
and OBrien (2002), Jorion (2002), Perignon, Deng and Wang (2006), Perignon and Smith (2006)
and Smith (2007). The regulatory considerations involved in backtesting are detailed by the Basle
Committee on Banking Supervision (1996a, 1996b, 2004). Lopez (1999) analyze the regulatory
approach to backtesting and Campbell (2007) provides a survey of backtesting that includes a
discussion of regulatory considerations. Backtesting of credit risk models is investigated in Lopez
and Saidenberg (2000).
This chapter first establishes procedures for backtesting the Value-at-Risk (V aR) metric (eqf15/004,
eqf15/008). Second, we consider increasing backtesting power by using explanatory variables to
backtest the V aR. Third, we consider increasing power by backtesting risk measures other than
V aR and we discuss various other issues in backtesting.

Backtesting V aR

By now Value-at-Risk (V aR) is by far the most popular portfolio risk measure used by risk management practitioners. The V aR revolution in risk management was triggered by JP Morgans
RiskMetrics approach launched in 1994. Supervisory authorities immediately recognized the need
for methods to backtest V aR and the first research on backtesting was published soon after in
Kupiec (1995) and Hendricks (1996). Christoersen (1998) extended Kupiecs test of unconditional
V aR coverage to tests of conditional V aR coverage. These concepts will be defined shortly.

2.1

Defining the Hit Sequence

p
First, define V aRt+1
to be a number constructed on day t such that the portfolio losses on day
p
t + 1 will only be larger than the V aRt+1
forecast with probability p. If we observe a time series of

past ex-ante V aR forecasts and past ex-post losses, P L, we can define the hit sequence of V aR

2
Electronic copy available at: http://ssrn.com/abstract=2044825

violations as
It+1 =

p
1, if P Lt+1 > V aRt+1
p .
0, if P Lt+1 < V aRt+1

(1)

The hit sequence returns a 1 on day t + 1 if the loss on that day is larger than the V aR number
predicted in advance for that day. If the V aR is not exceeded (or violated) then the hit sequence
returns a 0. When backtesting the risk model, we construct a sequence {It+1 }Tt=1 across T days
indicating when the past violations occurred.

2.2

The Null Hypothesis

If we are using the perfect V aR model then given all the information available to us at the time
the V aR forecast is made, we should not be able to predict whether the V aR will be violated. We
should be expecting a 1 in the hit sequence with probability p and we should be expecting a 0 with
probability 1 p and the occurrences of the hits should be random over time.

We will say that a risk model has correct unconditional V aR coverage if Pr (It+1 = 1) = p and

we will say that a risk model has correct conditional V aR coverage if Prt (It+1 = 1) = p. Roughly
speaking, correct unconditional V aR coverage just means that the risk model delivers V aR hits
with probability p on average across the days. Correct conditional V aR coverage means that the
risk model gives a V aR hit with probability p on every day given all the information available on
the day before. Note that correct conditional coverage implies correct unconditional coverage but
not vice versa.
We can think of V aR backtesting as testing the hypothesis
H0 : It+1 i.i.d. Bernoulli(p).
If p is one half, then the i.i.d. Bernoulli distribution describes the distribution of getting a a
head when tossing a fair coin. When backtesting risk models, p will not be one half but instead
on the order of 0.01 or 0.05 depending on the coverage rate of the V aR. The hit sequence from a
correctly specified risk model should look like a sequence of random tosses of a coin which comes
up heads 1% or 5% of the time depending on the V aR coverage rate.
Note that in general, the expected value of a binary sequence is simply the probability of getting
a1
Et [It+1 ] = Prt (It+1 = 1) 1 + Prt (It+1 = 0) 0 = Prt (It+1 = 1) t+1|t
We will denote this conditional hit probability by t+1|t and its unconditional counterpart is defined
E [It+1 ] .

We can therefore construct the following null hypothesis of correct conditional coverage for the

hit sequence from a V aR model

H0 : Et [It+1 ] = t+1|t = p
namely that the conditionally expected value of the hit sequence at time t + 1 given all the information available at time t is the promised V aR coverage rate p.
3

2.3

Unconditional Coverage Testing

We first want to test if the unconditional probability of a violation in the risk model, , is significantly dierent from the promised probability, p. We call this the unconditional coverage hypothesis.
We can write H0 : E [It ] = p.

The expected value of the hit sequence can be estimated by the sample average,
=

1
T

PT

t=1 It

T1 /T , where T1 is the number of 1s in the sample. Note that if the null hypothesis is true we have
P
that E [
] = T1 Tt=1 It = p. Assuming that the observations on the hit sequence are independent

over time, the variance of the

estimate is

1
TV

ar (It ) where V ar (It ) can be estimated as the sample

variance of the hit sequence. The hypothesis that E [It ] = p can therefore be tested in a simple
means test
MT =

p
N (0, 1)
Tp
V ar (It )

(2)

We can also implement the unconditional coverage test as a likelihood ratio test. For this we
write the likelihood of an i.i.d. Bernoulli() hit sequence
L () =

T
Y
(1 )1It+1 It+1 = (1 )T0 T1
t=1

where T0 and T1 are the number of 0s and 1s in the sample. We can easily estimate from

= T1 /T , that is the observed fraction of violations in the sequence. Plugging the ML estimates
back into the likelihood function gives the optimized likelihood as
L (
) = (1 T1 /T )T0 (T1 /T )T1 .
Under the unconditional coverage null hypothesis that = p, where p is the known V aR
coverage rate, we have the likelihood
T
Y
(1 p)1It+1 pIt+1 = (1 p)T0 pT1 .
L (p) =
t=1

And we can check the unconditional coverage hypothesis using a likelihood ratio test
)] 21 .
LRuc = 2 ln [L (p) /L (
Asymptotically, that is as the number of observations, T, goes to infinity, the test will be distributed
as a 2 with one degree of freedom.
The choice of significance level comes down to an assessment of the costs of making two types of
mistakes: We could reject a correct model (Type I error) or we could fail to reject (that is accept)
an incorrect model (Type II error). Increasing the significance level implies larger Type I errors
but smaller Type II errors and vice versa. In academic work, a significant level of 1%, 5% or 10%
is typically used. In risk management, the Type II errors may be very costly so that a significance
level of 10% may be appropriate.
4

Often we do not have a large number of observations available for backtesting, and we certainly
will typically not have a large number of violations, T1 , which are the informative observations.
It is therefore often better to rely on Monte Carlo simulated P-values rather than those from the
2 distribution. Christoersen and Pelletier (2004) discuss how to implement the Dufour (2006)
Monte Carlo P-values in a backtesting setting.

2.4

Independence Testing

There is strong evidence of time-varying volatility in daily asset returns as surveyed in Andersen
et al (2005). If the risk model ignores such dynamics then the V aR will react slowly to changing
market conditions and V aR violations will appear clustered in time. Pritsker (2001) illustrates this
problem using V aRs computed from Historical Simulation. If the V aR violations are clustered then
the risk manager can essentially predict that if today has a V aR hit, then there is a probability
larger than p of getting a hit tomorrow which violates that the V aR is based on an adequate model.
This is clearly not satisfactory. In such a situation the risk manager should increase the V aR in
order to lower the conditional probability of a violation to the promised p.
The most common way to test for dynamics in time series analysis is to rely on the autocorrelation function and the associated Portmanteau or Ljung-Box type tests. We can implement
this approach for backtesting as well. Let k be the autocorrelation at lag k for the hit sequence.
Plotting k against k for k = 1, ..., m will give a visual impression of the correlation of between a
hit in one of the last m trading days and a hit today. The Ljung-Box test provides a formal check
of the null hypothesis that all of the first m autocorrelations are zero against the alternative that
any of them is not. The test is easily constructed as
m
X
2k
LB(m) = T (T + 2)
2m
T k

(3)

k=1

where 2m denotes the chi-squared distribution with m degrees of freedom. Berkowitz, Christoersen
and Pelletier (2007) find that setting m = 5 gives good testing power in a realistic daily V aR
backtesting setting.
Independence testing can also be done using the likelihood approach. Assume that the hit
sequence is dependent over time and that it can be described as a first-order Markov sequence with
transition probability matrix
1 =

"

1 01 01

1 11 11

These transition probabilities simply mean that conditional on today being a non-violation (that is
It = 0) then the probability of tomorrow being a violation (that is It+1 = 1) is 01 . The probability
of tomorrow being a violation given today is also a violation is 11 = Pr (It = 1 and It+1 = 1) .The
probability of a non-violation following a non-violation is 1 01 and the probability of a nonviolation following a violation is 1 11 .

If we observe a sample of T observations, then we can write the likelihood function of the
first-order Markov process as
L (1 ) = (1 01 )T00 T0101 (1 11 )T10 T1111
where Tij , i, j = 0, 1 is the number of observations with a j following an i. Taking first derivatives
with respect to 01 and 11 and setting these derivatives to zero, one can solve for the Maximum
Likelihood estimates

T01
T11
,
11 =
.
T00 + T01
T10 + T11
01 ,
10 = 1
11 .
Using then the fact that the probabilities have to sum to one we have
00 = 1

01 =

Allowing for dependence in the hit sequence corresponds to allowing 01 to be dierent from

11 . We are typically worried about positive dependence which amounts to the probability of a
violation following a violation ( 11 ) being larger than the probability of a violation following a nonviolation ( 01 ). If on the other hand the hits are independent over time, then the probability of a
violation tomorrow does not depend on today being a violation or not and we write 01 = 11 = .
We can test the independence hypothesis that 01 = 11 using a likelihood ratio test
h
i
1 21
) /L
LRind = 2 ln L (
where L (
) is the likelihood under the alternative hypothesis from the LRuc test.

Other methods for independence testing based on the duration of time between hits can be
found in Christoersen and Pelletier (2004).

Backtesting with Information Variables

The preceding tests are quick and easy to implement. But as they only use information on past
V aR hits, they might not have much power to detect misspecified risk models. In order to increase
the testing power, we consider using the information in past market variables, such as interest
rate spreads or volatility measures. The basic idea is to test the model using information which
may explain when violations occur. The advantage of increasing the information set is not only to
increase power but also to help us understand the areas in which the risk model is misspecified.
This understanding is key in improving the risk models further.
If we define the q-dimensional vector of information variables available to the backtester at time
t as Xt , , then the null hypothesis of a correct risk model can be written as
H0 : Pr (It+1 = 1|Xt ) = p E [It+1 p|Xt ] = 0.
The hypothesis says that the conditional probability of getting a V aR violation on day t + 1
should be independent of any variable observed at time t and it should simply be equal to the
promised V aR coverage rate, p. This hypothesis is equivalent to the conditional expectation of the
hit sequence less p being equal to 0.
6

Engle and Manganelli (2004) develop a conditional autoregressive V aR by regression quantiles

approach. For backtesting they suggest the following dynamic quantile test

1 0
X (I p) / (T p (1 p)) 2q
DQ = (I p)0 X X 0 X

(4)

where X is the T by q matrix of information variables, and (I p) is the T by 1 vector of hits

where each element has been subtracted by the desired covered rate p.

Berkowitz, Christoersen and Pelletier (2007) finds that implementing the DQ test using simply
the lagged V aR from a GARCH model as well as the lagged hit gives good power in a realistic daily
V aR experiment. Smith (2007) also finds good power when applying the DQ test. A regressionbased approach to backtesting is also used in Christoersen and Diebold (2000) and Christoersen
(2003). Christoersen, Hahn and Inoue (2001) develop tests for comparing dierent V aR models.
Smith (2007) further suggest a probit approach where the potentially time-varying probability
of a hit is modeled using the normal cumulative density function and where Lagrange Multiplier
tests are used.
When backtesting with information variables the question of which variables to include in the
vector Xt of course immediately arises. It is dicult to give a general answer as it depends on the
particular portfolio at hand. It is likely that variables which are thought to be correlated with the
future volatility in the portfolio are good candidates. In equity portfolios option implied volatility
measures such as the VIX would be an obvious candidate. In FX portfolios option implied volatility
measures could be constructed as well. In bond portfolios variables such as term spreads and credit
spreads are likely candidates as are variables capturing the level, slope and curvature of the term
structure.

Other Issues in Backtesting

When correctly specified the one-day V aR measure tells the user that there is a probability p of
loosing more than the V aR over the next day. Importantly, it does not, for example, say how
much one can expect to loose on the days where the V aR is violated. Thus the V aR only contains
partial information about the distribution of losses. This limitation of the V aR is reflected in the
definition of the hit variable in (1) which in turn limits the possibilities for backtesting in the V aR
setting. We can only backtest on the hit occurrences and not for example on the magnitude of the
losses when the hits occur because the V aR does not promise hits of a certain magnitude.

4.1

Backtesting Expected Shortfall

The limitations of the V aR as a risk measure has to suggestions of alternative risk measures, most
prominently Expected Shortfall (ES), also referred to as Conditional VaR (CVAR) (eqf15/004,
eqf15/008), which denotes the expected loss on the days where the V aR is violated. We can define

it formally as

p
p
= Et P Lt+1 |P Lt+1 > V aRt+1
ESt+1

Note that ES provides information on the expected magnitude of the loss whenever the V aR is
violated. We now consider ways to backtest the ES risk measure.
Consider again a vector of variables, Xt , which are known to the risk manager, and which may
help explain potential portfolio losses beyond what is explained by the risk model. The ES risk
measure promises that whenever we violate the V aR, the expected value of the violation will be
p
. We can therefore test the ES measure by checking if the vector Xt has any ability
equal to ESt+1

to explain the deviation of the observed shortfall or loss, P Lt+1 , from the expected shortfall on the
days where the V aR was violated. Mathematically, we can write
p
p
= b0 + b01 Xt + et+1 , for t + 1 where P Lt+1 > V aRt+1
P Lt+1 ESt+1

where t + 1 now refers only to days where the V aR was violated. The observations where the
V aR was not violated are simply removed from the sample. The error term et+1 is assumed to be
independent of the regressor, Xt .
In order to test the null hypothesis that the risk model from which the ES forecasts were made
uses all information optimally (b1 = 0), and that it is not biased (b0 = 0), we can jointly test that
b0 = b1 = 0.
Notice that now the magnitude of the violation shows up on the left hand side of the regression.
But notice that we can still only use information in the tail to back-test. The ES measure does
not reveal any particular properties about the remainder of the distribution and therefore we only
use the observations where the losses were larger than the V aR.
Further discussion of ES backtesting can be found in McNeil and Frey (2000).

4.2

Backtesting the Entire Distribution

Rather than focusing on particular risk measures from the loss distribution such as the Value at
Risk or the Expected Shortfall, we could instead decide to backtest the entire loss distribution from
the risk model. This would have the benefit of potentially increasing further the power to reject
bad risk models.
Assuming that the risk model produces a cumulative distribution of portfolio losses, call it
Ft (). Then at the end of every day, after having observed the actual portfolio loss (or profit) we
can calculate the risk models probability of observing a loss below the actual. We will denote this
probability by pt+1 Ft (P Lt+1 ) .

If we are using the correct risk model to forecast the loss distribution, then we should not be

able to forecast the risk models probability of falling below the actual return. In other words,
the time series of observed probabilities pt+1 should be distributed independently over time as a
Uniform(0,1) variable. We therefore want to consider tests of the null hypothesis
H0 : pt+1 i.i.d. Uniform (0, 1) .
8

(5)

The Uniform(0,1) distribution function is flat on the interval 0 to 1 and zero everywhere else. As
the pt+1 variable is a probability it is must lie in the zero to one interval. Constructing a histogram
and checking if it looks reasonably flat provides a useful visual diagnostic. If systematic deviations
from a flat line appear in the histogram, then we would conclude that the distribution from the
risk model is misspecified. Diebold, Gunther and Tay (1998) contains a detailed discussion of this
approach.
Unfortunately, testing the i.i.d. uniform distribution hypothesis is cumbersome due to the restricted support of the uniform distribution. But we can transform the i.i.d. Uniform pt+1 to an
i.i.d. standard normal variable, zt+1 using the inverse cumulative distribution function, 1 (pt+1 ) .
We are then left with a test of a variable conforming to the standard normal distribution, which
can easily be implemented. The i.i.d. property of zt+1 can be assessed via the autocorrelation functions and by using the LB(m) test in (3) on zt+1 and |zt+1 | for example. The normal distribution

property can be tested using the method of moments approach in Bontemps and Meddahi (2005).
Regression based tests using information variables can also be constructed. Further analysis of
distribution backtesting can be found in Berkowitz (2001) and Crnkovic and Drachman (1996).

4.3

Dirty Profits and Losses

The backtesting procedures surveyed in this chapter all in one way or another compare the ex
ante forecast from a risk model to the ex post profit and losses (P/L). It is therefore obviously
essential but unfortunately not always the case that the ex post recorded P/L arise directly from
the portfolio used to make the ex ante model predictions. The risk model will typically produce a
risk forecast for the loss distributions of a particular portfolio of assets. The ex post profits and
losses should only contain cash flows directly related to the portfolio of assets assumed in the risk
models. But the total P/L of a trading desk may contain trading commission revenues as well as
costs that are not directly related to holding the portfolio of assets assumed in the risk model. This
extraneous cash flows should be stripped from the P/L before backtesting.
The daily P/L may of course also include cash-flows from intraday transactions that is the P/L
from selling an asset that was bought the same day. Such cash-flows are not directly related to
the end-of-day portfolio entered into the risk model and should ideally be stripped away as well.
OBrien and Berkowitz (2005) contain a detailed discussion of these issues.

4.4

Multiple-day Horizons and Changing Portfolio Weights

The backtesting procedures described above can be relatively easily adopted to the multi-day riskhorizon setting. If for example a 10-day V aR forecast is observed daily along with the ex post
10-day P/L, then the hit sequence can be constructed daily as in equation (1). The 10-day V aR
horizons implies that 9-day autocorrelation is to be expected in the hit-sequence and that this
should be allowed for when constructing its variance in (2). Similarly, in the DQ test in (4) one

should use information variables available 10 days prior to the observed P/L. Smith (2007) discuss
backtesting with multi-day V aRs.

4.5

Allowing for Risk Model Parameter Estimation Error

In the discussion so far we have abstracted from the fact that the risk models in use most likely
contain parameters that are estimated with errors. This parameter estimation error will in turn
render the hit sequence observed with error. As a consequence when backtesting the risk model
we may reject the true risk model just because its parameters were estimated with error. As
discussed in Engle and Manganelli (2004) this is mainly an issue when the risk model is evaluated
in-sample that is when the model is evaluated on the same data in was estimated on. In typical
external back-testing applications, the backtesting procedures are used in a more realistic out-ofsample fashion where the model is estimated on data up until day t and then used to forecast risk
for day t + 1. In this setting parameter estimation error is less likely to be critical. Parameter
estimation issues in relation to V aR modeling has been analyzed in detail in Escanciano and Olmo
(2007), and Giacomini and Komunjer (2005).

References
[1] Andersen, T., T. Bollerslev, P. Christoersen, and F. Diebold (2005), Practical Volatility and
Correlation Modeling for Financial Market Risk Management, in M. Carey and R. Stulz (eds.),
Risks of Financial Institutions, University of Chicago Press for NBER, 513-548.
[2] Basle Committee on Banking Supervision (1996a), Amendment to the Capital Accord to Incorporate Market Risks, Bank for International Settlements.
[3] Basel Committee on Banking Supervision, (1996b), Supervisory Framework for the Use of
Backtesting in Conjunction With the Internal Models Approach to Market Risk Capital
Requirements, Bank for International Settlements.
[4] Basle Committee on Banking Supervision (2004), International Convergence of Capital Measurement and Capital Standards: a Revised Framework.
[5] Berkowitz, J. (2001), Testing Density Forecasts, Applications to Risk Management Journal of
Business and Economic Statistics, 19, 465-474.
[6] Berkowitz, J., P. Christoersen, and D. Pelletier (2007), Evaluating Value-at-Risk Models with
Desk-Level Data, Manuscript North Carolina State University.
[7] Berkowitz, J. and J. OBrien (2002), How Accurate are the Value-at-Risk Models at Commercial Banks? Journal of Finance, 57, 1093-1112.

10

[8] Bontemps, C. and N. Meddahi (2005), Testing Normality: A GMM Approach, Journal of
Econometrics, 124, 149-186.
[9] Campbell, S. (2007), A Review of Backtesting and Backtesting Procedures, Journal of Risk,
9, 1-17.
[10] Christoersen, P. (1998), Evaluating Interval Forecasts, International Economic Review, 39,
841-862.
[11] Christoersen, P. (2003), Elements of Financial Risk Management, Academic Press.
[12] Christoersen, P. and F. Diebold (2000), How Relevant is Volatility Forecasting for Financial
Risk Management?, Review of Economics and Statistics, 82, 1-11.
[13] Christoersen, P., J. Hahn and A. Inoue (2001), Testing and Comparing Value-at-Risk Measures Journal of Empirical Finance, 8, 325-342.
[14] Christoersen, P.F. and D. Pelletier (2004), Backtesting Value-at-Risk: A Duration-Based
Approach, Journal of Financial Econometrics, 2, 84-108.
[15] Crnkovic, C. and J. Drachman (1996), Quality Control, Risk, September, 138-143.
[16] Diebold, F.X., T. Gunther, and A. Tay (1998), Evaluating Density Forecasts, with Applications
to Financial Risk Management, International Economic Review, 39, 863-883 .
[17] Dufour, J.-M., (2006), Monte Carlo Tests with Nuisance Parameters : A General Approach
to Finite-Sample Inference and Nonstandard Asymptotics in Econometrics, Journal of Econometrics, 133, 443-477.
[18] Engle, R.F. and S. Manganelli (2004), CAViaR: Conditional Autoregressive Value-at-Risk by
Regression Quantiles, Journal of Business and Economic Statistics, 22, 367-381.
[19] Escanciano, J. and J. Olmo (2007), Estimation Risk Eects on Backtesting For Parametric
Value-at-Risk Models, Manuscript, City University London.
[20] Giacomini, R., and I. Komunjer (2005), Evaluation and Combination of Conditional Quantile
Forecasts, Journal of Business and Economic Statistics, 23, 416-431.
[21] Hendricks, D. (1996), Evaluation of Value-at-Risk models using historical data, Economic
Policy Review, Federal Reserve Bank of New York, April, 39-69.
[22] Jorion, P. (2002), How informative are Value-at-Risk disclosures? Accounting Review, 77,
911-931.
[23] Kupiec, P. (1995), Techniques for Verifying the Accuracy of Risk Measurement Models, Journal
of Derivatives, 3, 73-84.
11

[24] Lopez, J. (1999), Regulatory Evaluation of Value-at-Risk Models, Journal of Risk, 1, 37-64
[25] Lopez, J. and M. Saidenberg, Evaluating Credit Risk Models, Journal of Banking and Finance,
24, 151-165.
[26] McNeil, A. and R. Frey (2000), Estimation of Tail-Related Risk Measures for Heteroskedastic
Financial Time Series: An Extreme Value Approach, Journal of Empirical Finance, 7, 271-300.
[27] OBrien, J. and J. Berkowitz (2005), Bank Trading Revenues, VaR and Market Risk, in M.
Carey and R. Stulz (eds.), Risks of Financial Institutions, University of Chicago Press for
NBER.
[28] Perignon, C., Z. Deng and Z. Wang (2006), Do Banks Overstate their Value-at-Risk? Manuscript, HEC Paris.
[29] Perignon, C. and D. Smith (2006), The Level and Quality of Value-at-Risk Disclosure by
Commercial Banks, Manuscript, Simon Fraser University.
[30] Pritsker, M. (2001), The Hidden Dangers of Historical Simulation, Finance and Economics
Discussion Series 2001-27. Board of Governors of the Federal Reserve System.
[31] Smith, D. (2007), Conditional Backtesting of Value-at-Risk Models, Manuscript, Simon Fraser
University.

12