- Question ID
-
2023_6747
- Legal act
- Regulation (EU) No 575/2013 (CRR)
- Topic
- Credit risk
- Article
-
180
- COM Delegated or Implementing Acts/RTS/ITS/GLs/Recommendations
- EBA/GL/2017/16 - Guidelines on PD estimation, LGD estimation and the treatment of defaulted exposures
- Article/Paragraph
-
pargraph 8
- Type of submitter
-
Credit institution
- Subject matter
-
PD calibration sample
- Question
-
Given the definition of PD calibration provided in EBA/GL/2017/16 section 2.4 paragraph 8, and the requirements for the calibration sample provided in section 5.3.5, paragraph 88 of the same guidelines, for developing a TTC model, clarification is needed on the expectation on the implementation of the back-testing performed in the validation phase.:
- Shall the back-testing at portfolio level verify that the average PD over historical observation period is aligned with LRA DR or, instead, shall the comparison be made between PD estimates current at the validation date and the LRA DR? Does it change according to the rating philosophy? Shall the back-testing always be performed on a 1-year validation sample, regardless the type of TTC calibration philosophy and regardless the length of the calibration sample?
- How shall the rating philosophy be taken into consideration when assessing the outcome of back-testing at grade level?
- Provided that the main aim of the calibration is to reflect the LRA DR, is the any case where the alignment to 1-year default rate should get a higher weight in validation assessment, although in a TTC calibration philosophy?
- Background on the question
-
EBA guidelines on PD estimation, LGD estimation and treatment of defaulted exposures (EBA/GL/2017/16) in section 2.4, paragraph 8 define the “PD calibration” as " The part of the process of the estimation of risk parameters which leads to appropriate risk quantification by ensuring that when the PD ranking or pooling method is applied to a calibration sample, the resulting PD estimates correspond to the long-run average default rate at the level relevant for the applied method". Furthermore the same paragraph defines the “PD calibration sample” as “The data set on which the ranking or pooling method is applied in order to perform the calibration.” Therefore the calibration sample is generically defined as “a data set” without qualifying if it should be a recent single snapshot portfolio or a multiyear sample. Subsequently, according to section 5.3.5, paragraph 88 of the same guidelines states that “In order to ensure compliance with Article 180(1)(a) or 180(2)(a) of Regulation (EU) No 575/2013, institutions should find an appropriate balance between the comparability of the calibration sample with the application portfolio in terms of obligor and transaction characteristics and its representativeness of the likely range of variability of default rates as referred to in section 5.3.4.”. Thus the calibration sample must be chosen in order to have an “appropriate balance” between its comparability with the application portfolio (that might be interpreted as a representativeness with obligor and transactions characteristics, as recalled in point c) of section 4.2.4. par. 28 as well as representativeness of the current riskiness) and its representativeness of the likely range of variability of default rates as for the Long run default rates (due to the reference to section 5.3.4) where a multiyear perspective would be required in order to observed a variability of 1-year default rate. As a consequence of this it is not clear if this paragraph requires mandatorily to adopt a calibration sample covering multi yearly snapshots (choosing nevertheless a calibration sample which guarantees the representativeness of the current obligors characteristics), or if the use of the current portfolio as calibration sample is preferred for a TTC model, and always allowed, in order to ensure to have current PD estimates corresponding to the long-run average default rate, at the level relevant for the applied calibration method.
In particular, the extract at page 24-25 (supporting the explanation provided in table 1, page 25 of EBA/GL/2017/16 on the possibility to adopt "TTC B" calibration philosophy) is not clear to us, and we would need a clarification on the meaning of this specific extract: " This would be an example of a ‘TTC B’ calibration, as indicated in Table 1, and would be applicable for own funds requirements calculation if all other requirements are met, in particular those on the calibration sample (the chosen point in time would need to reflect a sample comparable to the current portfolio and representative of the likely range of variability of one-year default rates).". It is critical to understand how the chosen point in time is representative of the likely range of variability of one year default rate considering that the PD estimates after calibration would be determined conditioned to the type of calibration sample defined. A clarification on this topic is relevant in order to define the approach for rating dynamics measurement and interpretability of calibration test outcomes within the validation phase.
- Submission date
- Final publishing date
-
- Final answer
-
As per the back testing, Article 185(b) CRR foresees that, in the context of validation process: “institutions shall regularly compare realised default rates with estimated PDs for each grade and, where realised default rates are outside the expected range for that grade, institutions shall specifically analyse the reasons for the deviation”. Furthermore, Article. 185(c) requires that: “Institutions' internal assessments of the performance of their rating systems shall be based on as long a period as possible”, and Article 185(e) requires that "institutions shall have sound internal standards for situations where deviations in realised PDs, LGDs, conversion factors and total losses, where EL is used, from expectations, become significant enough to call the validity of the estimates into question. These standards shall take account of business cycles and similar systematic variability in default experience. Where realised values continue to be higher than expected values, institutions shall revise estimates upward to reflect their default and loss experience;". As for para. 8 of EBA/GL/2017/16, the calibration aims at ensuring that the PD estimates are aligned to the long-run average default rate at the level relevant for the applied calibration method.
In addition to this, the EBA/REP/2023/29 collecting best practices suggests to run tests not only considering the full historical data, but also multiple sub-periods (e.g. for any single year - including the latest 12- months period, most recent three years, most recent five years, one economic cycle, etc.). Running tests on multiple sub-periods is considered as a best practice regardless the rating philosophy of the model, which however plays a central role in evaluating potential divergences between PD estimates and default rates (see paragraph 66(c) of EBA/GL/2017/16).
Paragraph 66(c) of EBA/GL/2017/16 recognises that: “Philosophies sensitive to economic conditions tend to estimate PDs that are better predictors of each year’s default rates. On the other hand, philosophies less sensitive to economic conditions tend to estimate PDs that are closer to the average PD across the various states of the economy, but that differ from observed default rates in years where the state of the economy is above or below its average.
Deviations between observed default rates and the long-run average default rate of the relevant grade will hence be more likely in rating systems less sensitive to economic conditions. In contrast, migrations among grades will be more likely in rating systems which are more sensitive to economic conditions. These patterns should be taken into account when assessing the results of back-testing and, where relevant, benchmarking analysis.”
Therefore, the abovementioned principle recognizes that, in case philosophies more sensitive to economic conditions are chosen (i.e. more Point In Time), PDs are expected to be better predictors of each year’s default rate, while philosophies less sensitive to economic conditions (i.e. more Through the cycle) tend to estimate PDs that are closer to the average PD across the various states of the economy. This consideration should be reflected in the evaluation of the back testing. Consequently, tests and validation procedures should be designed to take into proper consideration the rating philosophy adopted, for example by using the rating philosophy to ensure comparability between estimated and observed default rates. Considering that each model has its specificities and PD models may also have a hybrid rating philosophy, it is not possible to define generally which tests are more significant for assessing model performance. Hence, the significance of test results has to be tailored to the model the tests are applied to during validation.
To that effect, the good practice set out in EBA/REP/2023/29 further suggests that “for the back-testing of PD estimates, where the realised one-year default rate in a grade or pool falls outside the expected range for that grade or pool, the validation function is expected to analyse the deficiency… In this context, it is considered as best practice to consider the deviation in light of:
Whether this deviation happened during an extreme year considering the business cycle (i.e. bad year in case of default rate exceeding PD estimates and vice versa) as reflected by economic indicators that are relevant for the considered type of exposures;
The rating philosophy, i.e. how the business cycle interacts with the related systematic variability in default experience;
The results for other sub-periods”
- Status
-
Final Q&A
- Answer prepared by
-
Answer prepared by the EBA.
Disclaimer
The Q&A refers to the provisions in force on the day of their publication. The EBA does not systematically review published Q&As following the amendment of legislative acts. Users of the Q&A tool should therefore check the date of publication of the Q&A and whether the provisions referred to in the answer remain the same.