Response to consultation on Guidelines on ESG scenario analysis
Question 1: Do you have any comments on the interplay between these Guidelines and the Guidelines on the management of ESG risks?
It should be made clear that the focus is on financial risks faced by individual banks, not systemic risks, which are matters for regulators. This has profound implications for the kind of scenarios that banks need to use, since, as explained later, climate-related financial risks are much greater and more immediate than the conventional official scenarios generally suggest. Moreover, contrary to the presumptions of such scenarios, they stem not just from the physical climate or climate policy, but also from the volatile endogenous financial, political and economic dynamics that are absent from the models on which they rely.
Question 2: Do you have comments on the proposed definition of scenario analysis and its various uses as presented in Figure 1?
Paragraph (henceforth P) 10
Re. “scenario analysis is a process for identifying and assessing the potential implications of a range of plausible future states of the world under conditions of uncertainty. Scenarios are hypothetical constructs and not designed to deliver precise outcomes or forecasts.”
This it at odds with later references to 'probability-adjusted risks' and 'most likely' central scenarios...see paragraph 25
P11
Note - the complexity and radical uncertainty of ESG risks indeed justifies the use of scenarios. But also highlights the problems of the backward-looking and model-based approach of traditional risk management.
Figure 1
It should be emphasised that the use cases developed in the guidelines, relating to financial and business model resilience, relate to financial materiality (inbound risk). They require distinctly different types of scenarios from those needed for other use cases, notably alignment strategies and (impact) transition planning, which relate to impact materiality (outbound risk).
Question 3: Do you have comments on the proposed distinction made between short-term scenario analysis (CST) and longer-term scenario analysis (CRA) as illustrated in Figure 3?
P15
None of the currently published scenarios are fit for purpose. Long term systemic scenarios focused on global warming are of little use for assessing the short and medium term bank-specific financial risks that are the focus of CST and CRA (see later). Thus, knowing the global average temperature in 2050 will be of little help for a German bank assessing flood risks in 2026, let alone the possibility that a trade war might turn its exposure to a European auto manufacturer into a stranded asset before 2030.
See the following references:
Cliffe, M. A., Abrams, J.F., Clark, M., Lenton, T. M., Oliver, J. (2023) No Time to Lose. University of Exeter and Universities Superannuation Scheme.
Nowzohour, L., Dees, S. et al (2023). Conceptual note on short-term climate scenarios. Network for Greening the Financial System Technical document.
Cliffe, M. (2023) Planet NGFS Starts To Look Like The Real World https://markcliffe.wordpress.com/2023/10/06/planet-ngfs-starts-to-look-like-the-real-world/
Trust, S., Joshi, S., Lenton, T.M., Oliver, J. (2023). The Emperor’s New Climate Scenarios. Institute and faculty of Actuaries and University of Exeter.
Stern, N., Stiglitz, J., Taylor, C. (2022). The economics of immense risk, urgent action and radical change: towards new approaches to the economics of climate change. Journal of Economic Methodology, 181-216.
Cliffe, M. A., (2023). What planet are we on?. The Actuary.
Lenton, T. M. et al (2023). Global Tipping Points Summary Report 2023. University of Exeter. Hampton, S., Whitmarsh, L., Moorcroft, H. (2024).
Cliffe, M. A., (2024). Climate Scenarios Need Realism, Not Idealism. pp. 14-19 in Accelerating Transition, Kings Business School, Edited by Carlin, D., Aikman, D., Lepere, M., Taschini, L.
Transition Plan Taskforce (2024). Sector Guidance.
P16
The reassertion that specific channels need to be incorporated into models before scenarios are produced is questionable. The use case should define the key drivers, determined by their materiality and uncertainty. These frame the narratives, but since some channels are not modellable, they require expert judgement.
Figure 2
Two key channels are missing:
1. geo-politics
2. financial volatility
Re. 1
Banks need to consider scenarios that also include policy and regulatory reversals
Re. 2. Expectations of value of climate-related financial assets can shift suddenly and unexpectedly (aka 'Minsky moments')
Reference: Cliffe, M. A., Abrams, J.F., Clark, M., Lenton, T. M., Oliver, J. (2023) No Time to Lose. University of Exeter and Universities Superannuation Scheme.
P17
Many key variables are unmodellable and lack data - banks therefore need to cultivate and curate expert judgement on these variables by developing or accessing networks of internal and external experts on the relevant disciplines.
P20
Regarding resilience, the institution's success or failure to meet its impact goals (such as Net Zero) is of little relevance. However, its impact behaviour relative to its competitors is relevant to its financial performance. This is another reason why systemic (macro prudential) scenarios need to be clearly distinguished from bespoke institutional (micro-prudential) scenarios.
Reference: Cliffe, M. A., (2023). What planet are we on?. The Actuary.
Figure 3
Note that the table emphasises ‘hypotheses not forecasts’ – this is very important, and casts doubt on the preoccupation with data and modelling elsewhere in the document. It highlights the need for investment in expert judgement.
Re. the penultimate bullet on CRA, resilience should not be tested solely against a single 1.5C pathway.
P25
Re. “…factoring in the ambition set out in their transition plan”
For now, such ambitions are voluntary and the prospect of sanctions for failing to meet them is remote. Moreover, there is no recognition of the potential for 'paper decarbonisation', whereby banks off-load high emitting exposures. This highlights the fundamental weakness of the current policy framework's faltering attempts to co-opt banks into the pursuit of the EU's Net Zero goals.
Question 4: Do you have any comments on the interplay between these Guidelines and the Guidelines on institution’s stress testing?
See answer to Question 3. The key point is to make geo-political and financial volatility (endogenous) drivers of the scenarios. In particular, financial price volatility is a both a cause as well as a consequence of climate-related financial risk.
Question 5: Do you have comments on the Climate Scenario Analysis framework as illustrated in Figure 4?
Figure 4
1. Note (a) CST is focused purely on financial (single) materiality, and (b) even for CRA, impact (double) materiality is only indirectly relevant, since the business resilience will depend also on the behaviour of others (notably policy makers and competitors).
2. In both cases, perhaps it would be appropriate to focus also on a 'planned' balance sheet approach, i.e. to include the bank's transition plans.
7. re. CST also use to: Modify capital allocation plans (by product, geography and sector), credit approval processes, risk monitoring ('situational awareness') processes
Re. CRA also use to: Refine central scenario based on a fuller understanding of the range and profile of risks, modify strategic plans and business model, revise interim impact (e.g. Net Zero) targets and routes to achieving them (esp divest vs. engagement with borrowers)
P28
The other key driver of proportionality is uncertainty. Some variables may be material but are relatively certain (eg. chronic physical risk on horizons of less than 5 years), and therefore have little effect in differentiating between scenarios. By contrast, material variables which are highly uncertain should be key drivers of scenario narratives
Reference: Cliffe, M. A., Abrams, J.F., Clark, M., Lenton, T. M., Oliver, J. (2023) No Time to Lose. University of Exeter and Universities Superannuation Scheme.
P35
Re. “the work carried out by the NGFS on short- and medium-term scenarios, on physical risk scenarios and on nature-related risks, is another stream of work to be followed. Scenarios covering time horizons of 5 years or less could find their place in the implementation of climate stress tests”.
This work promises to substantially improve the relevance and usefulness of the NGFS scenarios, although it is likely to continue to suffer from some of the existing limitations of NGFS long term scenarios.
Reference: Cliffe, M. (2023) Planet NGFS Starts To Look Like The Real World https://markcliffe.wordpress.com/2023/10/06/planet-ngfs-starts-to-look-like-the-real-world/
P38
“All these limitations call for great caution when translating the outcomes of climate scenario analysis derived from traditional macroeconomic models into decisions.”
As noted earlier, the existing approach is not fit for purpose, featuring unrealistic narratives and inappropriate equilibrium modelling.
As an alternative, I would recommend evaluation the E3ME-FTT model from Cambridge Econometrics, see: https://www.e3me.com/
P41
Re. “For Climate Resilience Analysis (CRA), in view of the impossibility of assigning meaningful probabilities to each scenario, institutions should consider the findings from the full range of the scenario set and not only focus on low-impact scenarios.” This is a crucial point, which highlights the subjective nature of the institution’s choice of “most likely” central scenarios, which inevitably rest largely on expert judgement, NOT modelling.
Question 6: While respecting the definitions provided in other parts of the regulation, is there any concept/s used in these guidelines that it would be useful to include in an annexed glossary?
There are a number of concepts which are not used in these guidelines which should be addressed as well as added to the glossary.
In particular, financial and impact materiality, and relatedly single and double materiality, need to be explained and their relevance to different use cases explained. There should also be an explanation of the distinction between ‘climate risk’ (which in common usage tends to be associated with physical risk, overlooking transition risks) and ‘climate-related financial risk’, which can be triggered by factors not directly related to the physical climate or climate policy (e.g. political or economic shocks that impinge upon the value of climate-related financial assets).
Question 7: Do you have comments on section 4.1 Purpose and governance?
It is good to see the emphasis on embedding scenario thinking into the organisational culture.
In addition:
P 15
Re. the preparation of a “most likely” “credible, coherent narrative”, this should arguably be developed (or least revised) after the scenario set has been prepared.
Question 8: Do you agree that the proposed proportionality approach is commensurate with both the maturity of the topic and the size, nature and complexity of the institution’ s activities?
Yes, but it is not just about proportionality in terms of materiality of the risks, but also terms of the degree of uncertainty surrounding them. Some material risks face relatively little uncertainty (e.g. chronic physical risks over short horizons) and are therefore less important in differentiating scenario outcomes. In other words, choice of scenario drivers is a function both of materiality and uncertainty.
P27
Finance itself is a key source of transition risk. The value of climate-related financial assets is dependent on market expectations which are subject to potentially dramatic shifts (see also P29)
Re “population growth, economic development and social inequalities” - these factors are not major risk drivers over horizons of less than 5 years
Re. climate policies - Policy is heavily dependent on geo- and domestic politics. Scenarios need to consider the reality that policy can go into reverse!
P28
'Scientific knowledge' needs to be broadly defined, to include political and social sciences and financial analysis. There is an implicit tendency to equate ‘science’ with the physical sciences…
Question 9: Do you agree with the proposed references to organisations in paragraph 28? Would you suggest alternative or complementary references?
No, I do not agree…“…elaborated by widely recognised international or regional organisations such as the Network for Greening the Financial System (NGFS), the Joint Research Center of the EU Commission (EU JRC) or national (government) bodies.” See above for reservations on the methodology and focus of their scenarios…also comments on 27 & 28, and section 3.3 “A long journey ahead” on pages 15-17 of these guidelines.
Question 10: Do you have additional comments on section 5.1 Setting climate scenarios?
See below. Note especially the point that short term drivers are very different (29) and that institutions need bespoke scenarios (33, 34 and 36). Also note the non-linearity point re 39.
P29
In other words, the drivers of short term scenarios are very different! This means different narratives, assumptions, modelling and methods. Market sentiment - particularly in financial markets, is especially important.
For more details on a narrative driven approach, see:
Cliffe, M. A., Abrams, J.F., Clark, M., Lenton, T. M., Oliver, J. (2023) No Time to Lose. University of Exeter and Universities Superannuation Scheme.
P30
A good fit between the defined scenarios and the unique risk characteristics of the institution’s portfolios and business model means that global systemic risks (the focus of NGFS-style scenarios) are less relevant.
P34
The policy reversals triggered by the US Administration are a graphic case in point.
P35
Re. “Delays and international divergences in climate policy” – add “reversals” (missing from NGFS scenarios). An important implication is that even “green” assets may be stranded in some scenarios.
P39
“When defining their baseline scenario, institutions should assume a continuation of current conditions and trends and that, except in special cases, there is no significant worsening in terms of assumptions and in the underlying climate, macroeconomic and financial variables.”
Why? There is mounting scientific evidence and experience of non-linear increases in some risks, e.g. acute physical and geo-political risks.
P41
A key lesson of complexity science is that institutions should start with short horizons, given the path dependency of many variables.
Question 11: Do you have comments on the description of the climate transmission channels?
P45
See page 14 re proportionality - it is about uncertainty as well as materiality
P46
Note that the causation runs both ways: macroeconomic conditions will have a major impact on the pace and profile of the transition and hence on the profitability of climate-related financial assets
Question 12: Do you have comments on climate stress test (CST) tool and its use to test an institution’s financial resilience?
Expert judgement is essential given the uncertain, unprecedented changes involved, which are beyond the scope of conventional models.
Question 13: Do you have comments on the Climate Resilience Analysis (CRA) tool and its use to challenge an institution’s business model resilience?
A key problem surrounds the consistency of financial and impact materiality. The former depends on the institution's transition strategy RELATIVE to its peers, while the latter depends only on its absolute performance.
Reference: Cliffe, M. A., (2023). What planet are we on?. The Actuary.
P74
Re. “Institutions should provide detailed qualitative and quantitative information, considering proportionality as outlined in section 4.2, on the robustness of the strategy over a manageable time horizon (e.g., 5 years). Beyond that, as the time horizon lengthens, the approach may only offer a general approximation…” This is critical. Institutions should concentrate on their modelling efforts on sub 5 year horizons, and then only after having produced realistic narratives (given that key drivers such as politics are inevitably qualitative). Model choice is dictated by the use case. Long term equilibrium models are not appropriate for CST or CRA.
P75
What if the institution's central scenario is not consistent with 1.5C? - see also paragraph 80
P79
A key point here is that tail risks and negative skews are likely features of these scenario sets.
Question 14: Do you have any additional comments on the draft Guidelines on ESG Scenario Analysis?
It is critical to underline the key points made earlier:
1. The long term systemic (macroprudential) scenarios from the NGFS and other official bodies are generally not fit for purpose. CST and CRA primarily require short term financial risk scenarios with requisite volatility and alignment to the individual institution’s business*.
2. Such scenarios require different narratives, assumptions, modelling and methods. Geopolitics and market sentiment, particularly in financial markets, are key drivers of such scenarios.
Moreover, recent events and backtracking on ESG highlight the fact that regulators should not gloss over the primacy of financial materiality (inbound risk) over impact materiality (outbound risk). The resilience use cases in these guidelines are quite distinct from the transition planning use case noted earlier.
Is this context, it should also be noted that the recent EU Commission Omnibus package has effectively watered-down obligations on institutions to implement (as opposed to merely report) transition plans, and the roll out has in any case been delayed by two years. More importantly, there is still nothing to stop banks engaging in 'paper decarbonization' by reducing exposure to high emitting borrowers. In contrast to active engagement with such borrowers to encourage them to transition, this means that systemic risk will not be reduced or even increase insofar as exposure is switched to less lightly regulated private ownership or to NOCs beyond the reach of EU regulators.
Moreover, incentives to engage in regulatory arbitrage will be increased if non-EU regulators fail to mandate double materiality for banks in their jurisdictions. Keeping this in mind, it is not clear, as these guidelines suggest, that pursuing ESG goals will put EU banks at a competitive advantage. On the contrary, there are scenarios in which such a pursuit would put them at a competitive disadvantage.
Prudence dictates that given the risk of further changes to EU and global regulation, banks need to incorporate a variety of different regulatory pathways into their scenarios and factor this into their planning and risk management.
*in this context, note:
Re. Page 41
Point 3) Use of scenarios from widely recognised organisations
“Paragraph 3 of Article 87a(5) of Directive 2013/36/EU directs the choice towards the use of scenarios developed by widely recognised organisations. The organisations that develop such type of scenarios have significant expertise, which makes them a reliable and robust source. At the same time, the usage of scenarios developed by recognised organisations would allow for a better degree of comparability across different institutions. However, such scenarios may not fully adapt to institutions’ own characteristics and risks. Therefore, the EBA considers that it would be adequate to introduce a degree of flexibility and encourage institutions to make changes to these scenarios." [my emphasis]
This significantly understates the scale of the problem (see references referred to earlier).