Primary tabs

Operational Riskdata eXchange (ORX)

It is not clear that all of our comments will fit into this text box, please also see the attached document.

ORX supports the aims of the guidance, and sees it as a significant step towards promoting consistency within the EU. There are however, a few areas we believe could benefit from further clarification.

Within this response, ORX first seeks some clarity on the technical application of certain topics, and suggest some enhancements that may practically help the EBA achieve its goals with the proposals, and avoid some unintended pitfalls that ORX has identified may arise from a data collection perspective.

The below table lists the areas ORX believes would benefit from further clarity and elaboration. Any recommended changes to Articles have been highlighted with red text.

Article Topic Issue
1 Model Risk Referred to in the directive, but not defined
2 Legal Risk Lack of clarity due to the reference to “risk of being sued’ in the definition
5 Operational Risk events related to Market Risk Lack of clarity due to disconnect between article title and scope of text
7 Timing Losses Practicality of data collection and thresholds
17 External Loss Data Ability of ORX Members to comply
21 Date of last change to loss amount Practicality of data collection and thresholds
22&23 Modelling Interaction between model standards and Use Test
25 Expected Losses Clarification of key terms

The most common topic for feedback from ORX Members has been on Article 6 Fraud Events in the Credit Area. This is addressed in our response to Question 2.

Model risk is referred to directly in Article 1§2 and indirectly, in Article 5 paragraphs 2, 3 and 5, however, there is no supporting definition in Article 2. The lack of a definition for Model Risk in may result in inconsistency of interpretation and data collection.

An explicit reference should be made to Article 3 § 1 (11) of Directive 2013/36/EU (so-called CRD IV). However, we think that in order to promote consistency, it is important that any definition of model risk is kept in agreement with that of other regulatory bodies (APRA and the Federal Reserve ).

If the EBA considered adding supporting text in additional articles, the following concepts derived from ORX members may help the EBA document that text:
• Incorrect inputs into model (for example, data that is incomplete, inaccurate, or not fit for purpose)
• Inappropriate processing or programming logic applied to a model;
• Inappropriate underlying theory, assumptions, relationships or distributions used within the model;
• Incorrect modelling development (data and assumptions);
• Unreasonable assumptions used in a model build;
• Incorrect model implementation into the system;
• Using a system that has the potential of being tampered (intentionally or unintentionally) e.g. manual spreadsheet
• Use of a model for a purpose for which it was not intended and / or designed; incorrect or misleading interpretation of results from the model;
• Incorrect model outputs not being identified; or
• Lack of appropriate ongoing monitoring of model performance to confirm the model remains fit for purpose
• Approved / unapproved manipulation of the modelling parameters which change the original purpose of how the model output is applied.

There is a concern with the inclusion of “the risk of being sued” in the definition of Legal Risk. The risk of being sued is heavily influenced by the jurisdiction. In turn this makes the definition of Legal Risk subjective as it creates issues around reliably estimating the risk of being sued.

ORX has aimed to minimise this jurisdictional influence by having separate definitions and descriptions for legal risk and litigation. “Litigation” is generally perceived as a dispute resolution mechanism. As a result it includes not only going to court, but also tribunals and arbitration of various forms. For firms capturing this litigation data, the trigger is the receipt of a letter from a plaintiff or the plaintiff’s representative.

It is suggested that “the risk of being sued” is removed from the definition. The definition would then read

‘Legal Risk’ means the risk of loss from non-compliance with legal or statutory responsibilities and/or inaccurately drafted contracts. It also includes the exposure to newly enacted laws as well as to changes in interpretations of existing laws.

Such an inclusion in the RTS would provide additional clarity and focus for Article 4 – “Operational Risk Events related to Legal Risk”.

Some initial interpretations by ORX members limited the scope to the calculations of VAR or the Standardised Approaches. Further items mentioned in §2 and §3 are operational risk and are part of the lifecycle of a transaction, but not related to Market Risk. The lack of clarity has the potential to result in inconsistent interpretation and collection of loss data.

It is proposed that “Market Risk” be replaced, for example by “Trading Book”. Firms with Market Risk calculations will, by definition have a trading book. The trading book provides a recognised boundary and will promote consistency.

It is proposed to remove the requirement that transaction errors in the trading book be separately identifiable, as transaction errors are not limited to the trading book.

It is recommended that this Article is explicitly referred to in a forthcoming Market Risk RTS in recognition of regulatory consistency.

This section is silent on the materiality test to be applied to collecting this data. The absence of guidance may result in firms applying their own thresholds leading to inconsistencies is data used for capital calculation purposes.

Firms are likely to follow their accounting requirements, such as FASB or IASB. The IFRS guidance on materiality is:
Information is material if omitting or misstating it could influence decisions that users make on the basis of a reporting entity’s financial information. To determine whether information is material, both the nature and magnitude of the item(s) to which the information relates must be considered in the context of the individual entity’s financial report.

It is suggested the clause is amended to read:
“material timing losses that span more than one accounting year and give rise to the risk of litigation”.

With reference to the definition provided in Article 2§27, the definition of Timing Losses would benefit by referencing “annual accounting periods”. Whilst firms may be required to provide accounts quarterly, semi-annually or annually (dependent on their listing requirements), the selection of annual accounts would promote consistency between firms as these accounts tend to be the most formal (audited and published).

ORX welcomes the effort by the EBA to promote consistency around operational risk loss data.

With reference to §1,
1. EU Members may not be able to provide all of their internal data to data consortia without creating jeopardy. A firm may establish a reserve/provision related to the estimated litigation outcome, but under some jurisdictions this reserved amount is discoverable by the plaintiff’s lawyers. Provided that the size and purpose of the reserve/provision is not widely distributed then it is treated as privileged information and non-discoverable. Requiring the reporting of this data to a data consortium may create additional jeopardy for the firm. For ORX, reporting reserves/provisions related to on-going litigation is desirable, but voluntary. Once the litigation has completed then it is compulsory to report the loss amount to ORX.

2. The EBA definitions and requirements may not be consistent with supervisory jurisdictions outside the EU, thereby limiting the volume of data available for modelling. Consortia will receive loss data from members from many countries. Whist consortia data could be tagged to identify data points originated from a firm subject to the EBA requirement, thereby enabling EU firms to use external data that met EU requirements; this reduces the volume of data available for modelling. For example, consider a unit of measure (or risk category) of Technology Failure (e.g. Model Risk) in Trading & Sales. This unit of measure is unlikely to have many data points today, and in the future will have less due to the differences in definitions of Model Risk.

The EBA should give consideration to the issue of jeopardy and ways to ensure consistent and timely submission of data to the consortia.
ORX encourages the EBA and other regulatory groups, such as the Basel Operational Risk Working Group to work together to achieve greater consistency in the scope of losses to be categorised as operational risk. This will benefit the regulatory community, for example enabling benchmarking loss data, the firms and data consortia.

The interpretation of this requirement is that firms will need to record the last date on which the loss amount changed. This reference date is then to be used in determining the inclusion of the loss in the dataset for the AMA calculations.

This will add a significant level of complexity to the databases of firms. They will now need to track four date types:
1. Date of Occurrence
2. Date of Discovery
3. Date of Recognition and
4. Date of last change in loss amount.

The current guidance is that loss data should be included in the AMA dataset for 5 years after the date of recognition. This paragraph appears to be amending that requirement to 5 years after the last change in the loss amount.

To reduce the implementation burden and more quickly achieve the desired result it is recommended that a phased approach is adopted. The immediate scope could be larger losses and as experience is gained consideration is given to reducing the threshold.

Regarding “clear capital substitutes or otherwise”, it is not clear if it is restricted to Tier 1 or 2 eligible instruments or also include profits & losses. For the trading book, the positions are marked to market daily with the resultant “expected losses” being posted against profit & loss accounts. For credit impediments there is the ability to establish reserves/provision from the profit & loss accounts.

This issue is likely to result in inconsistencies between firms.

The EBA is requested to clarify what constitutes “clear capital substitutes or otherwise”.

The clarification and choices may be influenced by the granularity for the “Expected Loss”. For example, if “Expected Losses” are determined at the level of risk category or unit of measure, then these may not correspond to business units. If the instruments are Tier 1 or 2 eligible then these tend apply to the organisation rather than a risk category or unit of measure.

From a measurement perspective ORX welcomes the depth and clarity given within the EBA paper. ORX believes the guidance will help to ensure good modelling practice across the industry.

There are some instances where the requirements call for a high level of statistical specificity. In particular, two examples of this are:
a) Article 22 §2 “An institution shall strive to get operational risk categories with homogeneous, independent and stationary data”. This is a challenging requirement which is hard to satisfactorily demonstrate.

b) Article 23 §2 states that preference should be given to tools for “evaluating the appropriateness of the distributions to the data, giving preference to those most sensitive to the tail”. For Article 23 §2 ORX believes over reliance on statistical measures may be in obstruction of the Use Test.

ORX believes that more clarity relating to the balance between idealized statistical accuracy, model stability and the Use Test would be very welcome.

ORX welcomes this effort by the EBA to promote consistency around operational risk loss data. To facilitate the delivery of consistent data ORX established the Definitions Working Group (DWG). Much of the work of the DWG is devoted to responding to data categorisation queries raised by firms and distributing the outcomes in the form of Interim Updates to the ORX Operational Risk Reporting Standards. ORX encourages the EBA to establish a similar panel to discuss and publish the discussions on data categorisation to support national regulators and consistency amongst firms. This panel of experts could also serve as a contact point for discussions with individual non-EU regulators or other regulatory groups.
It is not clear that all of our comments wil fit into this text box, please also see the attached document.

ORX welcomes this effort by the EBA to clarify the boundary between Credit and Operational Risks.

This article raised the most concern from ORX Members. Conceptually, ORX supports the proposals, the issues relate to practicalities:
1. Active support by Credit Risk regulators and management functions within firms,
2. Transaction versus products
3. Thresholds,
4. Modelling Issues,
5. Implementation and Compliance Date

Experience of discussing the boundary between Credit and Operational risks strongly indicate that this cannot be achieved by the operational risk management functions alone. Categorising losses due to fraud in the credit space as operational risk losses will require the active support of the credit risk management functions. These functions will respond to the direction provided by the credit risk regulators. Without this support it is unlikely that the Operational Risk Management functions will be able to meet these requirements.

Effectively, the process of collecting the data will be part of the Credit Risk Management activities. Credit Risk Management, as part of the review of a default, will determine whether to subject it to a fraud investigation. Implementation of the required changes by Credit Risk management will be facilitated if there is active and visible support from the Credit Risk regulators. The transitional arrangements need careful consideration given the use of historic data in regulatory approved capital calculation models for Credit and Operational Risk.

At this stage we cannot comment in detail upon the capital implications. However, it is expected to result in a reduction in Credit Risk capital requirements and an increase in Operational Risk. Whether this change will be 1:1 is impossible to say with the available information. Nevertheless, there are likely to be implications for the amount of capital required by the firm and influence the timing of compliance.

We understand there is a forthcoming Regulatory Technical Standard on Credit Risk. It is proposed that the RTS Credit should refer to Article 6 in the Operational Risk RTS. This will provide the clear sense of direction from the Credit Risk regulators.

From discussions with operational risk specialists, there are uncertainties about the scope of a fraud. More specifically, is a fraud assessed at the level of an individual transaction or if a fraud has taken place does the determination apply to all transactions involving the same product, e.g. loans, with the same counterparty or is it all transactions with that counterparty?

Clarification of this will promote consistency especially in connection to Article 21 §7 with its reference to single root event.

Further this is an area where knowledge of current practices in Credit Risk Management will help a broader understanding of the scope of the data to be captured and promote consistency.

It is recommended that input is obtained from Credit Risk regulators and management functions as to the scope of a determination of fraud i.e. individual transaction or product.

The implication of Article 6 §3 is that the threshold for investigating and collecting fraud losses should be the same as other operational risk losses. There is no guarantee that the threshold, currently operated by Credit Risk Management, to investigate fraud is the same threshold as used to collect other Operational Risk losses. Potentially applying a lower threshold than currently used by Credit Risk Management will increase the system and manpower requirements, significantly increasing the implementation challenge.

Taking a phase-in approach starting with the larger events increases the likelihood of compliance within the 2 year timeframe.

The actual choice of initial threshold must be made in consultation with the Credit Risk regulators and management functions.

For the loss data collection of fraud events, it is recommended that phased-in thresholds are used. This will enable the focus to be upon the largest events being subject to a “fix-back” process, and allow events at lower thresholds to be included in a “fix-forward” process, thus reducing the volume of data that has to be re-categorised as Operational Risk losses, which will facilitate the capture and migration of the data to Operational Risk.

Article 24 relates to aggregate loss distributions and risk measures. Within this article, paragraph 4 includes requirements about capping individual losses. The issue is that with the fraud events taking place in the credit risk space it is possible to cap the maximum loss. These losses may be better suited to be modelled using exposure or factor based models.

With regard to modelling and capping the maximum possible loss, this may benefit from a small change to the drafting to Article 24 §4; for example “capping the maximum single loss, if an institution cannot provide a clear objective rationale for the existence of an upper bound (e.g. in the case of fraud events in the credit area).”

The proposed compliance date is 2 years after the Directive enters into force. Given the significant issues identified above this is unlikely to be achievable, and recommend extending this in line with outcomes from discussion with Credit Risk supervisors.
The identified issues with this paragraph relate to the scope of the requirements and the data to be collected. It is assumed that AMA management” means the operational risk management function at division or group or corporate level.

While the concept of opportunity costs is well recognised the issue is one of going from theory to practice. There appear to be no readily available formulas for the calculation of opportunity costs or gains that can be used consistently and robustly within and between banks. This is expected to result in inconsistent data.

An additional concern related to the possible confusion between “lost revenues” and “uncollected revenues”. Again this is a source of potential inconsistency.

With regard to capturing “internal costs” the concern is one of practicality. For large remediation projects, firms may establish a specific temporary cost centre. What is “large” will vary from firm to firm as will the scope of internal data captured. For example, it may capture salaries or total staff costs, but bonuses and overtime may not be separately identifiable.

Given the above issues, it is proposed that Article 7 §2 be reconsidered, in particular the bullet points relating to opportunity costs / lost revenues” and “internal costs, such as overtime and bonuses” or reduce the expectation from “shall”.
Please refer to ORX’s responses to Question 1 which address specifically Legal Risk, Operational risk events related to market risk and timing losses.
ORX recognises that the modelling of dependence is challenging and a conservative approach is sensible, although analysis of industry operational risk loss data consistently implies general low levels of tail dependence.

In Article 26(3) the guidance states that “The dependence structure shall not be based on Gaussian or Normal-like distributions”, in this case more clarity would be welcome on what constitutes a “Normal-like” copula, in particular at what point the number of degrees of freedom of a t-copular mean the copula is “Normal-like”.

The stated limitation on the number of degrees of freedom “with few degrees of freedom (e.g. 3 or 4) in most cases appears more appropriate to capture the dependencies between operational risk events” seems particularly restrictive, and may not be appropriate in some situations.

It is recommended that more flexible guidance could state that “The dependence structure shall not be based on assumptions that rule out high level of tail dependence a-priori (e.g. by using a Gaussian copula).”
ORX will not be submitting a response to this question.
Suzanne James, Manager Quality Assurance & Standards