Alveo Blog Data Governance

7 Data Sins: Insufficient model risk management

Stef Nielen
Stef Nielen Red Swan Risk Vlog

As a continuation of our 7 Data Sins series, Stef Nielen, Director Strategic Business Development at Alveo speaks with John Matway, CEO and founding partner at Red Swan Risk. During the discussion, Stef and John explore whether data models and data assignments are reliable enough to be trusted to navigate you through risky waters.

Q1: What are the challenges around modelling securities – why is it so challenging? I mean, when a company has just bought a risk system, doesn’t it deal with coverage out of the box?

A: Sometimes there is no suitable model or the right data might not be readily at hand (yet), which prompts one to resort to proxying. Here one wants to tread even more carefully to avoid creating additional model risk. Most generally speaking, model risk occurs when models don’t behave as they ought to. This may be due to an insufficient analytical model, misuse of the model, or plain input errors such as bad market data or incorrect terms & conditions or simply wrongfully chosen reference data such as sector classifications, ratings, etc.

Why is this so important?

Models can misbehave at the security level for long periods before showing up at the portfolio level.  Perhaps the size of the hedge was small and has grown larger, or the volatility suddenly changed.  This may suddenly create distortions at the portfolio, benchmark, or higher aggregate level. These problems often surface during times of market stress and can be very resource-intensive to troubleshoot at a critical time.

Q2: Why is it so resource-intensive to change, troubleshoot, and manage data?

A: When rules are hardcoded or implemented in an inflexible manner (i.e. model queries and scripts are being based on rigid and narrowly defined model schemas and inputs with too few degrees of freedom)  the problem is often exacerbated, making it truly difficult to interrogate and correct changes, when they are critically required.  Too often, the developer or analyst is given a set of functional requirements that are too narrowly defined, based on the current state of holdings and securities.

Given the dynamic nature of portfolio holdings, OTC instruments, available market data, and model improvements, it is essential to have a very flexible mapping process with and transparent and configurable rules that make it much easier to identify modeling issues and resolve them more efficiently.  A unified data model that tracks the data lineage of both model inputs and outputs (including risk stat, stress tests, and simulations), model choices, mapping rules, and portfolio holdings provides a highly robust and efficient framework for controlling this process. The benefit of working with a commercial tool is that it has been designed to address a very wide range of instrument types, data fields, and market data sources so you won’t outgrow its utility. So, in essence, having a unified model and data lineage capabilities combined together implies less digging and troubleshooting for the business user

Q3: Can we discuss some real-life examples perhaps?

A: Some examples are…

  1. Corporate bond credit risk derived from equity volatility using the credit grades model can cause significant distortions. A more direct method uses the observed pricing of single-name CDS prices or a sector-based credit curve. However, these must be properly assigned to the security with either the correct CDS red code or a waterfall structure for assigning the sector credit curve.  In the case of capital structure arbitrage where there are corporate bonds at various seniority and CDS, it is very important to be consistent in the mapping rules so that both the bond and the CDS have the same market data inputs.
  2. A similar issue occurs when using constant maturity commodity curves for convenience. This is easier to maintain than assigning the correct futures data set each time.  Calendar spread risk is underestimated with constant maturity curves that share data.  The negative front-month crude prices that occurred in March are an example of why constant maturity would have underestimated the risk significantly.  (I like this example because PassPort is a good solution for managing commodity future curve names in RiskMetrics).
  3. Changing over to the new Libor curves will likely be a very painful process for banks unless they have a very flexible mapping process that can easily be configured to assign the new curves to the right security types. (This is a simple procedure with the Map Editor and PassPort).
  4. But perhaps a more benign example is that of modelling one’s complete book with the right mapping for each individual security (i.e.: choosing the right risk factors as well as the correct reference data, such as ratings and sector classifications), whilst skipping to model all this stuff for its benchmark. This modelling inconsistency between portfolio and benchmark will introduce a TE-risk which can be contributed completely to inconsistent data mapping, rather than true market dynamics.

In summary, to model things properly – be it a simple proxy or something more granular and exact- one needs a setup that can dynamically configure the users ‘modelling choices and data mapping logic’. And as market conditions and data availability evolves over time, one should have a system that can adapt. Both Alveo and Red Swan allow the users to control their model and data mapping choices in a very flexible, transparent, user-friendly, and visual way. This doesn’t’ just help you during a setup or implementation phase but perhaps, more importantly, it drastically improves your ever-evolving modelling choices and (proxy) coverage over time as well as ongoing operational efficiencies. In short, it enables greater control over your model risk management.

Related insights

7 Data Sins Series: Serving Multiple Masters 7 Data Sins Series: Serving Multiple Masters Read Article
7 Data Sins Series: Achieving and keeping Data Quality from one-off to a continuous process 7 Data Sins Series: Achieving and keeping Data Quality from one-off to a continuous process Read Article
Alveo Blog Data Governance

7 Data Sins Series: Serving Multiple Masters

Robert Jan de Kraker

There are different paradoxes in data management. One is that, quite often, firms have multiple different “master” databases for their price data, their customer data and the terms and conditions of the products they invest in, trade or issue. The record we have seen is a firm that had 32 different widely used databases just to keep financial product terms and conditions. And this is not even counting a large number of small local databases and spreadsheets that also stored some of this information.

The “sin” here is clear: avoid the redundant storing of information! Having multiple places where you store information leads to the need to reconcile and cross-compare and, in general, causes uncertainty as to the validity of data points. At best, you could hope to be consistent across your different databases. But at what cost?

There have been several reasons why firms set up multiple databases with essentially the same data:

  • Decision making and budgeting across departmental lines made it easier to do something at the local level rather than establishing firm-wide services
  • Lack of sound data governance and the tracking of metadata historically made it difficult to track permissions and data usage when consolidating onto a single database or single (external) service
  • Departments often added their own custom fields to their datasets, which could be specific identifiers for homegrown applications, specific product taxonomies, industry classifications or derived data elements.
  • Departments may have wanted privileged access to a dataset or may have had performance concerns that caused them to have their own local copy.

Needless to say, departments that rely the most on aggregated, enterprise-wide information such as the risk and finance departments have suffered the most from a fragmented approach to data management and data storage causing endless rework, reconciliation and verification of data sets.

Setting up departmental level stores of data may have made some sense ten or even five years ago.

However, with today’s managed data services this is no longer needed and here’s why:

  • Managed data services have come a long way in offering concrete business user enablement and easy data access via browsers, enterprise search, easy integration into user workflows and APIs for those needing programmatic access.
  • Today’s managed data services include a comprehensive approach to tracking metadata including permissions, usage rights, quality checks performed and data lineage information – which provides a full explanation of what sources, formulas or human actions led to a certain data value.
  • New cloud-based services provide the required scalability and uptime requirements to serve different departments.
  • Providers such as Alveo via their Business Domain Models provide the capability of using a firmwide data set with different local requirements to cater to idiosyncratic needs – all in the same service.

Keeping data stored in redundant copies may have made sense at some point to prevent resource conflicts and stop applications or users from waiting for access. However, the flipside of different master databases also means redundant entry points of commercial data feeds into organizations – often leading to avoidable data spend. In our experience, teams can best be connected through shared and transparent data assets, that easily integrate into their existing workflows with the capability to augment data sets to cater to local requirements. Our PaSS managed data service does exactly that.

Related insights

7 Data Sins Series: Achieving and keeping Data Quality from one-off to a continuous process 7 Data Sins Series: Achieving and keeping Data Quality from one-off to a continuous process Read Article
Alveo Blog Data Governance

7 Data Sins Series: Achieving and keeping Data Quality from one-off to a continuous process

Boyke Baboelal

Moderator: Alexis Bisagni

Speaker: Boyke Baboelal

As a continuation of our 7 Data Sins series, Boyke Baboelal, Strategic Solutions Director in the Americas speaks with Alexis Bisagni about data quality and whether it’s a continuous fight against uncertainty. This surprise factor in data can arise from poor data quality management, and not keeping track of metadata such as changes, permissions, and quality checks.

Q: Leaning on your experience in financial data management – what have you observed with respect to data quality efforts? (Timemarker 2:00)

A: What I have observed is that there is a wide range of Data Quality maturity within organizations. Some organizations run regular data cleansing activities against their database (which requires manual effort and planning), some have triggers that check data when it is stored (but these systems are difficult to maintain and scale), and others have an Enterprise Data Management system that manages the entire data flow – but this is often still suboptimal.

Why is that? Data management teams have been downscaled in the last decade, while data volumes, types, and complexity have increased. There is a strong day-to-day focus in operations with little information where structural issues or bottlenecks are. This results in work performed in less optimal and reactive ways. In addition, organizations are under more scrutiny from regulators, requiring more controls, and from data vendors, who want to make sure entitlements are adhered to. All of this makes data management more complex. Existing EDM solutions are NOT able to meet new requirements in a cost-effective way.

Q: In your opinion, what is needed to make existing EDM solutions capable of meeting new requirements in a cost-effective way? (Timemarker 3:50)

A: Data management implementations and EDM platforms focus on automating the entire data flow end-to-end. However, simply processing data is not enough to ensure operational efficiency, transparency, and compliance. The critical component here is more information that can be used to understand what is going well and what can be improved.  Meta-data, operational metrics, usage statistics, audit trails, and data lineage information are key in taking data management to the next level.

Q: Where does an organization even start to get a grip on this? (Timemarker 5:05)

A: The first thing to do is to understand what is needed. A lot of organizations start with an inventory of what they currently have and the requirements from the driver for the change, for example, a regulatory requirement. This approach results in being less adaptable to future requirements. So how can we do better? First, it is important to have a data quality framework, including Data Governance. Starting with a Data Quality Framework forces you to look beyond your current needs, view the requirements from different angles. A framework also puts you in a mindset to continuously improve. A proper data management solution should support a data quality framework and collect all the meta-data.

Q: Do you think that buy vs build is a relevant question? (Timemarker 6:26)

A: No, in my opinion, this is not a relevant question. The reason for that is that data management is over-simplified due to a lack of understanding of data quality in a larger context. While I agree that if you need a small number of fields for your Securities from a specific vendor, every day, that would be easy to implement. Taking a moment to think through the concept, building a data management system in-house for today’s needs requires significant effort and detailed knowledge. Even with 20+ years of experience as a Financial Engineer in the Risk and Data Management space, when I think of building a system from scratch, I get anxious. The reason is that building a system in-house would involve large project risks, and the sad thing is that the system will most likely not be future-proof or benefit from the experience of peers in the industry. An adaptable off-the-shelf system will reduce a lot of that risk.

Q: When you have operational, usage, and lineage data, what comes next? (Timemarker 8:42)

A: This is when the magic starts. What I mean by that is it opens data management to the world of intelligence, analytics, and further automation. Having this information will give you more insight into your operations, what works well, and what doesn’t. The result is that you will gain more intelligence in your operations and that intelligence will enable you to comply with regulatory requirements, vendor agreements, and internal control frameworks. Having all this insight will allow your operations and data quality to get better day-by-day, resulting in continuous improvement.

Q: Continuous improvement sounds nice, but what about the bottom line? (Timemarker 10:18)

A: Increased operational efficiency, improved data quality, reduced data risks, compliance with regulatory requirements, vendor agreements, internal control frameworks, and SLAs, will in the end reduce overall TCO.

To summarize, for the financial services industry in the current environment, making the most of their data assets is not a nice to have – it is a critical must-have. Firms not only need to manage increasing volumes and diversity of data sources, they also need to keep close track of their metadata, i.e. different quality aspects that help with determining whether it is fit for purpose, optimizing sourcing and validation processes and, in general, help operational efficiency