Skip to main content

EDM & Data Sourcing (Part I)

Martijn Groot

Enterprise Data Management (EDM) has long been associated with the discipline of sourcing, mastering and distributing data that is widely used across different departments in a financial services firm. This typically includes valuation data, instrument master data and entity data. However, major changes in business and regulatory reporting requirements mean that firms have to be able to both respond to and manage real-time data requests. These requirements for increased frequency, granularity and availability of data, combined with changes to the enabling technologies, are set to fundamentally change the data sourcing aspect of a financial data management process.

Over the past few years, organizations have leveraged the increasing power of financial data management systems to bulk-load warehouse security master data on a typically daily basis.  But, while effective, such models no longer meet regulatory requirements. There is a necessary evolution amongst both data and software suppliers to move from the traditional end of day file based delivery of reference data towards a more specific model, where individual data items are sourced on demand via Application Programming Interfaces (APIs). On top of this, an increasing availability of content not available via the structured offerings from the enterprise data providers means an opportunity for data scientists to differentiate and extract new insights. This data will come with new indices and summary statistics which, when properly accessible, can be analysed and monitored for investment signals and risk management.

But what does this mean for the sourcing process?

The key to successfully leveraging real-time data is a different approach from the classic EDM model to creating the security master data source. Unlike the daily download and reconciliation, systems must now be able to respond to real-time user requests for information.

This requires the data management function to be able to capture system requests related to new client set up or instrument on-boarding in real-time; verify whether or not that request can be serviced from an existing data set, to prevent an unnecessary and costly hit on an external source; and only go out to look for the additional data when needed.

It is this ability to continuously listen to requests and then screen each one before seeking any additional external data, which will be key to efficiently and cost effectively managing this real-time data model.

While the decision by data providers to increasingly leverage APIs to enable more interactive and continuous, ad hoc individual data item requests has, in the main, been in response to regulatory change, this real-time system opens the door to a far more precise and relevant data sourcing model. With a real-time approach, organisations are able to actively source the specific data required as and when needed – which is very different to the current model of preemptively sourcing all and any data that may or may not be used and storing it within a warehouse.

The most obvious benefit, of course, is that data is refreshed in real-time, based on a trigger from the business, avoiding the risk of the data loaded into the warehouse becoming out of date. In addition, however, the data vendors are now by default moving away from the previous blunt delivery models where data was bundled up into, for example, all European Equities or all North American Corporate bonds. Now the option is to cherry pick just the specific instruments or data elements required – reducing the volume of data both purchased and stored.

In addition, there already is a vast array of new data sources including unstructured data available – with data providers currently looking at how best to monetize a mass of new information. Data management solutions able to efficiently explore (ie. master) and exploit (eg. distribute) these alternative sources – themes which we will explore in subsequent posts – will provide a new depth of insight and enable organisations to respond to ever shorter reporting cycle times and get new insights from spotting patterns across the spectrum of sources.

Ultimately the financial data model is changing as organisations face up to the opportunities provided by the broadening range of data sources and the challenges of regulatory demand for timely, efficient data to support compliance. The onus is on organisations to avoid redundant storage points of data, to find the connections between different sources through sufficiently powerful mapping and matching logic, and to avoid repurchasing existing information.

A financial data management function that can support real-time data demands enables organisations to embrace a far more conscious approach to data sourcing, purchasing only the specific data sets required but also aiding with data discovery across the new world of data sources. But this is just the start of operational enhancements; the regulatory demand for a near real-time response to servicing requests from trading and risk systems opens the door to hitherto unachievable operational benefits, from faster on-boarding processes to new instrument creation, and heralds a new dawn for unlocking data value.