Alveo Blog Data Management

Squaring the Circle

How to resolve the data control versus data access conundrum

Data management is often said to consist of sourcing, mastering and distribution. This understanding very much casts data management as a control function: getting the required source data in, cross-referencing, validating and approving it, and then releasing it to stakeholders in a controlled fashion, often through one or more batches during the day. Data management teams responsible for delivery to support the daily cycles of settlement, valuation and reporting prefer to work in this staged and controlled way to minimize the risk of disruption to operations.

However, increasingly users downstream demand ad-hoc access to cleansed data for use cases that require data exploration and where it is harder or impossible to define parameters a-priori. Data exploration can entail executing large queries and running advanced analytics including machine learning algorithms on massive sets of time series of financial information. To address these users, typically separate data stores are constructed for e.g. quantitative research, risk modelling or scenario management. Because these data solutions are not closely linked to the data acquisition and mastering platform issues arise in timeliness, quality and mapping. Also, in larger organizations this can happen multiple times over. All this leads to a high maintenance cost on top of the operational risk linked to using various unconnected or only loosely synced local data copies – as firms increasingly need to be able to explain the data values they used in their modelling.

Existing financial services focused Enterprise Data Management solutions fall short when it comes to dynamically acquiring, preparing and provisioning data sets for data exploration while generic NoSQL and analytics solutions miss the financial services data domain experience. This has left firms between a rock and a hard place with a choice of staying with focused, but rigid solutions or spending significant resources on implementing and customizing generic solutions.

Alveo has taken a new approach. Long known for its highly scalable and comprehensive solution for the automated acquisition and mastering of pricing and reference data, it has expanded its offering to cater to quants and data scientists in risk, product control and investment decision support.

Data operations are complicated due to the proliferation of identifiers, taxonomies, data formats and standards. Alveo’s Prime data mastering solution addresses this through services integration with all main data providers, cross-referencing this to a common industry model. Alveo’s data exploration solution Alpha takes mastered data in real-time from Prime and provides a dedicated environment for modelers and data scientists. The data uses the same business domain model – bringing structure to the underlying NoSQL technology and offering easy access to a range of data providers. Data lineage provides end-to-end transparency into the data supply chain – always securing the explainability of model results and data values used. State of the art user interfaces and APIs offer easy access and integration with customer libraries in Python, R or other languages.

With our new approach addressing the need for data management for insight, we make sure users make the most of the data sets the firm has and can be assured of its quality – with data lineage to drill down into the origins, transformation and business rules data has undergone. This should help users in sourcing, mastering and distribution too.

Alveo Blog Compliance

How to reduce cost and improve Service Levels using AI and Machine Learning with Data Management Automation

By Boyke Baboelal

Process automation

Financial Data Management has focused mostly on day-to-day operations to deliver quality data to critical downstream applications. Little has been done to take advantage of AI and Machine Learning to further improve service levels and reduce TCO. However, Financial Data Management contains many use cases where new algorithms can improve operations and quality of data, ranging from new instrument set-ups using NLP, allowing to process product sheets, emails and pdf files, to data enrichment activities such as proxying incomplete data with information from related risk factors, found using recommender systems built using explainable techniques. Using Data Management Automation operational efficiency is improved and risks are reduced with the elimination of manual activities, data quality is improved using algorithms that process more data in a better way, and turn-around time is reduced, reducing risks of not meeting SLAs.

Exception handling and anomaly detection

One of the key controls in Financial Data Management is the detection of suspect data and validation thereof. Validation can be a very time-consuming activity depending on the universe of instruments, but also on the performance of rules, i.e. the number of false positives that are generated. The use of Machine Learning allows for advanced rules that compare more related information with each other and therefore reducing the number of false positives, for example using similar instruments to confirm price movements or verify descriptive values, but also to check frequency of updates, combinations of values, and/or completeness of data. Data Management Automation contains building blocks for organizations to build efficient data checks using Machine Learning.

Advanced controls and monitoring

In the data flow, from acquiring source data to distribution of instruments, there are many data risks that can impact the quality of delivered data. As stated before, Exception Handling is an important control, but there are many more. For example, controls should monitor if all instruments have rules applied to them and if these rules are consistently applied? Data Management Automation and Data Quality Intelligence provide frameworks and building blocks for organizations to methodologically identify data risks, create controls, and monitor risks through KRIs.

Compliance checking

Data management processes require many manual activities and these activities should be performed according to documented procedures, for example to allow for transparency, lineage, and further analysis. Do resources follow procedures and show sufficient diligence? Are changes clearly documented? Do all instruments have the proper ownership and security settings? Anomaly detection, NLP, and automated process discovery can help organization check and improve compliance to internal guidelines.

In short, data management capabilities can be significantly expanded using Machine Learning. ML techniques can reduce workload and cost by automating manual processes, improve data quality with better exception handling, comply with internal and external guidelines with advanced analytics, and reduce risks with advanced controls.

Links from this page

Free download

Complex Data Risks with the Adoption of AI

Boyke Baboelal, Strategic Solutions Director Americas at Alveo shares his insights in the January edition of PRMIA’s Intelligent Risk