Machine Learning: Making Sense of Unstructured Data and Automation in Alt Investments – Traders Magazine
The following was written byHarald Collet, CEO at Alkymi andHugues Chabanis, Product Portfolio Manager,Alternative Investments at SimCorp
Institutional investors are buckling under the operational constraint of processing hundreds of data streams from unstructured data sources such as email, PDF documents, and spreadsheets. These data formats bury employees in low-value copy-paste workflows andblockfirms from capturing valuable data. Here, we explore how Machine Learning(ML)paired with a better operational workflow, can enable firms to more quickly extract insights for informed decision-making, and help governthe value of data.
According to McKinsey, the average professional spends 28% of the workday reading and answering an average of 120 emails on top ofthe19% spent on searching and processing data.The issue is even more pronouncedininformation-intensive industries such as financial services,asvaluable employees are also required to spendneedlesshoursevery dayprocessing and synthesizing unstructured data. Transformational change, however,is finally on the horizon. Gartner research estimates thatby 2022, one in five workers engaged in mostly non-routine tasks will rely on artificial intelligence (AI) to do their jobs. And embracing ML will be a necessity for digital transformation demanded both by the market and the changing expectations of the workforce.
For institutional investors that are operating in an environment of ongoing volatility, tighter competition, and economic uncertainty, using ML to transform operations and back-office processes offers a unique opportunity. In fact, institutional investors can capture up to 15-30% efficiency gains by applying ML and intelligent process automation (Boston Consulting Group, 2019)inoperations,which in turn creates operational alpha withimproved customer service and redesigning agile processes front-to-back.
Operationalizingmachine learningworkflows
ML has finally reached the point of maturity where it can deliver on these promises. In fact, AI has flourished for decades, but the deep learning breakthroughs of the last decade has played a major role in the current AI boom. When it comes to understanding and processing unstructured data, deep learning solutions provide much higher levels of potential automation than traditional machine learning or rule-based solutions. Rapid advances in open source ML frameworks and tools including natural language processing (NLP) and computer vision have made ML solutions more widely available for data extraction.
Asset class deep-dive: Machine learning applied toAlternative investments
In a 2019 industry survey conducted byInvestOps, data collection (46%) and efficient processing of unstructured data (41%) were cited as the top two challenges European investment firms faced when supportingAlternatives.
This is no surprise as Alternatives assets present an acute data management challenge and are costly, difficult, and complex to manage, largely due to the unstructured nature ofAlternatives data. This data is typically received by investment managers in the form of email with a variety of PDF documents or Excel templates that require significant operational effort and human understanding to interpret, capture,and utilize. For example, transaction data istypicallyreceived by investment managers as a PDF document via email oran online portal. In order to make use of this mission critical data, the investment firm has to manually retrieve, interpret, and process documents in a multi-level workflow involving 3-5 employees on average.
The exceptionally low straight-through-processing (STP) rates already suffered by investment managers working with alternative investments is a problem that will further deteriorate asAlternatives investments become an increasingly important asset class,predictedbyPrequinto rise to $14 trillion AUM by 2023 from $10 trillion today.
Specific challenges faced by investment managers dealing with manual Alternatives workflows are:
WithintheAlternatives industry, variousattempts have been madeto use templatesorstandardize the exchange ofdata. However,these attempts have so far failed,or are progressing very slowly.
Applying ML to process the unstructured data will enable workflow automation and real-time insights for institutional investment managers today, without needing to wait for a wholesale industry adoption of a standardized document type like the ILPA template.
To date, the lack of straight-through-processing (STP) in Alternatives has either resulted in investment firms putting in significant operational effort to build out an internal data processing function,or reluctantly going down the path of adopting an outsourcing workaround.
However, applyinga digital approach,more specificallyML, to workflows in the front, middle and back office can drive a number of improved outcomes for investment managers, including:
Trust and control are critical when automating critical data processingworkflows.This is achieved witha human-in-the-loopdesign that puts the employee squarely in the drivers seat with features such as confidence scoring thresholds, randomized sampling of the output, and second-line verification of all STP data extractions. Validation rules on every data element can ensure that high quality output data is generated and normalized to a specific data taxonomy, making data immediately available for action. In addition, processing documents with computer vision can allow all extracted data to be traced to the exact source location in the document (such as a footnote in a long quarterly report).
Reverse outsourcing to govern the value of your data
Big data is often considered the new oil or super power, and there are, of course, many third-party service providers standing at the ready, offering to help institutional investors extract and organize the ever-increasing amount of unstructured, big data which is not easily accessible, either because of the format (emails, PDFs, etc.) or location (web traffic, satellite images, etc.). To overcome this, some turn to outsourcing, but while this removes the heavy manual burden of data processing for investment firms, it generates other challenges, including governance and lack of control.
Embracing ML and unleashing its potential
Investment managers should think of ML as an in-house co-pilot that can help its employees in various ways: First, it is fast, documents are processed instantly and when confidence levels are high, processed data only requires minimum review. Second, ML is used as an initial set of eyes, to initiate proper workflows based on documents that have been received. Third, instead of just collecting the minimum data required, ML can collect everything, providing users with options to further gather and reconcile data, that may have been ignored and lost due to a lack of resources. Finally, ML will not forget the format of any historical document from yesterday or 10 years ago safeguarding institutional knowledge that is commonly lost during cyclical employee turnover.
ML has reached the maturity where it can be applied to automate narrow and well-defined cognitive tasks and can help transform how employees workin financial services. However many early adopters have paid a price for focusing too much on the ML technology and not enough on the end-to-end business process and workflow.
The critical gap has been in planning for how to operationalize ML for specific workflows. ML solutions should be designed collaboratively with business owners and target narrow and well-defined use cases that can successfully be put into production.
Alternatives assets are costly, difficult, and complex to manage, largely due to the unstructured nature of Alternatives data. Processing unstructured data with ML is a use case that generates high levels of STP through the automation of manual data extraction and data processing tasks in operations.
Using ML to automatically process unstructured data for institutional investors will generate operational alpha; a level of automation necessary to make data-driven decisions, reduce costs, and become more agile.
The views represented in this commentary are those of its author and do not reflect the opinion of Traders Magazine, Markets Media Group or its staff. Traders Magazine welcomes reader feedback on this column and on all issues relevant to the institutional trading community.
Follow this link:
Machine Learning: Making Sense of Unstructured Data and Automation in Alt Investments - Traders Magazine
- Darktrace enhances Cyber AI Analyst with advanced machine learning for improved threat investigations - Industrial Cyber - April 21st, 2025 [April 21st, 2025]
- Infrared spectroscopy with machine learning detects early wood coating deterioration - Phys.org - April 21st, 2025 [April 21st, 2025]
- A simulation-driven computational framework for adaptive energy-efficient optimization in machine learning-based intrusion detection systems - Nature - April 21st, 2025 [April 21st, 2025]
- Machine learning model to predict the fitness of AAV capsids for gene therapy - EurekAlert! - April 21st, 2025 [April 21st, 2025]
- An integrated approach of feature selection and machine learning for early detection of breast cancer - Nature - April 21st, 2025 [April 21st, 2025]
- Predicting cerebral infarction and transient ischemic attack in healthy individuals and those with dysmetabolism: a machine learning approach combined... - April 21st, 2025 [April 21st, 2025]
- Autolomous CEO Discusses AI and Machine Learning Applications in Pharmaceutical Development and Manufacturing with Pharmaceutical Technology -... - April 21st, 2025 [April 21st, 2025]
- Machine Learning Interpretation of Optical Spectroscopy Using Peak-Sensitive Logistic Regression - ACS Publications - April 21st, 2025 [April 21st, 2025]
- Estimated glucose disposal rate outperforms other insulin resistance surrogates in predicting incident cardiovascular diseases in... - April 21st, 2025 [April 21st, 2025]
- Machine learning-based differentiation of schizophrenia and bipolar disorder using multiscale fuzzy entropy and relative power from resting-state EEG... - April 12th, 2025 [April 12th, 2025]
- Increasing load factor in logistics and evaluating shipment performance with machine learning methods: A case from the automotive industry - Nature - April 12th, 2025 [April 12th, 2025]
- Machine learning-based prediction of the thermal conductivity of filling material incorporating steelmaking slag in a ground heat exchanger system -... - April 12th, 2025 [April 12th, 2025]
- Do LLMs Know Internally When They Follow Instructions? - Apple Machine Learning Research - April 12th, 2025 [April 12th, 2025]
- Leveraging machine learning in precision medicine to unveil organochlorine pesticides as predictive biomarkers for thyroid dysfunction - Nature - April 12th, 2025 [April 12th, 2025]
- Analysis and validation of hub genes for atherosclerosis and AIDS and immune infiltration characteristics based on bioinformatics and machine learning... - April 12th, 2025 [April 12th, 2025]
- AI and Machine Learning - Bentley and Google partner to improve asset analytics - Smart Cities World - April 12th, 2025 [April 12th, 2025]
- Where to find the next Earth: Machine learning accelerates the search for habitable planets - Phys.org - April 10th, 2025 [April 10th, 2025]
- Concurrent spin squeezing and field tracking with machine learning - Nature - April 10th, 2025 [April 10th, 2025]
- This AI Paper Introduces a Machine Learning Framework to Estimate the Inference Budget for Self-Consistency and GenRMs (Generative Reward Models) -... - April 10th, 2025 [April 10th, 2025]
- UCI researchers study use of machine learning to improve stroke diagnosis, access to timely treatment - UCI Health - April 10th, 2025 [April 10th, 2025]
- Assessing dengue forecasting methods: a comparative study of statistical models and machine learning techniques in Rio de Janeiro, Brazil - Tropical... - April 10th, 2025 [April 10th, 2025]
- Machine learning integration of multimodal data identifies key features of circulating NT-proBNP in people without cardiovascular diseases - Nature - April 10th, 2025 [April 10th, 2025]
- How AI, Data Science, And Machine Learning Are Shaping The Future - Forbes - April 10th, 2025 [April 10th, 2025]
- Development and validation of interpretable machine learning models to predict distant metastasis and prognosis of muscle-invasive bladder cancer... - April 10th, 2025 [April 10th, 2025]
- From fax machines to machine learning: The fight for efficiency - HME News - April 10th, 2025 [April 10th, 2025]
- Carbon market and emission reduction: evidence from evolutionary game and machine learning - Nature - April 10th, 2025 [April 10th, 2025]
- Infleqtion Unveils Contextual Machine Learning (CML) at GTC 2025, Powering AI Breakthroughs with NVIDIA CUDA-Q and Quantum-Inspired Algorithms - Yahoo... - March 22nd, 2025 [March 22nd, 2025]
- Karlie Kloss' coding nonprofit offering free AI and machine learning workshop this weekend - KSDK.com - March 22nd, 2025 [March 22nd, 2025]
- Machine learning reveals distinct neuroanatomical signatures of cardiovascular and metabolic diseases in cognitively unimpaired individuals -... - March 22nd, 2025 [March 22nd, 2025]
- Machine learning analysis of cardiovascular risk factors and their associations with hearing loss - Nature.com - March 22nd, 2025 [March 22nd, 2025]
- Weekly Recap: Dual-Cure Inks, AI And Machine Learning Top This Weeks Stories - Ink World Magazine - March 22nd, 2025 [March 22nd, 2025]
- Network-based predictive models for artificial intelligence: an interpretable application of machine learning techniques in the assessment of... - March 22nd, 2025 [March 22nd, 2025]
- Machine learning aids in detection of 'brain tsunamis' - University of Cincinnati - March 22nd, 2025 [March 22nd, 2025]
- AI & Machine Learning in Database Management: Studying Trends and Applications with Nithin Gadicharla - Tech Times - March 22nd, 2025 [March 22nd, 2025]
- MicroRNA Biomarkers and Machine Learning for Hypertension Subtyping - Physician's Weekly - March 22nd, 2025 [March 22nd, 2025]
- Machine Learning Pioneer Ramin Hasani Joins Info-Tech's "Digital Disruption" Podcast to Explore the Future of AI and Liquid Neural Networks... - March 22nd, 2025 [March 22nd, 2025]
- Predicting HIV treatment nonadherence in adolescents with machine learning - News-Medical.Net - March 22nd, 2025 [March 22nd, 2025]
- AI And Machine Learning In Ink And Coatings Formulation - Ink World Magazine - March 22nd, 2025 [March 22nd, 2025]
- Counting whales by eavesdropping on their chatter, with help from machine learning - Mongabay.com - March 22nd, 2025 [March 22nd, 2025]
- Associate Professor - Artificial Intelligence and Machine Learning job with GALGOTIAS UNIVERSITY | 390348 - Times Higher Education - March 22nd, 2025 [March 22nd, 2025]
- Innovative Machine Learning Tool Reveals Secrets Of Marine Microbial Proteins - Evrim Aac - March 22nd, 2025 [March 22nd, 2025]
- Exploring the role of breastfeeding, antibiotics, and indoor environments in preschool children atopic dermatitis through machine learning and hygiene... - March 22nd, 2025 [March 22nd, 2025]
- Applying machine learning algorithms to explore the impact of combined noise and dust on hearing loss in occupationally exposed populations -... - March 22nd, 2025 [March 22nd, 2025]
- 'We want them to be the creators': Karlie Kloss' coding nonprofit offering free AI and machine learning workshop this weekend - KSDK.com - March 22nd, 2025 [March 22nd, 2025]
- New headset reads minds and uses AR, AI and machine learning to help people with locked-in-syndrome communicate with loved ones again - PC Gamer - March 22nd, 2025 [March 22nd, 2025]
- Enhancing cybersecurity through script development using machine and deep learning for advanced threat mitigation - Nature.com - March 11th, 2025 [March 11th, 2025]
- Machine learning-assisted wearable sensing systems for speech recognition and interaction - Nature.com - March 11th, 2025 [March 11th, 2025]
- Machine learning uncovers complexity of immunotherapy variables in bladder cancer - Hospital Healthcare - March 11th, 2025 [March 11th, 2025]
- Machine-learning algorithm analyzes gravitational waves from merging neutron stars in the blink of an eye - The University of Rhode Island - March 11th, 2025 [March 11th, 2025]
- Precision soil sampling strategy for the delineation of management zones in olive cultivation using unsupervised machine learning methods - Nature.com - March 11th, 2025 [March 11th, 2025]
- AI in Esports: How Machine Learning is Transforming Anti-Cheat Systems in Esports - Jumpstart Media - March 11th, 2025 [March 11th, 2025]
- Whats that microplastic? Advances in machine learning are making identifying plastics in the environment more reliable - The Conversation Indonesia - March 11th, 2025 [March 11th, 2025]
- Application of machine learning techniques in GlaucomAI system for glaucoma diagnosis and collaborative research support - Nature.com - March 11th, 2025 [March 11th, 2025]
- Elucidating the role of KCTD10 in coronary atherosclerosis: Harnessing bioinformatics and machine learning to advance understanding - Nature.com - March 11th, 2025 [March 11th, 2025]
- Hugging Face Tutorial: Unleashing the Power of AI and Machine Learning - - March 11th, 2025 [March 11th, 2025]
- Utilizing Machine Learning to Predict Host Stars and the Key Elemental Abundances of Small Planets - Astrobiology News - March 11th, 2025 [March 11th, 2025]
- AI to the rescue: Study shows machine learning predicts long term recovery for anxiety with 72% accuracy - Hindustan Times - March 11th, 2025 [March 11th, 2025]
- New in 2025.3: Reducing false positives with Machine Learning - Emsisoft - March 5th, 2025 [March 5th, 2025]
- Abnormal FX Returns And Liquidity-Based Machine Learning Approaches - Seeking Alpha - March 5th, 2025 [March 5th, 2025]
- Sentiment analysis of emoji fused reviews using machine learning and Bert - Nature.com - March 5th, 2025 [March 5th, 2025]
- Detection of obstetric anal sphincter injuries using machine learning-assisted impedance spectroscopy: a prospective, comparative, multicentre... - March 5th, 2025 [March 5th, 2025]
- JFrog and Hugging Face team to improve machine learning security and transparency for developers - SDxCentral - March 5th, 2025 [March 5th, 2025]
- Opportunistic access control scheme for enhancing IoT-enabled healthcare security using blockchain and machine learning - Nature.com - March 5th, 2025 [March 5th, 2025]
- AI and Machine Learning Operationalization Software Market Hits New High | Major Giants Google, IBM, Microsoft - openPR - March 5th, 2025 [March 5th, 2025]
- FICO secures new patents in AI and machine learning technologies - Investing.com - March 5th, 2025 [March 5th, 2025]
- Study on landslide hazard risk in Wenzhou based on slope units and machine learning approaches - Nature.com - March 5th, 2025 [March 5th, 2025]
- NVIDIA Is Finding Great Success With Vulkan Machine Learning - Competitive With CUDA - Phoronix - March 3rd, 2025 [March 3rd, 2025]
- MRI radiomics based on machine learning in high-grade gliomas as a promising tool for prediction of CD44 expression and overall survival - Nature.com - March 3rd, 2025 [March 3rd, 2025]
- AI and Machine Learning - Identifying meaningful use cases to fulfil the promise of AI in cities - SmartCitiesWorld - March 3rd, 2025 [March 3rd, 2025]
- Prediction of contrast-associated acute kidney injury with machine-learning in patients undergoing contrast-enhanced computed tomography in emergency... - March 3rd, 2025 [March 3rd, 2025]
- Predicting Ag Harvest using ArcGIS and Machine Learning - Esri - March 1st, 2025 [March 1st, 2025]
- Seeing Through The Hype: The Difference Between AI And Machine Learning In Marketing - AdExchanger - March 1st, 2025 [March 1st, 2025]
- Machine Learning Meets War Termination: Using AI to Explore Peace Scenarios in Ukraine - Center for Strategic & International Studies - March 1st, 2025 [March 1st, 2025]
- Statistical and machine learning analysis of diesel engines fueled with Moringa oleifera biodiesel doped with 1-hexanol and Zr2O3 nanoparticles |... - March 1st, 2025 [March 1st, 2025]
- Spatial analysis of air pollutant exposure and its association with metabolic diseases using machine learning - BMC Public Health - March 1st, 2025 [March 1st, 2025]
- The Evolution of AI in Software Testing: From Machine Learning to Agentic AI - CSRwire.com - March 1st, 2025 [March 1st, 2025]
- Wonder Dynamics Helps Boxel Studio Embrace Machine Learning and AI - Animation World Network - March 1st, 2025 [March 1st, 2025]
- Predicting responsiveness to fixed-dose methylene blue in adult patients with septic shock using interpretable machine learning: a retrospective study... - March 1st, 2025 [March 1st, 2025]
- Workplace Predictions: AI, Machine Learning To Transform Operations In 2025 - Facility Executive Magazine - March 1st, 2025 [March 1st, 2025]
- Development and validation of a machine learning approach for screening new leprosy cases based on the leprosy suspicion questionnaire - Nature.com - March 1st, 2025 [March 1st, 2025]