Research seminar: Extracting the Low-Coverage Juice: Exploiting Information of Alternative data with High Missing Ratio for Financial Forecasting

In the ever-evolving landscape of financial forecasting, the emergence of machine learning has revolutionized the way we approach predicting company financials and stock prices. Alternative data sources, ranging from social media sentiment to credit card transactions, offer a wealth of untapped insights for financial forecasting. Yet, their integration poses unique challenges, including the absence of historical data, limited coverage across companies and time-variant features. Additionally, the acquisition costs associated with these datasets can be prohibitively high, further complicating model development and backtesting processes. This presents a dual challenge: how to effectively leverage these valuable features with sparse data and how to strategically prioritize data acquisition efforts with to optimize model performance. 

In our study, we delineate different scenarios mirroring varying degrees of missing values, each with customizable rates of absence. We advance by contrasting two approaches designed to glean insights from features with limited coverage: (i) model guidance, inspired by multi-task learning, and (ii) crafting new dense features through feature engineering, achieved by amalgamating observations from sparse features. Furthermore, we introduce a framework to perform online feature selection to strike a balance between exploring new features and exploiting existing ones within the acquisition process. 

Access on Teams

Alexander Shestopaloff Bio (from Queen Mary University of London) 

Alex Shestopaloff is a Senior Lecturer in Statistics at the School of Mathematical Sciences, Queen Mary University of London and a Fellow of the Alan Turing Institute. He is interested in statistical computing, in particular, Markov Chain Monte Carlo methods for performing Bayesian inference for complex stochastic models. Before joining QMUL he was a Research Fellow at The Alan Turing Institute and completed his PhD in Statistics at the University of Toronto in 2016.