Machine Learning in forecasting: critical investment or running before we can walk?

The COVID-19 pandemic has proven that planning and predicting future outcomes has become increasingly difficult.

Yet in the face of such uncertainty, the Finance function’s forecasts of the future became more in-demand and important than ever, as executives scrambled to understand and react to the likely impact of the pandemic on the bottom-line.

Unsurprisingly, many Finance functions are thus reconsidering how they improve the speed and accuracy of their forecasting process. An avenue being explored with increasing frequency is the introduction of machine learning. Done right, it offers significant potential benefits:

1. Speed -Latest research suggests machine learning can drive a tenfold increase in forecasting speed for just 10% of the cost versus existing processes.
2. Accuracy – Estimates suggest machine learning can significantly increase forecast accuracy, with an average accuracy of around 90% for a 30-day horizon, and 96% for a 7-day horizon forecast.
3. More data, more trends, more quickly – Compared to spreadsheets or simplistic modelling tools, machine learning can incorporate more data, from more sources, in more formats, more quickly. In fact, research suggests ML technologies can identify trends in multi-variate analyses 100 times faster than human data scientists. This enables more sophisticated multivariate time series forecasts vs. the status quo, further bolstering speed and accuracy.
4. In-built continuous improvement – Machine learning focuses on the use of statistical methods, training data and in-built feedback loops to imitate the way that humans learn, leading to further improvements in accuracy over time.
5. Relocation of effort to value-add tasks – If a machine learning tool executes more of the forecast, FP&A teams can direct their effort to more value-adding activities like business partnering and decision support.

However, as with the introduction of any technology, there are several key considerations for Finance functions before taking the plunge in earnest:

1. Ensuring underlying business forecasting processes are sound – Organisations must ensure that their underlying forecasting process and operating model are sound. Machine learning will be ineffective if it is being implemented in the middle of a broken process. Investing in fundamental busines process improvement upfront may deliver more benefit in the short-term and increase the benefits of any future ML implementation.
2. Unfit IT Architecture – Data churning capabilities are an essential prerequisite for successful machine learning deployment. Legacy systems will often get stretched, are incapable of handling the workload, and fail under pressure. Organisations must check if their infrastructure can handle machine learning in the first place, and address any deficiencies with IT architecture upgrades, flexible storage, and hardware acceleration.
3. Availability of sufficient historical data to train algorithms – Machine learning is fundamentally dependent on sufficient historical data volumes of multiple different desired types to ‘train’ the algorithms. A recent study showed that over 50% of organisations do not have enough data to deploy machine learning effectively. Where firms do not have the necessary quantum or types of data in house, they will need to invest either in enhanced in-house data gathering or buy data in from external vendors.
4. Availability of knowledge and expertise in the organisation – Harnessing the full potential of machine learning requires expertise in specialist ML fields such as neural networks, natural language processing, distributed computing, prototyping and data modelling. There is a skills deficit in most Finance functions in these areas today, leaving organisations either to ‘buy in’ expertise in the short-term, or develop expertise in-house over a longer timeframe. Whatever the solution, long-term in-house expertise is crucial to maintain the ML solution post-implementation.

So, machine learning has the potential to significantly increase the speed and accuracy of financial forecasts, based on a larger range of predictive data and with some in-built future-proofing: but only if organisations put the fundamental pre-requisites in place first. Without them, introducing machine learning may simply add further complexity to what is far too often an already arduous process.

However, organisations without these pre-requisites in place shouldn’t view this as an excuse to delay. Given the vast potential benefits machine learning offers, it’s imperative organisations that aren’t machine learning ready begin their preparation journey now or risk being left behind, particularly in the race for a limited pool of machine-learning talent. Conversely, organisations that are ready should act quickly to secure and maximise their competitive advantage.

In our view, machine learning in forecasting is coming. So learn to walk before you can run; but the race is coming. Don’t risk being left in the starting gate.