As I have previously commented, “the moment we attempt to predict the future, all semblance of certainty is lost.”
Typically predictions of weather and political results can be significantly incorrect.
And even the use of sophisticated data analytics does not guarantee accuracy.
Obviously, financial models are predictive tools and based on years of modelling experience, combined with the application of international best practice methodology, and using relevant research and investigations, we attempt to produce reliable models with maximum dependability.
As we are all aware, AI is making significant inroads into all aspects of our lives and education, and on a whim I posed the following question to ChatGPT.
“What factors determine the reliability of predictions.”
Seconds later the following response was generated.
There are several factors that determine the reliability of predictions, including:
- Data quality and quantity: The reliability of predictions is directly linked to the quality and quantity of the data used to make them. The more accurate and complete the data, the more reliable the predictions are likely to be.
- Statistical methods and models: The selection of appropriate statistical methods and models is crucial for reliable predictions. Different methods and models have different levels of accuracy and reliability, and the choice of the best one to use depends on the specific context.
- Time horizon: The reliability of predictions decreases as the time horizon increases. This is because more uncertainties arise over longer periods, making it harder to predict outcomes accurately.
- Expertise and experience: The expertise and experience of the people making the predictions can also impact its reliability. Domain experts and experienced analysts are more likely to make better predictions than novices.
- External factors: External factors, such as changes in market conditions, regulatory environment, or technology, can also affect the reliability of predictions. It’s important to consider these factors when making predictions and adjust accordingly.
I suppose vindication of one’s thoughts and beliefs brings about a degree of satisfaction – and that is how I felt when ChatGPT delivered its results.
However significantly more important than a pat on my back, was the vindication of a core principle of predictive financial models – i.e. the critical importance of the various input assumptions used to drive the model.
As any modeller with some experience will confirm, the results produced by a model are entirely dependent on the predetermined input assumptions.
Clearly the assumptions relating to revenue, costs, overheads, capex, funding, interest rates, inflation and tax vary from project to project -i.e. they are project specific.
And, as such, need to be carefully determined.
Of course, if the project relates to the creation of some entirely new product or service – not preciously in existence – the challenge is much more complex and may require the modeller to use probabilities and Monte Carlo simulations.
Whilst I respect, support and practice all the technical recommendations pertaining to model building such as;
- flexibility,
- suitability to meet the purpose (applicability)
- flow,
- logical and sequential structure,
- the use of support schedules
- the method used to balance the balance sheet, which provides confirmation of the correct application of accounting principles, and
- the need for transparency and ease of understanding
It is quite clear that if the fundamental model drivers are incorrect, everything that follows will be incorrect to a lesser or larger degree.
Whilst this is true, the standard practice of sensitivity analysis of course determines the relative importance of the various input assumptions – so all errors are not equal.
Based on years of building practical real-world financial models, my conclusion is that the modeller needs to use all information available, both historical and current, prior experience, domain knowledge and what ever research is necessary strive for maximum accuracy.