Forecasts: how to get them right

The traditional hand count of last week's British general election is a spectacular ancient institution, the race towards which brings a plethora of polls carried out by national research companies. Accountants and psephologists are similar tailors in a fashion, one forecasting the amount of tea likely to be drunk, the other analysing through microscopic sampling of the tea leaves, what everyone thought of the tea and whether it might become habit forming.

The tools employed by the two professions are different; the materiality of background information investigated is at polar opposites: review versus sampling. Finance flash reporting share the most elements with polling, striving to give a reasonable indication as to how a result may finally come to rest.

Accountants and psephologists share outsider misconceptions on what constitutes a fair forecast or a poll.

As field work is often compiled by junior personnel, there is sometimes a presumption that lack of experience will lead to unrealistic projections. This is mistaken as well defined processes exist. Both look to what has occurred, taking onboard current and planned structures, analysing the outlook and building a model, which is populated and a likely outcome extrapolated. Refusing to face a probable future reality says more about the individual than anything in a forecast.

Expect a voice declaring that everyone that person has spoken to disagrees with the direction of the exercise. As human beings we tend to form the most open relations with those who share a common world view, that voice is therefore likely to be a clarion for one concerned group. Being able to identify these representative people will allow you to collate your raw data more effectively and has other business uses.

By seeking to get a view of something backed up by substantive reasoning, these are mostly collaborative affairs with minimal disruption to those involved. This can create a sense of exclusion. Some might ask whether their opinion has been taken into account. In a business environment one person is usually assigned to interact with a forecast exercise and thus that function speaks with one voice. The UAE publishes regular national polls but the chances of being picked for one are tiny, representative sampling leaving you a once-in-400 years opportunity.

Engaging with a representative small sample is more likely to give you a 95 per cent accurate picture of whatever it is you are investigating. The US-printed Literary Digest held polls of their readers who responded in their millions. In the 1936 presidential campaign they called the election against the future president Roosevelt by underestimating his vote by nearly 50 per cent. In forecasting relevance not size is your guide. If your budget process consumes a forest, think small glade for your forecast.

Forecasts are often accused of being sweet music to a receptive ear. Putting together a proper forecast is a serious and exacting task. It removes resources from their daily roles, as a periodic exercise it needs to be and be seen as adding value and it costs money. There are less costly forms of wrapping ones narcissism in sycophantic ink. Decision makers crave useful information and report writers are wary of damaging their professional reputations.

Someone will inevitably lay an accusation that the whole exercise was conceived and executed in a coffee shop by a self interested cabal. Fully auditable trails must be established backing up and justifying the basis behind the final results. These, with certain safeguards, should be made available to contributing parties dampening down anyone hinting at bias.

On production of a researched report, the most senior person may ask what is your level of confidence? You are not being tested. As the person closest to the results, the user wants to gauge the reasonability of the process. Is there an additional margin of error that needs to be factored in or was testing material enough to reveal useful answers.

Inherently dangerous is utilising an old familiar tool. Forecasting models that ignore their environment and ask the wrong questions based on misplaced assumptions will get spectacularly spurious results. Review the efficacy of models after each finished report utilising them. Weaknesses in approach, straying assumptions and changes that were required to set piece financial models should be identified and corrective action taken.

I have seen reasonably accurate forecasts eventually arrive at the same conclusion. The longer the period under review or relative difference between the sample size to the whole, the less likely the outcome. Last Thursday saw UK pollsters misdiagnose the issues, party support and ultimately the final result. Bizarrely, the final exit poll was so extremely accurate that it was dismissed at the beginning of the vote count as a rogue poll, an expensive last gasp attempt to forecast the outcome.

Who can tell when the rogue result is none other than that rarest of creatures, the black swan. Nassim Taleb identifies three characteristics to aid identification.

- Left field event: It was not reasonable to expect it.

- Impact: Everyone was taken aback by the findings.

- Rationalisation: The findings are made to retrospectively fit in with contemporary reality.

Black swan events are even less likely to professionally arise than being asked to participate in a poll. You may one day face findings that defy your experience. When every explanation and other possibility have been proven false, what remains must be true.

I would rather face the titanic maelstrom of a black swan event over the nebulous fog of an inconclusive report. In the former you identify the pressing issues and (try to) deal with them, sparing a thought for how portents of the outbreak were missed. In the latter you return to the original brief to discern how such an outcome is even possible, question, refocus and relaunch the exercise.

Different professions often face the same fundamental external misunderstandings while approaching their tasks with similar frameworks. The challenge to the professional is to recognise such meetings of minds and share experiences in approach while deftly avoiding technical detail.

- The author is a senior financial consultant based in Dubai. The views expressed are his own