“Some of them wanted to sell me snake oil and I’m not necessarily going to dismiss all of these, as I have never found a rusty snake.”
-Author Terry Pratchett on some of the 60,000 emails that he received after announcing he had been diagnosed with early onset Alzheimer’s disease
(You can read his full speech on the matter here at http://www.freerepublic.com/focus/f-chat/1986843/posts . It is well worth the read for its humour and its candour alone. If you have not read his books I would also recommend that you start)
What we tell people consistently when we work with them is that “How you think is more important than what you think” and there are two simple tests you can use straight away:
The First Test – Are they more right about the future than the average on a consistent basis
If I demonstrated to you over 10 weeks that I could predict which way the stock market would go each week would you then give me $10,000 to advise you for the next week so you could take out an option on the share market index result at the end of the month?
If the answer is yes then you had just fallen for one of the classic scams which is possible on the internet. This is how works:
- On day one you send out 10 million emails stating that you can predict which way the stock market will go in the following seven days.
- In half of those emails you tell the people that the stock market will go up and the other half you tell them that the stock market will go down.
- The following week you send 5 million e-mails to the people that received the previous correct “prediction” and in half of those e-mails you tell the people that the stock market will go up and the other half you tell them that the stock market will go down.
- In the third week you send out 2.5 million emails to the people that received the previous two correct “prediction” and in half of those e-mails you tell the people that the stock market will go up and the other half you tell them that the stock market will go down.
And so on ……
At the end all the 10 weeks you will have 9,766 people who have received a correct prediction from you 10 weeks in a row. If 10% of those send you $10,000 you have just netted a cool $9,766,000. Not bad for ten weeks work pressing an email button once a week!
The point of this story is that many people have been propelled into the spotlight as “knowing what the future will be” purely by chance.
During 2008 as oil prices surged the media were full of stories about the analysts who got the oil price forecast right:
The price of crude oil could soar to $200 a barrel in as little as six months, as supply continues to struggle to meet demand, a report has warned………….. Mr Murti correctly predicted three years ago – when oil was about $55 a barrel – that it would pass $100, which it reached for the first time in January of this year.
- BBC News May 7, 2008
Maverick economist Jeff Rubin, who is at the top of his game these days, says Canadians shouldn’t be surprised to see gasoline at $1.50 a litre and oil at $150 (U.S.) a barrel within the next four years – possibly much sooner……. He correctly predicted in 2000 that oil would average $50 by mid-decade and, two years ago, was right when he said oil would hit $100 by the end of 2007 (though Goldman Sachs made the prediction a year earlier).
- The Star January 11 2008
……the investment guru Jim Rogers the latest to predict that the only direction is upward. Mr Rogers, who two years ago correctly predicted oil would reach $US100 a barrel, told investors at a conference in China over the weekend that he expected prices to go even higher than their present level. They will keep rising, “unless someone finds a major oilfield very quickly, in accessible areas”, he said.
- Sydney Morning Herald 30th June, 2008
Now I wish to make it clear that I am not criticising the individuals who are quoted in these articles. but expecting that one successful prediction (if in fact they really are that) in a volatile market makes you an expert about what the future will bring is clearly ridiculous. If I wrote on 27 pieces of paper a price of oil between $40 a barrel and $300 dollars a barrel and stuck them on a wall and then asked monkeys to throw darts at them then one of the monkeys will almost certainly correctly predict the price of oil in a years time. I would be laughed off the stage if I then tried to sell that monkey as an expert on oil prices and rightly so.
 http://www.thestar.com/Business/article/293042 . Accessed July 8, 2008. The article goes on to say: “I think my calls have been pretty good,” said Rubin, who first grabbed the spotlight in 1989 when he went against the grain and correctly predicted a collapse in theToronto real estate market. But sometimes his calls have fallen flat. In 1992, Rubin forecast an economic recovery from the 1990-’91 recession and for several years nothing much happened. And in 1995 he called for more aggressive cuts to the federal deficit, only to backtrack a year later and blame the deficit payoff for sluggish economic growth. More recently, he predicted the S&P/TSX composite index would hit 15,000 by the end of 2007, but reality came nowhere close. And now he is locked into a prediction it will reach 16,200 by the end of this year at a time when the economy is slowing.
 http://business.smh.com.au/no-relief-as-opec-tips-us170-oil-20080629-2yup.html Accessed July 8, 2008
The Second Test (and most important one) – Can they tell you a coherent logical story about why they have made their predictions
This test is important for two reasons:
Firstly it allows you to gauge for your self the depth of understanding and thinking that has gone into the forecast that is being presented so you can gauge how well the person who presented it thinks.
Secondly if you are told a logical and coherent story you can elicit from that story a number of important things:
- What are the assumptions that the author of the story has relied upon (you should be able to tell this even if they can’t themselves). These assumptions can take a number of forms including the author’s view of how the world works (world view).
- What are the key trends that the author has looked at as the basis for their forecast? You have to be careful here because there is no such thing as a forecast trend, such a term is an oxymoron because trends are by definition recognisable patterns that are based on historical data.
- What assumptions has the author made by extrapolating trends forwards.
- What are the key forces at work in the forecast that has been described, both as individual players but other forces such as group behaviour, political actions, etc
Once you have that information then you are now in a much stronger position to assess the worth of the forecast but you can also now use the forecast to navigate as you go along. For example if you want to sit down every three months and re-examine the forces and assumptions and change any that you see changing in the real world you can alter the forecast yourself. You now have a tool, not a stone tablet handed down from on high.
Following on from the oil forecasting stories mentioned above I had a story published in The Age in August 2008 saying that I was willing to bet on the price of oil being below US$40 a barrel before then end of 2009. (You can access the article at our articles download page) I want to emphasise that this was a bet rather than a prediction. What that means is that I believed there was a better than even chance that this price would be reached in the time frame. At the time I was subject to a torrent of abuse and criticism from all over the world. I offered this bet at a number of conferences that I spoke at during this period and made a number of bets. Fellow futurist Mike McCallum was the only one that paid me but others made donations to charities at my request. I do not hold myself out as an oil expert but you can judge for yourself if I met my own criteria.
 For a useful description of how to surface explicit and implicit assumption in stories and strategies go to the following page on Wikipedia: http://en.wikipedia.org/wiki/Assumption_based_planning