If I tell you that it’s not going to rain tomorrow, and it doesn’t, then I have given you a successful forecast. If I repeat this success a few times, then you might come to think that I know what I’m talking about, that I might have a secret method that lets me look into the life of clouds. And, if the presence or absence of rain meant something to you, you might be willing to pay me a tidy fee for my prognostications.
But not if my forecasts were for Tucson, Arizona, a place where it hardly ever rains. Why would you pay for something which is (nearly) obvious?
Part of what makes a good forecast, then, incorporates the idea of difficulty. Accurate forecasts of events which are difficult to predict are more valuable than accurate predictions of events which are easy to forecast.
Now let’s suppose that we’re still in Tucson, and my forecast is instead for rain, an event difficult to predict accurately. You ask, “Is this forecast for tomorrow or the next day?” And I answer, “Oh, let’s not bother about details. Just wait and see.”
Obviously, if we wait long enough it will eventually rain in Tucson. My reforming this statement as a “prediction” is just as obviously worthless.
Therefore, another dimension of forecast quality is precision. Exactly when does this forecast hold? You might think this too trivial to make notice of, and that nobody would ever pay for a prediction with amorphous bounds, but this is not so, as we shall see.
OK, still in Tucson, and still a forecast of rain for the next day. But now you learn that I am the proud owner of Briggs’s Cataract Cloud Seeding (“Success Guaranteed!”). How much would you pay for my forecast of rain?
If I have the power to cause, or to substantially influence, the event which I am predicting, then you should not pay much. You might, of course, pay me to do the influencing or causing, but you wouldn’t pay for my forecast. This notion is made even stronger when you consider that I might have an interest in the thing that I am forecasting (and am able to influence).
To summarize: An unskillful or low-value forecast is one which predicts something easy, is vague in its details, and is for an event which the forecaster might influence. Nobody would pay for such unskillful forecasts, right?
Wrong. They often do, usually in the form of paying a trading fee for buying a stock recommended by a broker.
Consider: a broker tells you that stock in company ABC is doing well, its “financials are sound”, and so forth. He is not quite saying, but hinting strongly, that the stock price will rise. By how much and over what period of time is never specified. The details, that is, are amorphous.
The stock might be part of a sector, which might include the entire market, which is embedded in a general upswing. For example, the technology sector in the late 1990s, or the housing sector in the mid-2000s. Forecasting the price of a stock will increase in these times requires no skill (similarly, forecasting it will decrease after the bubble bursts, also requires no skill).
Worst, the company in which the broker works might not be disinterested. The broker’s prediction might be influential, but you might not be aware of the influence.
The situation is better for bonds, because of their built-in maturity. But even bonds are not entirely trouble free. Some bonds pay regular interest, others don’t. And bonds are often incorporated into complex financial “instruments” which are much like stocks in their behavior.
Firms like Moody’s issue company bond ratings such as “Aaa” and “Aa”, a bond which is said to be “high quality by all standards.” The ratings are, of course, forecasts and should be treated as such.
And they are influential forecasts, too. It’s not that Moody’s, or other bond-rating agency, necessarily has an interest in the company which issued the bond, but it’s people’s perceptions of the bond given the rating by Moody’s that is influential. That is, the company that issued the bond might do better, and thus have a higher chance of paying off the bond, given that the company garners a high rating.
Investors look at the bond rating and say to themselves, “Since this company has such a high rating, they must be doing well. I’ll invest.” And those investments, of course, can cause the company to do well, which thus justify the ratings (forecasts).
As you can imagine, it can be extraordinarily tricky to prove that the rating (forecast) was skillful in the presence of influence.
There are two problems here: how to tell if a bond or stock rating is skillful, and how influence affects skill. We’ll look at these another time.