Is Forecasting Just Vibes?

You say that like it's a bad thing

Cats are very sensitive to vibes

When we can’t definitively predict outcomes, forecasting depends on an intuitive reading of complex situations. But while forecasting may in that sense be “just vibes” that doesn’t mean it can’t be done well.

You sometimes hear that “forecasting is just vibes.” The implication is often that forecasting is a lot of hand-waving, with no more basis in reality than astrology. It is true that there are no clear, hard-and-fast rules we can use to determine the outcomes of political and social situations, so in the absence of reliable predictive algorithms, vibes—the complex totality of circumstances—may be all we have to go on.

But the vibes aren’t always dumb. Vibes in this sense are certainly noisy and hard to read. We generally have too much confidence in our ability to interpret them accurately. We often make strong inferences on the basis of weak data. Nevertheless, we are able to make accurate judgments about situations too complex for brute analysis in our daily lives all the time. Situations in which it’s possible to definitively determine what will happen through using some neat algorithm are special cases rather than the rule. Our ability to navigate the world and to understand one another—both of which we do quite well—largely depends on a tacit ability to interpret complex sets of facts. My fellow forecaster Nathan Young notes in his substack

Predictive Text

that many of the expert opinions we rely on are essentially “made up.” That doesn’t mean they have no basis whatsoever—they may be based on long experience and informed by scientific research—but they’re still individual judgments, rather than logically necessary conclusions. They’re educated guesses, not indisputable facts.

Quantitative models help us think clearly and methodically about complex questions by decomposing them into simpler problems. Good forecasters do often use models to arrive at their forecasts. But in many cases we don’t know for sure they’ll produce accurate forecasts. The best forecasters use their expert intuitions—what Misha Yagudin calls “epistemic feelings”—to judge whether model outputs makes sense instead of just accepting them uncritically. Models that have accurately predicted outcomes in the past—like Nate Silver’s election models—still depend on unproven hypotheses about what factors are decisive; they rely on judgment, if only about the model design. Silver’s expertise entitles him to dismiss some things as insignificant to the outcome of elections, but the distinction he seems to make between rigorous modeling and loose guesswork isn’t actually that clear. The fact that models involve mathematical calculations doesn’t make them any less educated guesses.

I know a lot of forecasting is bullshit. We can and should evaluate the quality of a forecaster’s reasoning and the methods they use to arrive at their forecasts. You should read my posts critically. But what people sometimes dismiss as “vibes” often contain meaningful information skilled forecasters can use to estimate the probability of future events. Since there’s no method for definitively predicting outcomes in most cases, forecasting necessarily depends on the intuitions of the forecaster about complex situations. Some forecasters—like Nate Silver—have developed pretty good intuitions about the subjects they forecast. We can be confident their intuitions are good because they have extensive track records of accurately forecasting intractable subjects. Forecasting may just be vibes, but it can still be valuable.

My sincere thanks to everyone who renewed their paid subscription in August! Telling the Future would absolutely not be possible without your support over the last year. If you’re not yet a paid subscriber but would like to support this newsletter, you can buy a paid subscription by clicking on the button below.