One thing I’ve always liked about the forecasting community is that how it continuously calibrates itself against the goal of accuracy, which it also measures explicitly and clearly.
In other fields, you often see a trend of credentialised experts becoming obscurantist and vague when probed on the limits of their expertise. Perhaps they won’t even have a clear sense themselves of what the limits of their knowledge are. It’s rare to see the kind of meta norms that exist in forecasting, where experts are often characterised by a shrewd selfawareness and acknowledgement of what they can and can’t bring to the table.
I hope you are now doing well after your recent health issue.
Thanks, Mark! I like that about the forecasting community too. I hate to admit that I'm wrong as much as anyone else, but I find the discipline of acknowledging my mistakes really helps me avoid the temptation of overconfidence and confirmation bias.
The main weakness of simple rules is that in forecasting *competitions* you have to beat the average and not merely do well. This often means "cheating" and giving say a 93% event a 99% chance of happening, but it also sometimes means using more advanced techniques, but tbh I think the most underestimated technique is time-decay, where you say "will x happen by date Y" and you continuously move that probability downward as the date draws closer.
Near as I can tell there are roughly 10 or so basic forecasting techniques which when stated sound stupid but end up destroying the competition.
Namely
1. Read wikipedia for the base rate
2. Time decay your forecast
3. Incredible stuff rarely happens
4. The main way dictators get replaced is dying of old age
and I don't remember the rest off the top of my head
You're right that it's hard get enough of an edge to beat the average in a forecasting competition, Edmund! You can "cheat" by being overconfident—and hoping you're not unlucky—but you're also right that you can do quite well by following a few simple rules of thumb and process guidelines.
Great article Robert
One thing I’ve always liked about the forecasting community is that how it continuously calibrates itself against the goal of accuracy, which it also measures explicitly and clearly.
In other fields, you often see a trend of credentialised experts becoming obscurantist and vague when probed on the limits of their expertise. Perhaps they won’t even have a clear sense themselves of what the limits of their knowledge are. It’s rare to see the kind of meta norms that exist in forecasting, where experts are often characterised by a shrewd selfawareness and acknowledgement of what they can and can’t bring to the table.
I hope you are now doing well after your recent health issue.
Thanks, Mark! I like that about the forecasting community too. I hate to admit that I'm wrong as much as anyone else, but I find the discipline of acknowledging my mistakes really helps me avoid the temptation of overconfidence and confirmation bias.
The main weakness of simple rules is that in forecasting *competitions* you have to beat the average and not merely do well. This often means "cheating" and giving say a 93% event a 99% chance of happening, but it also sometimes means using more advanced techniques, but tbh I think the most underestimated technique is time-decay, where you say "will x happen by date Y" and you continuously move that probability downward as the date draws closer.
Near as I can tell there are roughly 10 or so basic forecasting techniques which when stated sound stupid but end up destroying the competition.
Namely
1. Read wikipedia for the base rate
2. Time decay your forecast
3. Incredible stuff rarely happens
4. The main way dictators get replaced is dying of old age
and I don't remember the rest off the top of my head
You're right that it's hard get enough of an edge to beat the average in a forecasting competition, Edmund! You can "cheat" by being overconfident—and hoping you're not unlucky—but you're also right that you can do quite well by following a few simple rules of thumb and process guidelines.