How to predict the future

In an incredibly chaotic world, wouldn’t it be nice to have your own crystal ball? The next best thing might be to learn the protocol of the Superpredictors

“How many countries were there in the UN in 1980?” — take a guess before you keep reading…

After 9/11, the US government realized the world was becoming harder to anticipate. To prevent being caught behind the eightball unexpectedly again, it did a reset to try and fill the information gap.

As a part of those efforts, they created an international competition open to all comers to forecast the future on 100 issues a year from terrorism to disease outbreak. One team emerged heads and shoulders above the rest. It’s called the Good Judgement Project, and it kicked off a revolution in forecasting. The GJP started with thousands of volunteer forecasters. They then downselected to hundreds of superforecasters who batted way above average.

Michael Story was one of those superforecasters and I was lucky enough to attend a talk he gave on the topic recently. “The best way to get people to think better is to get a bunch of knowledgeable people together and follow a program.”

Selection is important: who are you going to listen to? The best forecasters are great at pattern recognition. They also tend to have high openness as a personality trait, but you can’t just select for openness to look for superforecasters because as it turns out, sometimes high-openness people just believe in crazy stuff.

Once you get a bunch of superforecasters you’ll want to work in teams to get a better answer. “People are afraid of working by committee and groupthink. But even with all of those issues, teams are still better.”

Once you have your team of superforecasters, there’s a five-step process that’s like a checklist to help your team cover all their bases. “Imagine a royal wedding. We want to know how long the couple would stay together. How might we predict that?”

1. Base rate/outside view — It’s tempting to go straight into what’s specific about this situation. But the best way to start is by going as wide as possible. In this case, how old are they? Do they have money problems? Let’s say the average marriage for young people without money problems is 15 years.

2. Inside view, breaking the problem down — now we go into specifics. “People start narrow, but you should start wide and then go narrow.” And you go up and down from 15 years with this information.

3. Scope and scale — would you be super surprised if they lasted 25 years? Or just one year? “forcing yourself to think about different scales is a really good way to test your assumptions.”

4. Relevant sources — this is where you consult the experts making sure to check in with all the different camps.

5. Pre-Mortem: how am I most likely to be wrong. If you got it wrong, why did you think you got it wrong? What might you have missed?

Michael actually took our audience and ran those steps on the question of “how many countries were there in the UN in 1980?” He had us vote on an app. There was quite a spread of number of countries with the median being 120.

Michael questioned us for why we arrived at our number, paying special attention to the outliers. After hearing all the theories, people revoted and the clusters tightened and the median moved higher to 145. Nobody in the room actually knew the answer — 154 — but we got very close as a group!

This is a level of accuracy that just doesn’t seem to be possible without going through the process. I mean, what was your guess? “What if it’s possible to be as accurate about the future as we just predicted the past?”

Of course, this being a Taiwan-focused blog we have to bring in a Taiwan angle. Are there any predictions of the chance of Chinese invasion of Taiwan out there? Yes, it is a question that is being continually tracked on the crowdsourced prediction site Metaculus. According to the wisdom of the Metaculus crowd, the chances of an invasion by 2025 is just 1%, the chances of an invasion by 2030 is 21% and increases up to 31% by 2035. If you think differently, you can add your opinion to the mix.

In a way, those numbers are comforting. Compared to the breathless prediction of invasion, a third to a fifth chance sounds manageable. But looking back to another prediction “Will Russia invade Ukrainian Territory before 2023?” Shows the community predicting the chances of an invasion at as low as 25% even in January 2022.