Life
3 Tactics Ordinary Citizens Used To Out Predict CIA Experts
In the intelligence community, it’s crucial to be able to predict future events. The ability for the CIA and its intelligence counterparts to be able to forecast world events based on limited information is what gives us advantages on the international stage.
In a recent interview on The Science of Success Podcast, New York Times best-selling author Dan Gardner shares the incredible story of how a group of citizen volunteers led by Philip Tetlock, changed the way our intelligence communities’ approach analyzing and forecasting future political events.
It all started in 2011. The Office of the Director of National Intelligence (ODNI) decided it was time to evaluate and improve on how The United States’ 16 intelligence agencies forecast future events. The idea was to analyze these agencies past predictions and find ways to learn from previous mistakes.
“The intelligence community actually spends a lot of its time not just spying, but also analyzing information to try and figure out what’s going to happen next. Will Russia try to seize the Crimea? They’ll try to make forecasts. On all parts of geopolitical events, including economic events like what’s going to happen at the Chinese economy in the fourth quarter, that sort of thing.”
So in order to gain perspective the intelligence community agreed to fund a number of private researches to assemble teams of unpaid volunteers to take their best shot at it. “You set up a team to make forecasts, and we’ll ask questions, and they’ll be the real world questions that we have to answer all the time.” Gardner explains.
Ultimately they came up with five university based research teams for this tournament. One of which was headed up by Gardner’s co-author, Philip Tetlock. They decided to name their team The Good Judgment Project. The Project successfully recruited thousands of volunteers. The ODNI set several benchmarks that all of the researchers thought we’re much too ambitious. “Prior to beginning the project, looking at these benchmarks, nobody could do this!”
The results were beyond surprising. Certain volunteers blew away each benchmark and predicted future events with incredible accuracy. “The Good Judgment Project won hands down. They found that a certain group of these volunteers were consistently good at forecasting these events. Anyone can get lucky once, twice, maybe three times. These volunteers however were consistent, and that is key.”
These “Superforecasters” were normal people, with average IQ scores, no previous intelligence community experience, and zero access to classified information. Each of these volunteers had a common interest in world events and stayed up to date on their world politics, but had nothing else you or I couldn’t find ourselves on Google. Yet they were making predictions that economists said should have been impossible.
So how did they do it? Dan points out three things the researchers from The Good Judgment Project found these volunteers had in common.
1. Unpack Your Problems
Many of us look at a problem as one thing. We are presented with the issue, we look it over, stroke our chins, and then try to solve. However, looking at the problem or question as one big issue will lead to terrible predictions.
“What they do is that they methodically unpack the question. They take a big question, and they unpack it, and make a whole series of smaller questions, and then they unpack those and they make a series of smaller questions, and they methodically examine them.”
You’re taking one complex problem, breaking it down into its smaller, simpler components. Allowing you to answer these simpler questions and ultimately piece them back together to create an extremely accurate and effective answer to a larger problem.
“If you’re overwhelmed by the size of a problem, break it down into smaller pieces.” – Chuck Close
2. Craving Cognition
Each of these “superforecasters” had a desire to be intellectually challenged and learn. “These are people who like to learn, they’re constantly picking up bits and pieces of information, and no surprise, when you spend a lot of time picking up this sort of information, eventually you will have quite a number of dots in your intellectual arsenal for you to connect.”
Also, each volunteer enjoyed pushing himself or herself mentally. “They’re the kinds of people who do puzzles for fun and the harder the puzzle is, the more fun it is,” explains Gardner.
So next time you decide to do some recreational problem solving whether it be a puzzle or mental game, crank the difficulty up a little bit and give your brain a challenge.
3. It’s All About Your Mindset
Each “superforecaster” volunteer was extremely open to hearing other’s opinions. “This means okay, I’ve got my perspective but I want to hear your perspective. I want to hear somebody else’s perspective. I want to hear different ways of thinking about this problem.”
By listening to as many other opinions as possible, these volunteers were able to then synthesize ideas from multiple sources and viewpoints and create the most well rounded and analyzed prediction possible.
It’s easy to want to take all the credit, to finish first, and be the star of the show. We all have the impulse to think we’ve done our due diligence and our viewpoint is the right one. It’s only through keeping an open mind to other viewpoints and outside information that we can truly begin to consistently make better predictions and decisions.
“Once your mindset changes, everything on the outside will change along with it.” – Steve Maraboli
The Good Judgment Project is still being conducted to this day and is open for volunteers. To learn more about the project, forecasting tactics, and Dan’s book Superforecasting, listen to his full hour interview on The Science of Success Podcast.