In “Forecasting Fox” Bobo breathlessly tells us how one man unveils a strategy for better forecasting. Not weather forecasting. He’s all about political forecasting… Prof. Krugman, in “Treasure Island Trauma,” says from the way things are going with Cyprus, you’d think the financial crisis didn’t teach our leaders anything. Here’s Bobo:
In 2006, Philip E. Tetlock published a landmark book called “Expert Political Judgment.” While his findings obviously don’t apply to me, Tetlock demonstrated that pundits and experts are terrible at making predictions.
But Tetlock is also interested in how people can get better at making forecasts. His subsequent work helped prompt people at one of the government’s most creative agencies, the Intelligence Advanced Research Projects Agency, to hold a forecasting tournament to see if competition could spur better predictions.
In the fall of 2011, the agency asked a series of short-term questions about foreign affairs, such as whether certain countries will leave the euro, whether North Korea will re-enter arms talks, or whether Vladimir Putin and Dmitri Medvedev would switch jobs. They hired a consulting firm to run an experimental control group against which the competitors could be benchmarked.
Five teams entered the tournament, from places like M.I.T., Michigan and Maryland. Tetlock and his wife, the decision scientist Barbara Mellers, helped form a Penn/Berkeley team, which bested the competition and surpassed the benchmarks by 60 percent in Year 1.
How did they make such accurate predictions? In the first place, they identified better forecasters. It turns out you can give people tests that usefully measure how open-minded they are.
For example, if you spent $1.10 on a baseball glove and a ball, and the glove cost $1 more than the ball, how much did the ball cost? Most people want to say that the glove cost $1 and the ball 10 cents. But some people doubt their original answer and realize the ball actually costs 5 cents.
Tetlock and company gathered 3,000 participants. Some got put into teams with training, some got put into teams without. Some worked alone. Some worked in prediction markets. Some did probabilistic thinking and some did more narrative thinking. The teams with training that engaged in probabilistic thinking performed best. The training involved learning some of the lessons included in Daniel Kahneman’s great work, “Thinking, Fast and Slow.” For example, they were taught to alternate between taking the inside view and the outside view.
Suppose you’re asked to predict whether the government of Egypt will fall. You can try to learn everything you can about Egypt. That’s the inside view. Or you can ask about the category. Of all Middle Eastern authoritarian governments, what percentage fall in a given year? That outside view is essential.
Most important, participants were taught to turn hunches into probabilities. Then they had online discussions with members of their team adjusting the probabilities, as often as every day. People in the discussions wanted to avoid the embarrassment of being proved wrong.
In these discussions, hedgehogs disappeared and foxes prospered. That is, having grand theories about, say, the nature of modern China was not useful. Being able to look at a narrow question from many vantage points and quickly readjust the probabilities was tremendously useful. The Penn/Berkeley team also came up with an algorithm to weigh the best performers. Let’s say the top three forecasters all believe that the chances that Italy will stay in the euro zone are 0.7 (with 1 being a certainty it will and 0 being a certainty it won’t). If those three forecasters arrive at their judgments using different information and analysis, then the algorithm synthesizes their combined judgment into a 0.9. It makes the collective judgment more extreme.
This algorithm has been extremely good at predicting results. Tetlock has tried to use his own intuition to beat the algorithm but hasn’t succeeded.
In the second year of the tournament, Tetlock and collaborators skimmed off the top 2 percent of forecasters across experimental conditions, identifying 60 top performers and randomly assigning them into five teams of 12 each. These “super forecasters” also delivered a far-above-average performance in Year 2. Apparently, forecasting skill cannot only be taught, it can be replicated.
Tetlock is now recruiting for Year 3. (You can match wits against the world by visiting www.goodjudgmentproject.com.) He believes that this kind of process may help depolarize politics. If you take Republicans and Democrats and ask them to make a series of narrow predictions, they’ll have to put aside their grand notions and think clearly about the imminently falsifiable.
If I were President Obama or John Kerry, I’d want the Penn/Berkeley predictions on my desk. The intelligence communities may hate it. High-status old vets have nothing to gain and much to lose by having their analysis measured against a bunch of outsiders. But this sort of work could probably help policy makers better anticipate what’s around the corner. It might induce them to think more probabilistically. It might make them better foxes.
Please to be remembering that Bobo is lavishly remunerated for producing this. Here’s Prof. Krugman:
A couple of years ago, the journalist Nicholas Shaxson published a fascinating, chilling book titled “Treasure Islands,” which explained how international tax havens — which are also, as the author pointed out, “secrecy jurisdictions” where many rules don’t apply — undermine economies around the world. Not only do they bleed revenues from cash-strapped governments and enable corruption; they distort the flow of capital, helping to feed ever-bigger financial crises.
One question Mr. Shaxson didn’t get into much, however, is what happens when a secrecy jurisdiction itself goes bust. That’s the story of Cyprus right now. And whatever the outcome for Cyprus itself (hint: it’s not likely to be happy), the Cyprus mess shows just how unreformed the world banking system remains, almost five years after the global financial crisis began.
So, about Cyprus: You might wonder why anyone cares about a tiny nation with an economy not much bigger than that of metropolitan Scranton, Pa. Cyprus is, however, a member of the euro zone, so events there could trigger contagion (for example, bank runs) in larger nations. And there’s something else: While the Cypriot economy may be tiny, it’s a surprisingly large financial player, with a banking sector four or five times as big as you might expect given the size of its economy.
Why are Cypriot banks so big? Because the country is a tax haven where corporations and wealthy foreigners stash their money. Officially, 37 percent of the deposits in Cypriot banks come from nonresidents; the true number, once you take into account wealthy expatriates and people who are only nominally resident in Cyprus, is surely much higher. Basically, Cyprus is a place where people, especially but not only Russians, hide their wealth from both the taxmen and the regulators. Whatever gloss you put on it, it’s basically about money-laundering.
And the truth is that much of the wealth never moved at all; it just became invisible. On paper, for example, Cyprus became a huge investor in Russia — much bigger than Germany, whose economy is hundreds of times larger. In reality, of course, this was just “roundtripping” by Russians using the island as a tax shelter.
Unfortunately for the Cypriots, enough real money came in to finance some seriously bad investments, as their banks bought Greek debt and lent into a vast real estate bubble. Sooner or later, things were bound to go wrong. And now they have.
Now what? There are some strong similarities between Cyprus now and Iceland (a similar-size economy) a few years back. Like Cyprus now, Iceland had a huge banking sector, swollen by foreign deposits, that was simply too big to bail out. Iceland’s response was essentially to let its banks go bust, wiping out those foreign investors, while protecting domestic depositors — and the results weren’t too bad. Indeed, Iceland, with a far lower unemployment rate than most of Europe, has weathered the crisis surprisingly well.
Unfortunately, Cyprus’s response to its crisis has been a hopeless muddle. In part, this reflects the fact that it no longer has its own currency, which makes it dependent on decision makers in Brussels and Berlin — decision makers who haven’t been willing to let banks openly fail.
But it also reflects Cyprus’s own reluctance to accept the end of its money-laundering business; its leaders are still trying to limit losses to foreign depositors in the vain hope that business as usual can resume, and they were so anxious to protect the big money that they tried to limit foreigners’ losses by expropriating small domestic depositors. As it turned out, however, ordinary Cypriots were outraged, the plan was rejected, and, at this point, nobody knows what will happen.
My guess is that, in the end, Cyprus will adopt something like the Icelandic solution, but unless it ends up being forced off the euro in the next few days — a real possibility — it may first waste a lot of time and money on half-measures, trying to avoid facing up to reality while running up huge debts to wealthier nations. We’ll see.
But step back for a minute and consider the incredible fact that tax havens like Cyprus, the Cayman Islands, and many more are still operating pretty much the same way that they did before the global financial crisis. Everyone has seen the damage that runaway bankers can inflict, yet much of the world’s financial business is still routed through jurisdictions that let bankers sidestep even the mild regulations we’ve put in place. Everyone is crying about budget deficits, yet corporations and the wealthy are still freely using tax havens to avoid paying taxes like the little people.
So don’t cry for Cyprus; cry for all of us, living in a world whose leaders seem determined not to learn from disaster.