Thinking in Bets by Annie Duke (November 2018)
A critical skill for any decision maker is knowing how to analyze the outcome of a decision. This means a person must know how to weigh the importance of the factors that went into a decision and understand how each factor influenced the final observed outcome. This isn’t as simple as it sounds because most people are unable to overcome the biases in their own views or positions.
The most obvious way people do this is by interpreting events through filters that allow their analysis to remain consistent with prior conclusions. A person who believes people run faster in the sunlight, for example, will always consider weather conditions when assessing the outcome of a road race and might dismiss the presence of other more important variables. This tendency is natural and therefore somewhat unavoidable – plus, the difference between a bias and a variable is often a matter of openness to rethink. The issue isn’t necessarily the filter itself but our unwillingness to acknowledge when a filter is self-serving.
People who use their intelligence to construct narratives that explain away contradicting information compound this problem. Instead of recognizing a possible bias, these people construct ever more elaborate explanations that preserve their worldview. A good example might be the student who believes his or her failing test scores are first the fault of the questions, then the teacher – anyone but the student! In the context of Thinking In Bets, what these narratives prevent is the kind of open analysis required for cultivating an ongoing process of continuously improving a decision making process.
The way most people consider the roles of luck and skill in their outcomes undergoes a similar process. A poor analysis will involve seeking plausible reasons for an outcome that paints us in a flattering light. This usually means someone analyzing an outcome that had a lot more to do with luck than skill will explain the result with a series of characteristics or factors that are difficult to prove – character, toughness, perseverance, and so on. As stated a couple of times above, the key to overcoming this flaw in the way we naturally assess outcomes is to be aware of how our explanations serve our interests, positions, or worldviews and to question any conclusion that paints us in too flattering a light.
One up: Of course, I fail to acknowledge above how difficult it is to analyze outcomes without bias. My guess is that although some people can come pretty close to being entirely without bias in their individual analysis, no one out there is above benefiting from a little help. Duke cites research that two people will often move closer together on an issue after a debate or skilled explanation of the opposing position and leveraging the wisdom of this finding is one way to strip away bias in an analysis. A good start toward this goal might be to find a trustworthy partner or form a diverse group that will always find ways to play Devil’s advocate and explore the many possible factors involved in a certain outcome.
One down: My adventures in business and analytics often bring me across KPI – key performance indicators. This magical-sounding concept is a way to proactively track important outcomes. The methodology for calculating these metrics ranges from the simple to the complex. The important thing isn’t the difficulty level of the math but rather what the metrics reveal about performance.
Now, though helpful in many ways, one significant danger of a KPI is how it encourages lazy outcome analysis. Even if a truth about a KPI is well established – for example, when everyone agrees that if ‘X’ goes down, it’s because ‘Y’ went up – someone should investigate the underlying causes of why the metric moved as it always did. The teams and organizations that can maintain a fresh perspective to data they’ve seen before will always do well while those who glance at the outcome before moving on will someday find themselves blindsided by the unexpected.
Just saying: I liked the thought that a good scientist should report everything that might make his or her findings invalid. It really captures the essence of this book – no individual is infallible to certain reasoning errors, so we must do everything in our power to leverage different perspectives and make it easy for our peers or colleagues to check our thinking.
