Thinking Fast and Slow Notes (Part 3)
Jan. 7, 2020
Thinking: Fast and Slow
Daniel Kahneman, 2011 Thinking Fast And Slow The following are sections and ideas in condensed form that I found particularly salient.
Part 3 - Overconfidence
We think we understand the past, which implies that the future is knowable.
We create flawed stories of our past which shape our views of the world and future expectations.
~ Nassim Taleb’s Black Swan illustrates human narrative fallacy to describe how
~ The fantastic story of Google.
~ People who knew about the financial crash of 2008. The human mind does not do well with non-events.
Hindsight Bias leads us to assess how good a decision was based on it’s outcome.
Studies asked people to predict outcomes of certain events that gripped public interest: Nixon meeting Mao, OJ Simpson trial, President Clinton’s proposed impeachment. When the result was as predicted, people exaggerated their initial confidence. When it was different, they recalled their estimation different from what it was, and more aligned with what actually happened. This is called Outcome Bias.
Decision Makers take blame for decisions that are obvious only after the results are obtained. Example: September 11, 2001 attack “predicted” by July 10, 2001 intelligence that Al-Qaeda were planning a major attack against the US. Who could have known really though?
Hindsight and Outcome Bias breeds bureaucracy and risk aversion.
Operating procedure is difficult to second-guess. They also breed massive reward for lucky risk-seekers. Entrepreneurs can gain the crown of boldness and prescience with a few lucky gambles.
Consumer demand for illusory certainty - an explanation for a businesses success or failure. So much is attributed to a CEO, whereas his or her influence on success is quite small.
Overstated messages of business books, that good managerial practice can be identified and will provide successful results. Humans crave this causality, and ignore the effect of luck and inevitability of regression.
Illusion of Validity - “Considering how little we know - the confidence we have in our beliefs is preposterous - and it is also essential.”
The Leaderless Group Challenge - Soldiers have to haul a heavy log over a six foot wall without it touching the ground or the wall. Observations of individuals’ characters (who takes charge, who resists, who is submissive) decided who to send to officer school. Later feedback from commanders at the school indicated that the observers’ forecasts had little ability to predict who would succeed as a leader.
The Illusion of Stock Picking Skill - A Random Walk Down Wall Street
Terry Odean (UC Berkeley finance professor) data analysis of 10,000 brokerage accounts, 163,000 trades, indicated that sold stocks (believed will go down) actually outperformed bought stocks (believed to head up). On average, the sold stocks did 3.2% better. Doing nothing would have yielded better results.
“Trading is Hazardous to Your Wealth” shows that the most active traders had the poorest results. “Boys Will be Boys” showed that men generally trade stocks more than women do, and as a result women achieved better returns.
Large historical events determined by luck.
Hitler’s embryo had a 50/50 chance of being female.
Television’s Panel of Experts
Philip Tetlock’s book Expert Political Judgement: How Good is it? How Can we Know?
exposed the truth that people who spend their time and earn their living studying a particular topic produce poorer predictions than if they evenly distributed their choices over all the options (i.e. dart throwing monkeys). He also discovered that...
The most famous predictors are the most flamboyant ones.
Tolstoy’s “The Hedgehog and The Fox”
Hedgehogs - Confident in forecasts and reluctant to admit error. Incorrect predictions are only “only off on timing” or “very nearly right”.
Foxes = Complex thinkers who believe reality emerges from largely uncontrollable interactions between different agents and forces. Their predictions are on average better than those of the hedgehogs, but not by much. They are much less likely to be invited on a television show.
To Be Right? Or To Be Known? - You’ll probably be wrong regardless.
But it’s not the experts fault, the World is difficult.
1. Errors in predictions are inevitable because the world is unpredictable.
2. High subjective confidence is not an indicator of accuracy. (Low confidence might be though) 3. Do not deny the validity of all tests. If it can predict an outcome 20-30% of the time, the test is worth using.
Prediction also depends on the realm of interest. The question should not be how well the experts are trained, but how predictable their subject matter is.
Intuition vs. Formulas - A predictive Formula with few equal weighted parameters beats an ‘expert’ forecast.
Paul Meehl’s “my disturbing little book”
Goal was to predict college freshmen grades at the end of the school year. Counselors
had a 45 minute interview with each freshman, high school grades, several aptitude tests, and a four page personal statement. A predictive formula took into account only high school grades and one aptitude test. The formula was more accurate than 11 of 14 counselors.
Vintage wine pricing predictive formula. Prevailing knowledge is that quality of vintage is determined predominantly by weather during the grape growing season. Ashenefelter’s formula provides accurate pricing decades into the future taking only into account three parameters: summer temperature, harvest-time rainfall, and previous winter’s rainfall. Correlation between his prediction and actual prices is above 0.90.
“Robust Beauty of Improper Linear Models in Decision Making” by Robyn Dawes
Why are experts inferior to algorithms?
1. Attempts at cleverness reduce validity. Experts weigh personal impressions too heavily over the statistics.
2. Inconsistent judgement (experienced radiologists evaluating chest X-rays as ‘normal’ or ‘abnormal’ contradict themselves 20% of the time.
Conducting interviews is likely to diminish the accuracy of a selection procedure!
Algorithms which assigned equal weighting to all factors are often superior to those with subjective weighting because they are not affected by accidents of sampling.
Example: Marital Stability = Frequency of Lovemaking - Frequency of Quarrels (negative is bad)
Virginia Apgar Test in the delivery room determines if just-born infants require immediate intervention or not. Many clinicians focused too closely on one factor.
score = heart rate + respiration + reflex + muscle tone + color
each parameter given value 0, 1, or 2 depending on robustness of sign score > 8 indicate healthy baby
score < 4 needs immediate attention
A Checklist Manifesto by Atul Gawande
Difference in emotional reaction to algorithm failure vs. human error. We are more ready to
forgive human error.
Algorithm role in everyday life increasing
Software music and book recommendations
Cost of rookie players to professional sports teams (Money Ball) When to punt on fourth down.
Kahneman’s evaluations of fitness for combat duty.
Six factor formula including Responsibility, Sociability, and Masculine Pride.
Also factual questions like previous jobs, punctuality at work/studies, frequency of
interactions with friends, interest in sports. Factual questions combat the Halo Effect.
Interviewer protests led Kahneman to provide some subjective analysis. “When you are done conducting the interview exactly as instructed, close your eyes and imagine the recruit as a soldier, then assign him a score of 1 to 5.”
Algorithm was more successful at predicting future performance than previous typical interviews. Also now, the interviewers’ subjective scores were also better. Kahneman concluded that intuition adds value, but only after disciplined collection of objective information and scoring of specific traits.
Do not trust intuitive judgement, but do not dismiss it.
You can make your own choice-making algorithms. Six factors is a good number to use, and be disciplined about data collection. Do not skip around.
Expert Intuition: When Can We Trust It?
~ Sources of Power by Gary Klein - Experienced firefighters ‘know’ intuitively to get out of a burning building right before it collapses.
~ Blink by Malcom Gladwell - Art experts gut feeling that a kouros (sculpture) is fake. Triumph of intuition.
Intuition = Recognition
(RPD - Recognition Primed Decision)
Acquiring expertise in a domain is a large collection of mini skills, which can take a lot of time. Time to become a Chessmaster? Over 10,000 hours (6 years of play 5 hours a day)
Two basic conditions to acquire a skill
1. Sufficiently regular environment to be predictable
2. Opportunity to learn the regularities through prolonged practice
Stock pickers and political scientists operate in a zero validity environment.
Worse than zero “wicked environment” = Lewis Thomas’s example of a physician in 1900’s who was clinically infallible at determining if someone had typhoid. His predictions were accurate because he didn’t wash his hands between tongue palpitations.
Intuition cannot be trusted in the absence of stable regularities in the environment.
Importance of feedback
Driving a car provides immediate feedback Piloting a harbor ship has delayed feedback
On the journey to near perfection, some mistakes are made with great confidence.
The Outside View
The proper way to elicit information from a group is not by starting with a public discussion, but by confidentially collecting each person’s judgement.
1. Forecasts are unrealistically close to best-case scenarios.
2. Plans can be improved by consulting the statistics of previous similar cases.
Kitchen remodelers expect to pay $18,658. On average, they actually spent $38,769. Government projects (Scottish Parliament in Edinburgh estimated at £40 million, ended up costing £431 million)
Use the “Outside View” to Mitigate Planning Fallacy
1. Identitfy reference class (kitchen renovation, railway project)
2. Gather statistics to generate a baseline (cost per mile or sq. ft., percentage overbudget) 3. USe specific information to adjust the baseline prediction.
Neglecting the outside view, people are often too optimistic. This leads them to take on risky projects. “It probably contributes to why they litigate, start wars, and open small businesses.”
An optimistic attitude is largely inheritied, and it is part of a general disposition for well-being, which may also include a preference for seeing the bright side of everything. If you allowed one wish for your child, seriously consider wishing him or her optimism.
35% = Chance a small business will survive for 5 years in America.
The financial benefits of self-employment are mediocre: given the same qualifications, people achieve higher average return by selling their skills to employers than by setting out on their own.
90% of drivers rate themselves as “better than average”
Overconfidence - Analysis of 11,600 market forecasts from CFOs of large companies revealed a correlation of less than zero! When they said the market would go down, it was actually more likely to go up.
President Truman’s demand for a “one-armed economist” who would take a clear stand. Tired of hearing “on the other hand...”
Confidence is valued over Uncertainty
Martin Seligman’s Positive Psychology says that “an optimistic explanation style” contributes to resilience by defending one’s self image.
For cold callers, losing a call because “she was an awful woman” is better than “I am an inept salesman.”