The Black Swan: The Impact of the Highly Improbable

by Nassim Nicholas Taleb

Rating(1-10): 9

Prologue notes:
What you don't know is more relevant than what you know.
The inability to predict outliers implies an inability to predict the course of history.
Platonicity: we reply too much on idealized models.
Poeple think they know what is going on but they don't.
After the fact people invent reasons why things happened.
History does not crawl, it jumps.
Opinions tend to cluster and so they do not consider unlikely events.

Scalability:
Some professions are "scalable", that is, you do not need to be present to make money. These are the "idea" professions. Other professions depend on you being there and so you can't "stick it rich". These are the "physical" professions. Scalable Professions allow you to get rich but mostly they allow you to stay poor. Effort is not necessarily rewarded. It is "winner take all".

Scalability is new, with modern communication and transportation. For example, musicians.

Mediocristan and Extremistan:
In Mediocristan particular events are not important and don't affect the aggregate strongly. When your sample is large, no single instance will significantly change the aggregate or the total. In Extremistan, inequalities are such that one single observation can disproportionately impact the aggregate or the total. Not physical.

Black swans are not a factor in Mediocristan but are very important in Extremistan.

Mediocristan: height, weight, income for a baker, dentist, car accidents, mortality rates. etc.

Extremistan: wealth, income, book sales per author, book citations per author, name recognition of a celebrity, number of references on Google, populations of cities, sizes of companies, stoick ownership, inflation rates, economic data.

Turkey story:
A turkey is treated well for 1000 days and then killed on day 1001. His estimates of human regard for his welfare will be incorrect.

The problem of induction:
Usuually induction works, not not on scalable things. Hume was an induction skeptic.

The error of confirmation:
"I had lunch with O. J. Simpson and he didn't kill anybody."
People confuse "no evidence of possible black swans" with "evidence of no possible black swans" -- the "round-trip fallacy".
It is easy to find confirming evidence of almost anything. Hence it is not that useful or important.
One non-comforming event is all that is needed to disprove an idea -- the asymmetry of evidence.

Karl Popper and "falsification". Practical people like Popper. Charles Sauders Pierce had the same ideas.

When testing theories, people look for confirming evidence and don't usually look hard enough for falsifying evidence.

Charlie's thoughts:
In most situations, induction works. We can make generalizations. How do we know if we are in Extremistan?

The narrative fallacy:
People like stories. They like events to have causal relationships. It makes them easier to remember, more compact. People will go a long way to invent a story to explain events. They are uncomfortable with unexplained events. We impose an order than makes us think we understand the situation more than we actually do.

You always interpret what you see. it takes effort not to, to not theorize about the story of the events you see.

The same condition that makes us want to simplify pushes us to think the world is less random that it actually is.

Information is costly to abtain, costly to store, and costly to manipulate and retrieve. So we try to reduce those costs. Compact, organized information is more cost-effective. We want to reduce the dimensionality of the information.

This is a general problem with information. Information wants to be reduced.

Memory is not fixed. Each time we remember we add details and change the memory. We do this to make more sense of the information, to make it a better story, more memorable, simpler, more compact, more causal.

Biologically we are selected for quick response and making good stories because these helped us survive.

Rewards:
We seem to respond more to the number of rewards than to their intensity. So winning a little each day is better than winning a lot on one day. So we prefer activities that give us slow, steady rewards over ones where we lose almost all the time but win big occasionally. That is, we prefer to live in Mediocristan. But almost all of socioeconomic life is in Extremistan. It is hard psychologically to deal with things as they should be dealt with in Extremistan.

Bleed or blowup.

Most of life in nonlinear, hard to model accurately.

The problem of silent evidence:
The survivor bias, you usually don't see the failures.
The successes may be just there by chance.
Silent evidence is what events use to conceal their own randomness.
Species seem stable because you don't see all the ones which are extinct.
Crime may pay, you don't see the criminals who get away with it.
Drugs that hurt a few people but save a lot of people are generlaly not approved, even though approving them would save lives, overall.

Beginner's luck: This is a real thing, because people who are unlucky when they are beginner's stop gambling at much higher levels. So among current gamblers, many of them were lucky at the beginning.

Taking risks: If you are lucky you might think it is skill or you will remain being lucky.
"that we got here by accident does not mean that we should continue taking the same risks."

The anthropic principle: This is very similar. The universe is just right for life because we are here to notice that.

Causes: Of course, causes exist and many things are caused but you should not rush to judgement and assign causes to things that might be random.

The Ludic Fallacy:
Real life is not like a game.
Game theory is not like real life.
Las Vegas-style gambling is not subject to Black Swans and so is a very mild and manageable form of risk.
Gambling is not really risky at all, it is a game and well-understood.
The biggest loss in Las Vegas history: the lost $100 million on a non-replaceable performer maimed Roy of Siegfried and Roy.

Part I conclusions:
The cosmetic and the Platonic rise naturally to the surface.
You need to be empirical and avoid narration.

The Scandal of Prediction:
People are terrible at prediction.
Expects are hardly any better but only think they are better predictors and that makes them dangerous.
People do not realize how bad they are at predicting, or how much they don't know.
Guessing and predicting come down to the same thing. So we are bad at guessing too.

Information is bad for knowledge: Often you form an opinion quickly and then use the rest of the information to confirm your first opinion.

Experts who tend to be experts:
livestock judges, soil judges, astronomers, test pilots, accountants, grain inspectors

Experts who tend NOT to be experts:
stockbrokers, councilors, court judges, clinical psychologists, college admissions officers, psychiatrists, economists, political scientists, "risk experts", personal finance advisors.

Experts do not know what they do not know.

Expert'e prediction records:
There are not a lot of such studies but they show that experts are not any better at prediction than other people, or predicting that the future will be like the past.
But they always have some reason why their predictions did not turn out to be correct.

the Lake Woebegone Effect:
Humans tend to credit successes to their own skill and their failures to outside events, that is, randomness.
Most people think they are above average and they do not know they are overestimating themselves.

Anchoring:
People anchor on a number even if they know it is random. When a wheel is spun and a number comes up and then people are asked to guess some numerical value, their guesses correlate with the wheel number, even though they know it is totally random.

Discoveries and predictions:
Most discoveries are serendipitous and hence impossible to predict.
You must know something about the discovery to predict it and then you will have already discovered it.

Chaos:
Taleb thinks that chaos is over-hyped but it is a good example of not being able to predict things.

Taleb's favorite economists: J. M. Keynes, Friedrich Hayek and G. L. S. Shackle.

Unpredictibility:
He ties this in with chaos and dynamical system where tiny changes in input can cause huge changes in output. He references Poincare and the three-body problem.

Happiness research:
Taleb ties this in with the happiness studies (like Daniel Gilbert) that show people are terrible at predicting how happy something will make them.

The melting ice cube:
It is easy to predict what the result will be when an ice cube melts but it is impossible to determine what the ice cube was like from the pool of water left after it melts. This is a case where predicting the forward process is easy but the backward process is hard, in fact, impossible. (This is an information-losing transformation.) There could be trillions of possible causes for a single result. So it makes no sense to focus on these causes because you can never know which one will be important.

This relates to the famous "butterfly" phenomenon. Just because it is true does not mean that the beating of any particular butterfly's wings is important.

The difference between true randomness and deterministic chaos is mathematical not practical. In real life there is no difference.

Imcomplete information:
Randomness is sometimes just incomplete information. The sex of a pregnat woman I see on the street is random to me but not to her doctor.

History:
History is a great narrative and it is pleasing to know what happened in the past. But we need to be careful about looking for causes and applying history to predict the future. History has a large random element. History can provide negative confirmation but not positive conformation.

Prediction:
It is okay to predict in small matters. It is human. But avoid predictions in large, long-term matters. Instead, be prepared for all eventualities.

Positive accident:
Expose yourself to risks, to positive black swans.

Barbell strategy:
Keep most of your money in very conservative investments (say 85-90% T0bills) and keep some of it in extremely risky investments (like leveraged options) where there is a "floor" in what you can lose, and where the potential payoff is scalable and you could win big.

Don't keep all your money in medium-risk investments.

Seize opportunities: Try lots of things, expose yourself to as much volatility as possible.

The Great Asymmetry:
It is impossible to predict rare events or know their probability but it is relatively easy to predict their effects. So insure yourself against the effects and don't try to figure out what the probability is.

The world is unfair: It is controlled by randomness. As the world moves from being natural to man-made, that is, the systems that control our lives are man-made, this unfairness has increased because randomness and black swans increase.

Winner take all:
Research of economist Sherwin Rosen, "the economics of superstars". Inequality keeps getting greater because of the tournament effect. You run a tournament and the winner gets by far the biggest prize, even if the actual differences in ability or performance are very small. In fact, the differences are probably due to randomness anyway.

The Matthew Effect or the Power Law:
Research by sociologist Robert K. Merton. (Named after a Bible verse, Matthew 25:29). An initial advantage tends to build up to a greater and greater advantage. This is the "power law" we see in lots of places. the rich get richer. A random paper citation early can make your academic career. "cumulative advantage". This is related to the "herding effect" of opinions, reviews, etc.

Zipf's Law:
The same effect. A few words in English are by far the most used. Power laws. Preferential attachment.

The Long Tail:
The Web causes concentration but it also has a long tail who items waiting to get big and who, taken together, constitute a collective force. This acts as a counteracting force to the unfairness of randomness. It is a threat to the big guy. No one is safe. It should promote cognitive diversity.

Globalization:
Globalization has made everything interrelated, like the world banking system. That makes us vulnerable to a black swan.

This is related to network theory (Duncan Watts, Albert-Laszlo Barabasi, Steven Strogatz).

Reversing the inequality:
Economic inequality can be reserved somewhat with legislation but the superstar inequality and success inequality cannot.

Gaussian vs Mandelbrotian:
Gauss just did the mathematics, other people pushed it as a model for everything in the world. Bell curves work for physical things where gravity applies but not for social things that are scalable.

Models of "risk" are usually based on Bell curves and their associated assumptions.

People like Gaussian models because they are predictable.

Mandelbrotian models have much more inequality.

Taleb says he is in favor of equality but wants us to see the world as it really is. Reality is not Mediocristan so we should leanr to live with it.

Correlation and regression: Both only make sense in a Gaussian model. They are useless in Extremistan.

80/20 Rule:
Another example of power laws.

Bell Curve assumptions:
(1) each observation is independent of the others: we have seen how cumulative advantage and prefereneial attachment make this not true in most situations.
(2) no "wild" jump, all jumps are one unit (like iterated coin tossing). This is not realistic.

Fractal randomness:
Benoit Mandelbrot. A better model of randomness than the Gaussian. Scalable.

Power laws:
polynomials rather than then exponentials of Gaussian randomness. Hence the extreme cases are much more likely.

Typical: power of two. Twice the income, one fourth of the number with that income.

Describe a wide range of phenomena: work frequency, website hits, books sold, magnitude of earthquakes, diameter of lunar craters, intensity of solar flares, net worth, population of cities, markewt moves, company size, etc.

Network theory leads to many examples of power laws.

Taleb likes power laws but says you cannot accurately estimate the exponent because you need so many cases to establish it. Unless you assume the distribution you cannot know how many cases you need to estimate the parameters. The models might fit reality when we tweak the parameters but that does not mean that the model is really a good model of reality.

It may well be that the power exponent changes as you change scales. The are self-affine not self-similar.

He did not balck box experiments where inside was a power law generator and on the outside he used standard methods to estimate the power. He always overestimated the power, that is, erred on the side of underestimating the randomness.

Stock market:
Over the last 50 years, half the returns come from 10 days. If you remove those 10 days, the markets only went up half as much.

Nobel economists:
He is contemptuous of them.

Harry Markowiz and William Sharpe, modern portfolio theory. He thinks it is worthless and dangerous. Based on Gaussian assumptions. The Nobel rewards mathematical models, even if they are incorrect.

Myron Scholes and Robert C. Morton. He thinks their Nobel was for junk science too.

Uncertainty Principle:
Taleb thinks it strange that people talk about the Heisenberg Uncertainty Principle as if it was about real uncertainly. It is about very mild, Gaussian uncertainty which is hardly uncertain at all. Life is much more uncertain but somehow people don't talk about that as much.

Final words:
He is conservative where he could be hurt by negative black swans. He never misses an opportunity or a chance to expose himself to a positive black swan. He likes to take big risks on things everyone knows are risky, but he is conservative on things not normally thought to be risky, i.e., blue chip stocks. Don't worry about the small risks but protect yourself from the big ones.