The Signal and the Noise

First book read of 2018! Blocking out half to an hour before sleeping to read has been working well.

David recommended this book to me as an introduction to Bayesian thinking / probability. More generally, Bayesian thinking helps to correct, or at least bring into awareness, many cognitive biases and therefore help us be less wrong. I found the book to have clear explanations and quite entertaining. It was interesting to me that the author felt the need to explain the rules of poker but not baseball or political polling, the reverse would've been helpful to me personally. I do really like how the book is printed on super smooth (but not glossy!) paper.

The introduction basically outlines the pitfalls:

The instinctual shortcut that we take when we have "too much information" is to engage with it selectively, picking out the parts we like and ignoring the remainder, making allies with those who have made the same choices and enemies of the rest. 

The story the data tells us is often the one we'd like to hear, and we usually make sure that it has a happy ending. 

We face danger whenever information growth outpaces our understanding of how to process it. 

Prediction is important because it connects subjective and objective reality.

Once you learn a new idea, you start seeing it everywhere, much like when you learn a new word, you start hearing it in every conversation. I recently started listening to the Secular Buddhism podcast, and in episode 2 (or was it 3?) it tells the story of 6 blind men describing an elephant, which is the same idea as "the forecaster's next commitment is to realize that she perceives [the objective truth] imperfectly". 

Part 1 describes how predictions about the 2008 financial crisis fail because it used out-of-sample data and defines the difference between uncertainty (not quantifiable) and risk (quantifiable).  "When you can't state your innocence, proclaim your ignorance: this is often the first line of defence when there is a failed forecast." reminds me the concept of due diligence in the engineering profession, whose purpose is exactly to safeguard against proclamations of ignorance. 

Part 2 describes the difference between foxes and hedgehogs, which represent two different personality types. Hedgehogs tend to let strong ideology cloud their judgement. 

Part 3 describes the litmus test of good predictions: that the accuracy should improve with additional data. "The key to making a good forecast [is] having a good process for weighing [both qualitative and quantitative] information appropriately".

Part 4 is the rare success story of weather forecasting, where computational models are amended with human judgement. The easy part is that we know a good amount of atmospheric science so the models are based on sound theory, the hard part is that weather systems are chaotic (non-linear and dynamic). 

Part 5 describes how in contrast, earthquakes forecasting haven't been so successful. Some failures are because the models are overfitted, meaning that too specific of a solution has been found for a general problem. Earthquakes are also complex (simple elements but complex interaction, I know I just used a word to define itself). 

Part 6 brings up the famous quotation of "correlation doesn't imply causation", and says that:

To not confused correlation for causation, one need to have a theory of what the cause is

Technology did not cover for the lack of theoretical understanding about the economy; it only gave economists faster and more elaborate ways mistake Noise for a signal.

The amount of confidence someone expresses in a prediction is not a good indicator of its accuracy.

Part 7 describes how human behaviour messes everything up even more:

In many cases involving prediction about human activity, the vey act of prediction can alter the way that people behave. Sometimes, as in economics, these changes in behaviour can affect the outcome of the prediction itself, either nullifying it or making it more accurate. Predictions about the flu and other infectious diseases are affect by both sides of this problem.

Pretty much every course I've taken has made a joke of if only we can get rid of the pesky occupants.

Part 8 formally introduces Bayesian probability and bashes a great deal on the stats that's most commonly taught. I don't feel as bad that I forgot most of what I've learned.

We learn about [the universe] through approximation, getting closer and closer to the truth as we gather more evidence. [...] The Bayesian viewpoint regards rationality as a probabilistic matter.

The frequentist approach towards statistics seeks to wash its hands of the reason that predictions most often go wrong: human error. It views uncertainty as something intrinsic to the experiment rather than something intrinsic to our ability to understand the real world.
It emphasizes the objective purity of the experiment - every hypothesis could be tested to a perfect conclusion if only enough data were collected. However, in order to achieve that purity, it denies the need for Bayesian priors or any other sort of messy real-world context. These methods neither require or encourage us to think about the plausibility of our hypothesis

Part 9 describes chess as an example of a deterministic problem that is not within our (or even a supercomputer's) practical capabilities to fully solve. Instead we employ heuristics (rules of thumb).

Part 10 gives a step by step example of Bayesian probability applied to poker.

Part 11 describes that unfortunately more accessible information cancels out the wisdom of the crowds as we adopt a herd mentality. Our decisions becoming dependant, which creates a positive feedback loop for mistakes instead of vice versa.

Part 12 is even more depressing as it describes the politicization of science and the difficulty of communicating uncertainty (it was already bad enough learning about it in ENV221).

Part 13 describes "our propensity to mistake the unfamiliar for the improbable" using terrorist attacks as a case study.

...

I'm not sure how well I'll actually apply the lessons learned to my life. Perhaps I'll at least analyze data for my courses/thesis better?