If you are over 50 and male this could be the most important post you read this year.
I have just finished reading The Signal and the Noise: The Art and Science of Prediction by Nate Silver. The book discusses a how a diverse set of forecasts ranging from politics, baseball and the weather are prepared, the errors that are often made and how in many cases ‘expert predictions’ should be treated with many grains of salt.
However a real strength of the book is the description of Bayesian reasoning in Chapter 8 which is a technique every manager should learn. A lot of management effort is spent altering forecasts as new information is received Bayes allows to make better predictions. This book made me really learn about Bayes Theorem.
Simply put one takes a prior probility and compares it to a new event and obtains a posterior probability. Say x equals the prior probability, y equals a new event probability that is true, and z equals a new event probability that is false. The posterior probability is xy over xy plus (1-x)z. The secret is calculating both the true and false positives.
Say for example there has been an accident in a city involving a taxi cab.
* 85% of the cabs in the city are white, and 15% are silver.
* A man identified the cab involved in a hit and run as silver.
* The court tested the witness' reliability, and the witness was able to correctly identify the correct color 80% of the time, and failed 20% of the time.
What is the probability the taxi cab was silver? Here's how we figure it out using Bayes theorem.
If the cab was silver, a 15% chance, and correctly identified, an 80% chance, the combined probability is .15 * .8 = .12, a 12% chance. These are true positives.
If the cab was white, an 85% chance, and incorrectly identified, a 20% chance, the combined probability is .85 * .2 = .17, a 17% chance. These are false positives
Since the cab had to be either white or silver, the total probability of it being identified as silver, whether right or wrong, is .12 + .17 = .29. In other words, this witness could be expected to identify the cab as silver 29% of the time whether he was right or wrong.
The chances he was right are .12 out of .29, or 41%.
Now recently I took a PSA test and my reading was above the supposed danger level. What is the probability I have prostate cancer?
The chances of have prostate cancer at various ages are as follows:
For a man in his 40s - 1 in 1000
For a man in his 50s - 12 in 1000
For a man in his 60s - 45 in 1000
For a man in his 70s - 80 in 1000
I am 69 so my chances of prostate cancer would be 63 in 1000. However I have now had a positive PSA test result.
Now according to a medical website for every 100 men over age 50, with no symptoms, who have the PSA test:
10 men out of 100 tested will have a higher than normal level of PSA. These men must then go through other tests and examinations. At the end of these tests :
• Three of the ten men with a higher than normal PSA level will be found to have prostate cancer
• Seven of the 10 men with a higher than normal PSA level will be found not to have prostate cancer at the time of screening
90 men out of 100 tested will have a normal PSA level. Of these 90 men :
• 88 of the men with a normal PSA level will not have prostate cancer.
• One or two of the men with a normal PSA level will actually have prostate cancer, undetected by the test.
The probability that the PSA test gives a true positive for me is 0.063 x 88/90 or 0.0616 (xy in the formula; note two people out of 90 are missed.)
The probability that the PSA test gives a false positive for me is 0.927 x .07 or 0.0656 ((1-x)z in the formula.
The sum of the true and false positives is 0.1272 and so according to Bayes the probability that I have prostate cancer is 48% which is much higher than I originally thought and means that I will go forward with a biopsy. I never would have come to this conclusion without reading Silver's book
I did find the final chapter of climate science to be weak. I confess I am a sceptic but Silver tries to justify Climategate and in my opinion fails badly. As reviewer Robert has noted the Climategate scandal had nothing to do with the global temperatures, the greenhouse effect, or basic climate science. Climategate is all about how some prominent scientists fudged data, erased embarrassing results, and sought to control the peer review process in leading scientific journals to suffocate dissenting opinions.
Indeed the proponents of Climategate sound very similar to political pundits that Silver so effectively lacerates in the first chapter of his book.
Loading your book clubs
There was a problem loading your book clubs. Please try again.
Not in a club? Learn more
Join or create book clubs
Choose books together
Track your books
Bring your club to Amazon Book Clubs, start a new book club and invite your friends to join, or find a club that’s right for you for free.
Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required. Learn more
Read instantly on your browser with Kindle for Web.
Using your mobile phone camera - scan the code below and download the Kindle app.
Flip to back Flip to front
Follow the Author
Something went wrong. Please try your request again later.
Book recommendations, author interviews, editors' picks, and more. Read it now
Frequently bought together
- +
- +
Total price:
To see our price, add these items to your cart.
Choose items to buy together.
Customers who viewed this item also viewed
Page 1 of 1 Start overPage 1 of 1
About the author
Follow authors to get new release updates, plus improved recommendations.
Top reviews from the United States
There was a problem filtering reviews right now. Please try again later.
Reviewed in the United States 🇺🇸 on October 11, 2013
Reviewed in the United States 🇺🇸 on September 21, 2013
Reviewed in the United States 🇺🇸 on November 21, 2015
Reviewed in the United States 🇺🇸 on August 9, 2015
Top reviews from other countries
Stiven Skyrah
An amazing read on predictions
Reviewed in the United Kingdom 🇬🇧 on May 1, 2019
Rob Kitchin
a good introductory overview of predictive analytics
Reviewed in the United Kingdom 🇬🇧 on September 13, 2013
John Tierney
Very U.S. centric, but some interesting chapters nonetheless
Reviewed in the United Kingdom 🇬🇧 on July 31, 2013ROBIN
Very good but possibly overrated
Reviewed in the United Kingdom 🇬🇧 on March 10, 2021
