Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification [Paperback]

Jonathan Zdziarski (Author)

Editorial Reviews

Review

A highly recommended read for anyone in charge of controlling spam in a corporate environment. -- Midwest Book Review, September 2005 (http://www.midwestbookreview.com/mbw/sep_05.htm)

Does a good job of addressing advanced, complicated issues, but putting it in terms that readers can grasp. -- Netsecurity.about.com, August 29, 2005, http://netsecurity.about.com/od/5/fr/aabrendspam.htm

Does a great job educating us on logic and thought taken to combat this SPAM blight on the Internet. -- MacCompanion, August 2005, 5 out of 5 stars

Highly recommended read for anyone in charge of controlling spam in a corporate environment [and] on their own system. -- Readers Preference, readerspreference.com/reviews/endingspam.html

IT managers who want a better understanding of how anti-spam products work should shell out the $39.95 price at once. -- eWeek, July 25, 2005

If you’re looking for a primer on how the anti-spam battle is fought, you can’t do much better. -- InfoWorld, July 14, 2005,

Leads the charge against what has become a very significant challenge to both productivity and sanity -- Book News, September 2005 (http://www.booknews.com/issues/sci-current.pdf)

Not only enjoyable but actually captivating. -- Linux Magazine, October 2005 (http://www.linux-magazine.com/issue/59/Book_Reviews.pdf)

The first book explaining the fine details of the theoretical models and machine-learning algorithms implemented in these filters. -- Slashdot, August 15, 2005

This book is down, and dirty, loaded with information, and will make your head-hurt (in a good way). -- MyMac, August 29, 2005, mymac.com/showarticle.php?id=2074

Product Description

Join author John Zdziarski for a look inside the brilliant minds that have conceived clever new ways to fight spam in all its nefarious forms. This landmark title describes, in-depth, how statistical filtering is being used by next-generation spam filters to identify and filter unwanted messages, how spam filtering works and how language classification and machine learning combine to produce remarkably accurate spam filters.

After reading Ending Spam, you’ll have a complete understanding of the mathematical approaches used by today’s spam filters as well as decoding, tokenization, various algorithms (including Bayesian analysis and Markovian discrimination) and the benefits of using open-source solutions to end spam. Zdziarski interviewed creators of many of the best spam filters and has included their insights in this revealing examination of the anti-spam crusade.

If you’re a programmer designing a new spam filter, a network admin implementing a spam-filtering solution, or just someone who’s curious about how spam filters work and the tactics spammers use to evade them, Ending Spam will serve as an informative analysis of the war against spammers.

See all Editorial Reviews

Product Details

• Paperback: 312 pages
• Publisher: No Starch Press; 1 edition (July 1, 2005)
• Language: English
• ISBN-10: 1593270526
• ISBN-13: 978-1593270520
• Product Dimensions: 9.1 x 7 x 0.7 inches
• Shipping Weight: 1.1 pounds (View shipping rates and policies)
• Average Customer Review: 4.1 out of 5 stars  See all reviews (15 customer reviews)
• Amazon Bestsellers Rank: #502,272 in Books (See Top 100 in Books)

  #24 in	Books > Computers & Internet > Security & Encryption > Viruses
  #9 in	Books > Computers & Internet > Hardware > Microprocessors & System Design > DSPs

Customer Reviews

Most Helpful Customer Reviews

17 of 19 people found the following review helpful:

4.0 out of 5 stars Actually quite entertaining, July 16, 2005

By	Anthony Lawrence "Unix, Linux and Mac OS X" (Middleboro, MA USA) - See all my reviews (VINE VOICE)    (REAL NAME)

This review is from: Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification (Paperback)

The sub-title of this scared me a bit, because it sounds like heavy geek territory. A review of chapter titles raised my eyebrows a too: "Fifth Order Markovian Discrimination" - I visualized page after page of unintelligible mathematical symbols.

That's not the case at all. Actually Markovian Discrimination is a technique I've used in other programming efforts, and the author explains it in simple and entertaining language. There's nothing here that any competent programmer can't grasp.

I'm a little hesitant to call this book entertaining, although it absolutely is. I only hesitate because that might give the impression that it's more fluff than substance, and that's not the case at all. There's a lot of substance here, both in theory and in practical advice. And although the subject is definitely spam, some of the techniques and methods discussed here apply to other programming challenges as well.

The first part of the book is especially enjoyable. It's a history of spam, and I learned things I hadn't known before about spam's early days. It then segues into analysis; in a sense you get desert before the meat and potatoes.

Overall, worth reading, even by non-programmers wanting to understand more about what current anti-spam efforts are all about.

4 of 4 people found the following review helpful:

5.0 out of 5 stars Excellent discussion of spam, July 30, 2005

By	Harold McFarland (Florida) - See all my reviews (VINE VOICE)    (TOP 100 REVIEWER)    (REAL NAME)

This review is from: Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification (Paperback)

Author Jonathan A. Zdziarski starts this book by giving the reader a history of Spam as well as the historical approaches to fighting Spam. This is followed by a very practical guide for the serious Spam fighter; including details on statistical filtering, tokenization, Markovian discrimination, and Bayesian filtering. Although it is very technical in many respects most readers should be able to comprehend the text if they read carefully. Readers who already understand the basics of filtering and email analysis will find it both easy and educational to read.

The author includes an excellent section on spammer tricks and how they get past fileters as well as what to do about it. This section alone makes the book worth the price. Ending Spam is a highly recommended read for anyone in charge of controlling spam in a corporate environment as well as on their own system.

10 of 13 people found the following review helpful:

4.0 out of 5 stars Nice overview ... but leaves you wanting more, September 18, 2005

By	Nils Valentin (Tokyo, Japan) - See all my reviews

Amazon Verified Purchase

This review is from: Ending Spam: Bayesian Content Filtering and the Art of Statistical Language Classification (Paperback)

Ending Spam from Mr. Zdziarski is a well written BASIC and easy to understand INTRODUCTION to get a technical overview of todays spam fighting solutions on the market.

Also it is written on the cover that it is f.e focused towards developers, network admins etc. I would consider the target customer to be IT Managers, or other curious people who want to get an overview.

Thats what it does and it does it very well in my eyes.
The book provides simplified, abstract overviews of some available spam filters solutions.

The book is provided into 3 parts

- An Introduction part to spam filtering (Chapter 1-4)
- A part describing "Fundamentals of Statistical Filtering" (Chapter 5-9)
- an the third part describing "Advanced Concepts of Statistical Filtering" (Chapter 10-14)

Its a bit confusing that Chapter 4 has the same title than Part II. So perhaps Chapter 4 should have been part of "Part II" ?

The Chapters which I found most interesting were:

Chapter 4 "Fundamentals of Statistical Filtering"
Chapter 7 "The Low down dirty Tricks of spammers"
Chapter 9 "Scaling in Large Environments"

I am sure the author could have easily filled the book with Chapter 7 alone. The book is very entertaining and has a nice motivating writing style. You might at times find some rant about the spammers which I have chosen to ignore as it doesnt contain any valuable information or anything which I didnt know already. While I might agree to some of the authors views, I believe that the rant does unfortunately do exactly the opposite in my eyes and does give spammers credit to how they do their work.

I personally was actually looking for a companion book to "The Book of Postfix" to help me further explore new anti spam technology.
I was hoping to find overview charts, being able to compare different solutions,features, (dis)advantages. So in this sense, I was actually looking for workshop style instructions, tuning advice, troubleshooting advice etc.

The authors does explain f.e (Chapter 14) Collaborative Algorithms but he does not go into detail which products support the feature and how to perform the setup. He does provide some weblinks in his book from which the interested reader might further investigate the topic.

From reading the Chapter10 on "Testing Theory" its easier to conclude why the author doesnt go into more detail. If he would have done so, the book could have been easily 2-3 times the size.

I assume, this is partly due to the fact that the anti spam technology /products/market is still fairly young .


Summary:

"Ending Spam" gives a very BASIC INTRODUCTION to the current available Anti spam technology and some chosen products. After you have read the book you have a first vague idea what type of solutions exist. You will actually need other books to intensify the "knowledge" you have gained here.

The fact that the book is written in simple terms makes it easily acessable for a wide market, however if you are a technichian you will perhaps find that the book just doesnt contain enough "meat" for you.

I would still recommend the book for Managers which need to know only the rough details, beginners, or a first time read for newcomers.