The Tao of Network Security Monitoring: Beyond Intrusion Detection [Paperback]

Richard Bejtlich (Author)

Book Description

ISBN-10: 0321246772 | ISBN-13: 978-0321246776 | Publication Date: July 22, 2004 | Edition: 1

"The book you are about to read will arm you with the knowledge you need to defend your network from attackers—both the obvious and the not so obvious.... If you are new to network security, don't put this book back on the shelf! This is a great book for beginners and I wish I had access to it many years ago. If you've learned the basics of TCP/IP protocols and run an open source or commercial IDS, you may be asking 'What's next?' If so, this book is for you."

         —Ron Gula, founder and CTO, Tenable Network Security, from the Foreword

"Richard Bejtlich has a good perspective on Internet security—one that is orderly and practical at the same time. He keeps readers grounded and addresses the fundamentals in an accessible way."

         —Marcus Ranum, TruSecure

"This book is not about security or network monitoring: It's about both, and in reality these are two aspects of the same problem. You can easily find people who are security experts or network monitors, but this book explains how to master both topics."

         —Luca Deri, ntop.org

"This book will enable security professionals of all skill sets to improve their understanding of what it takes to set up, maintain, and utilize a successful network intrusion detection strategy."

         —Kirby Kuehl, Cisco Systems

Every network can be compromised. There are too many systems, offering too many services, running too many flawed applications. No amount of careful coding, patch management, or access control can keep out every attacker. If prevention eventually fails, how do you prepare for the intrusions that will eventually happen?

Network security monitoring (NSM) equips security staff to deal with the inevitable consequences of too few resources and too many responsibilities. NSM collects the data needed to generate better assessment, detection, and response processes—resulting in decreased impact from unauthorized activities.

In The Tao of Network Security Monitoring, Richard Bejtlich explores the products, people, and processes that implement the NSM model. By focusing on case studies and the application of open source tools, he helps you gain hands-on knowledge of how to better defend networks and how to mitigate damage from security incidents.

Inside, you will find in-depth information on the following areas.

• The NSM operational framework and deployment considerations.
• How to use a variety of open-source tools—including Sguil, Argus, and Ethereal—to mine network traffic for full content, session, statistical, and alert data.
• Best practices for conducting emergency NSM in an incident response scenario, evaluating monitoring vendors, and deploying an NSM architecture.
• Developing and applying knowledge of weapons, tactics, telecommunications, system administration, scripting, and programming for NSM.
• The best tools for generating arbitrary packets, exploiting flaws, manipulating traffic, and conducting reconnaissance.

Whether you are new to network intrusion detection and incident response, or a computer-security veteran, this book will enable you to quickly develop and apply the skills needed to detect, prevent, and respond to new and emerging threats.

Editorial Reviews

About the Author

Richard Bejtlich is founder of TaoSecurity, a company that helps clients detect, contain, and remediate intrusions using Network Security Monitoring (NSM) principles. He was formerly a principal consultant at Foundstone--performing incident response, emergency NSM, and security research and training--and created NSM operations for ManTech International Corporation and Ball Aerospace & Technologies Corporation. For three years, Bejtlich defended U.S. information assets as a captain in the Air Force Computer Emergency Response Team (AFCERT). Formally trained as an intelligence officer, he is a graduate of Harvard University and of the U.S. Air Force Academy. He has authored or coauthored several security books, including The Tao of Network Security Monitoring (Addison-Wesley, 2004).

Excerpt. © Reprinted by permission. All rights reserved.

Welcome to The Tao of Network Security Monitoring: Beyond Intrusion Detection. The goal of this book is to help you better prepare your enterprise for the intrusions it will suffer. Notice the term “will.” Once you accept that your organization will be compromised, you begin to look at your situation differently. If you’ve actually worked through an intrusion—a real compromise, not a simple Web page defacement—you’ll realize the security principles and systems outlined here are both necessary and relevant.

This book is about preparation for compromise, but it’s not a book about preventing compromise. Three words sum up my attitude toward stopping intruders: prevention eventually fails. Every single network can be compromised, either by an external attacker or by a rogue insider. Intruders exploit flawed software, misconfigured applications, and exposed services. For every corporate defender, there are thousands of attackers, enumerating millions of potential targets. While you might be able to prevent some intrusions by applying patches, managing configurations, and controlling access, you can’t prevail forever. Believing only in prevention is like thinking you’ll never experience an automobile accident. Of course you should drive defensively, but it makes sense to buy insurance and know how to deal with the consequences of a collision.

Once your security is breached, everyone will ask the same question: now what? Answering this question has cost companies hundreds of thousands of dollars in incident response and computer forensics fees. I hope this book will reduce the investigative workload of your computer security incident response team (CSIRT) by posturing your organization for incident response success. If you deploy the monitoring infrastructure advocated here, your CSIRT will be better equipped to scope the extent of an intrusion, assess its impact, and propose efficient, effective remediation steps. The intruder will spend less time stealing your secrets, damaging your reputation, and abusing your resources. If you’re fortunate and collect the right information in a forensically sound manner, you might provide the evidence needed to put an intruder in jail.

Audience

This book is for security professionals of all skill levels and inclinations. The primary audience includes network security architects looking for ways to improve their understanding of their network security posture. My goal is to provide tools and techniques to increase visibility and comprehension of network traffic. If you feel let down by your network-based intrusion detection system (NIDS), this book is definitely for you. I explain why most NIDS deployments fail and how you can augment existing NIDS with open source tools.

Because this book focuses on open source tools, it is more likely to be accepted in smaller, less bureaucratic organizations that don’t mandate the use of commercial software. Furthermore, large organizations with immense bandwidth usage might find some open source tools aren’t built to handle outrageous traffic loads. I’m not convinced the majority of Internet-enabled organizations are using connections larger than T-3 lines, however. While every tool and technique hasn’t been stress-tested on high-bandwidth links, I’m confident the material in this book applies to a great majority of users and networks.

If you’re a network security analyst, this book is also for you. I wrote this book as an analyst, for other analysts. This means I concentrate on interpreting traffic, not explaining how to install and configure every single tool from source code. For example, many books on “intrusion detection” describe the Transmission Control Protocol/Internet Protocol (TCP/IP) suite and how to set up the Snort open source IDS engine with the Analysis Console for Intrusion Databases (ACID) interface. These books seldom go further because they soon encounter inherent investigative limitations that restrict the usefulness of their tools. Since my analytical techniques do not rely on a single product, I can take network-based analysis to the next level. I also limit discussion of odd packet header features, since real intrusions do not hinge on the presence of a weird TCP flag being set. The tools and techniques in this book concentrate on giving analysts the information they need to assess intrusions and make decisions, not just identify mildly entertaining reconnaissance patterns.

This book strives to not repeat material found elsewhere. You will not read how to install Snort or run Nmap. I suggest you refer to the recommended reading list in the next section if you hunger for that knowledge. I introduce tools and techniques overlooked by most authors, like the material on protocol anomaly detection by Brian Hernacki, and explain how you can use them to your advantage.

Technical managers will appreciate sections on best practices, training, and personnel issues. All the technology in the world is worthless if the staff manning it doesn’t understand their roles, responsibilities, and escalation procedures. Managers will also develop an intuition for the sorts of information a monitoring process or product should provide. Many vendors sell services and products named with combinations of the terms “network,” “security,” and “monitoring.” This book creates a specific definition for network security monitoring (NSM), built on a historical and operational foundation.

Prerequisites

I’ve tried to avoid duplicating material presented elsewhere, so I hope readers lacking prerequisite knowledge take to heart the following reading suggestions. I highly recommend reading the following three books prior to this one. If you’ve got the necessary background, consider these titles as references.

• Internet Site Security, by Erik Schetina, Ken Green, and Jacob Carlson (Boston, MA: Addison-Wesley, 2002). This is an excellent “security 101” book. If you need to start from the ground floor, this book is a great beginning.


• Counter Hack: A Step-by-Step Guide to Computer Attacks and Effective Defenses, by Ed Skoudis (Upper Saddle River, NJ: Prentice Hall PTR, 2001). Counter Hack offers the best single-chapter introductions to TCP/IP, Microsoft Windows, UNIX, and security issues available.


• Hacking Exposed: Network Security Secrets and Solutions, 4th ed., by Stuart McClure, Joel Scambray, and George Kurtz (New York: McGraw-Hill, 2003). Hacking Exposed explores the capabilities and intentions of digital threats. By knowing how to compromise computers, you’ll understand the sorts of attacks network security monitoring practitioners will encounter.

If you need an introduction to intrusion detection theory, I recommend the following book:

• Intrusion Detection, by Rebecca Gurley Bace (Indianapolis, IN: New Riders, 2000). While not strictly needed to understand the concepts in this book, Intrusion Detection provides the history and mental lineage of IDS technology. As The Tao of Network Security Monitoring focuses on network-based tactics, you can turn to Intrusion Detection for insight on host-based detection or the merits of signature- or anomaly-based IDS.

It helps to have a good understanding of TCP/IP beyond that presented in the aforementioned titles. The following are a few of my favorite books on TCP/IP.

• Internet Core Protocols: The Definitive Guide, by Eric A. Hall (Cambridge, MA: O’Reilly, 2000). Many people consider Richard Stevens’ TCP/IP Illustrated Volume 1: The Protocols (Reading, MA: Addison-Wesley, 1994) to be the best explanation of TCP/IP. I think Eric Hall’s more recent book is better suited for modern network traffic analysts.


• Network Analysis and Troubleshooting, by J. Scott Haugdahl (Boston, MA: Addison-Wesley, 2000). Troubleshooting books tend to offer the more interesting explanations of protocols in action. Scott Haugdahl works his way up the seven layers of the Open Systems Interconnect (OSI) model, using packet traces and case studies.


• Troubleshooting Campus Networks: Practical Analysis of Cisco and LAN Protocols, by Priscilla Oppenheimer and Joseph Bardwell (Indianapolis, IN: Wiley, 2002). This title is considerably broader in scope than Scott Haugdahl’s work, with coverage of virtual local area networks (VLANs), routing protocols, and wide area network (WAN) protocols like Asynchronous Transfer Mode (ATM).

One other book deserves mention, but I request you forgive a small amount of self-promotion. The Tao of Network Security Monitoring is primarily about detecting incidents through network-based means. In some senses it is also an incident response book. Effective incident response, however, reaches far beyond network-based evidence. To learn more about host-based data, such as file systems and memory dumps, I recommend Real Digital Forensics (Boston, MA: Addison-Wesley, 2005). I wrote the network monitoring sections of the book, and coauthors Keith Jones and Curtis Rose did the host- and memory-level forensics. If you’d like to see the big picture for incident response, read Real Digital Forensics.

A Note on Operating Systems

All of the tools I discuss in this book run on the FreeBSD (http://www.freebsd.org) operating system. FreeBSD is a UNIX-like, open source environment well suited for building network security monitoring platforms. If you’re familiar with Linux or any other Berkeley Software Distribution (OpenBSD or NetBSD), you’ll have no trouble with FreeBSD. I strongly recommend running NSM tools on UNIX-like platforms like the BSDs and Linux.

You might consider trying a live CD-ROM FreeBSD distribution prior to committing a hard drive to install...

Product Details

• Paperback: 832 pages
• Publisher: Addison-Wesley Professional; 1 edition (July 22, 2004)
• Language: English
• ISBN-10: 0321246772
• ISBN-13: 978-0321246776
• Product Dimensions: 9.3 x 7 x 1.7 inches
• Shipping Weight: 2.6 pounds (View shipping rates and policies)
• Average Customer Review: 4.8 out of 5 stars  See all reviews (23 customer reviews)
• Amazon Best Sellers Rank: #128,789 in Books (See Top 100 in Books)
  • #100 in Books > Computers & Technology > Certification > CompTIA

More About the Author

Richard Bejtlich is Chief Security Officer and Security Services Architect for MANDIANT. He was previously Director of Incident Response for General Electric, where he built and led the 40-member GE Computer Incident Response Team (GE-CIRT). Prior to GE, Richard operated TaoSecurity LLC as an independent consultant, protected national security interests for ManTech Corporation's Computer Forensics and Intrusion Analysis division, investigated intrusions as part of Foundstone's incident response team, and monitored client networks for Ball Corporation. Richard began his digital security career as a military intelligence officer at the Air Force Computer Emergency Response Team (AFCERT), Air Force Information Warfare Center (AFIWC), and Air Intelligence Agency (AIA). Richard is a graduate of Harvard University and the United States Air Force Academy. He wrote "The Tao of Network Security Monitoring" and "Extrusion Detection," and co-authored "Real Digital Forensics." He also writes for his blog (taosecurity.blogspot.com) and teaches for Black Hat.

Customer Reviews

Most Helpful Customer Reviews

33 of 36 people found the following review helpful:

5.0 out of 5 stars Superb and exclusive security book!, August 4, 2004

By 

Dr Anton Chuvakin "Dr. Anton Chuvakin" (CA, USA) - See all my reviews

This review is from: The Tao of Network Security Monitoring: Beyond Intrusion Detection (Paperback)

Here is a really cool security book, that made me lose half a nigh sleep when I first got it. Richard Bejtlich "Tao of Network Security Monitoring" ("Tao of NSM") covers the process, tools and analysis techniques for monitoring your network using intrusion detection, session data, traffic statistical information and other data. Here are some of the book highlights.

The book starts from a really exciting and fun background on security, risk and the need to monitor networks and systems. Topics such as the classic "threat x vulnerability x value = risk" formula to threat modeling and limitation of attack prevention technologies are included. A nice thing on the process side is the "assess -> protect -> detect -> respond" loop, that defines a security process for an organization on a high level. Threat analysis material seems to have military origin, but is enlightening for other types of organizations as well.

NSM is introduced as being 'beyond IDS' with some coverage on why IDS deployments fail and what else is needed (NSM process and tools, that is).

A great and rarely appreciated idea expressed in the book is that the intruders are often smarter than defenders. It presents a stark contrast to all this "staying ahead of the hackers", which makes no sense in many cases as the attackers are in fact far ahead. NSM approach will indeed work against the advanced attackers, albeit a high resource cost to the defending organization. Such 'worst case' scenario preparations are extremely rare in other security books. Detecting such intruder is covered during their five phases of compromise (from reconnaissance to using/abusing the system).

Another gem is an idea of a "defensible network"; not 'secure' or 'protected', but defensible. 'Defensible network' can be watched, is configured to limit possible intruder actions, can be kept up to date and runs only the minimum necessary services, that assures that if bad things happen there, they can be handled effectively.

I also liked how the tools are covered in the book. It is not a tool manual rephrased, but rather the whole tool use context related to the rest of the NSM. While the paradigm 'products perform collection, people perform analysis' might be faulty as the products are getting smarter, having training analysts still is one of the best investments in security. On the process side, the book covers complete analyst training. People are indeed the critical component of NSM, since most of the decision-making relies on trained analysts and their investigation, classification and escalation of alerts.

A chapter on netflow and other types of session/connectivity data presents considerable interest to those monitoring networks. Example case studies show how such data helped identify intrusion action that did not directly product IDS alerts. Same applies to traffic visualization and statistical tools that enrich the IDS data and can sometimes provide early anomaly indications as well.

Of course, NSM event-driven analysis is centered on Sguil - a new GUI frontend to NIDS, session and other context data, facilitating easy and effective event classification and escalation (if needed).

Emergency NSM vs ongoing monitoring NSM procedures are also covered in the book. Even if the organization does not maintain an ongoing security monitoring program, it can still benefit from NSM that is deployed after a suspected intrusion.

Attacks against NSM processes and technologies also fill dedicated section. Such attacks include intruder tools as well as attacks against the human (such overwhelming the analysts) and process components of the NSM.

Overall, the book is a required reading for any security professional and those wishing to become one. It helps to broaden the horizons of seasoned professionals as well as educate the beginners in monitoring techniques. While value of NSM as an approach can be debated in modern organizations where tuned sensors and skilled analysts are an exception rather than the rule, the book is a superb security resource even for those who do not choose to implement NSM at the moment.

Anton Chuvakin, Ph.D., GCIA, GCIH is a Senior Security Analyst with a major security information management company. He is the author of the book "Security Warrior" (O'Reilly, 2004) and a contributor to "Known Your Enemy II (AWL, 2004). His areas of infosec expertise include intrusion detection, UNIX security, forensics, honeypots, etc. In his spare time, he maintains his security portal http://www.info-secure.org

13 of 13 people found the following review helpful:

5.0 out of 5 stars An excellent and comprehensive security book, November 12, 2004

By 

Johannes Faustus - See all my reviews

This review is from: The Tao of Network Security Monitoring: Beyond Intrusion Detection (Paperback)

Richard Bejtlich hits one out of the park with this terrific book. In one stroke, he moves the art and science of intrusion detection out of the little leagues and into the majors. If you've already run through articles and books with advice like "just load SNORT and start tuning", this book will shift you to an all-star level in which thousands of machines across enterprise networks can be monitored and protected.

Network security monitoring (NSM) is the discipline of collecting and interpreting detailed network traffic to find and foil attackers. Although it may seem like Intrusion Detection (and IDSs), the relationship between IDSs and NSM is like that between Bonzo the chimp and King Kong. Almost anybody could handle a chimp for a few hours - or you'd think so from watching the movies - but bringing King Kong into your neighborhood means you really have to know what you're doing. He'll take a lot of feeding and special care. On the other hand, he does much more than Bonzo can to protect your assets. Network security monitoring is the King Kong of intrusion detection techniques.

The author presents detailed information on a large variety of network traffic capture and analysis tools, techniques, and topologies. Nearly all are public domain and open source. The few exceptions are tools specialized for industry-dominating Cisco and its proprietary formats and protocols. A few hours on the Internet with this book in hand can give you just about all the tools needed to follow his examples and to build your own network security monitoring environment.

Basic network activity capture is addressed through packages like the fundamental libpcap libraries, and the tools Tcpdump, Tethereal, Ethereal, and Snort (in its packet-capture mode). Tools for converting, combining, and subsetting captured data receive equal attention, with working examples based on editcap, mergecap, tcpslice, the Berkeley packet filter (BPF) language, tcpflow, ngrep and others. GUI tools are touched on as well, including EtherApe and NetDude. For the more advanced topic of session data or "flow" capture (using the Cisco NetFlow data format), there are equally-detailed discussions of the Flow-Tools package, the Argus analysis tools, tcptrace, and others.

Statistical reporting and analysis gets a chapter, while alert processing (the classic IDS functions of Snort) get two, covering Bro, Prelude, and Sguil. (Although the book mentions Snort briefly, it assumes you have access to sufficient information to load and use Snort without assistance.) Much of the remainder of the book addresses the practical issues of installing, operating, and administering network security monitoring in the environment of an enterprise or Internet service provider.

It's refreshing that the software tools are not just mentioned, they are shown in operation in several scenarios each. The reader can see why they are important to the craft of network security monitoring, and can follow the examples on their own computer once the tools are installed. The author's style is not quite a tutorial, but it's easy to learn from him.

Most striking, perhaps, is the author's focus on completely professional installation and operation of this sensitive security function. He talks about network topologies and their effect on sensor placement. He provides alternative designs for the collection of data and for its analysis, usually on separate workstations. His stated experience is on large and very busy networks, so he addresses some difficult techniques (such as merging data from separate sensors to simulate a real-time data flow on a single machine) that are valuable and often mandatory in distributed enterprises. At the same time, his advice supports smaller networks and more limited security goals - you just have to pick and choose the items you need from the very large smorgasbord he presents.

So impressive is the technical detail, you could forgive it for being less than polished. But the writer is not just competent, he is entertaining and enjoyable to read. Between Bejtlich's skills and those of the editors, this book has no bizarre jumps of topic, no dead space, none of the clanging infelicities and groaners that haunt most of the technical books we read each month.

We should be clear about this book's audience: it is not an executive overview or a manager's guide. This is a manual for practitioners. It is pitched toward those who are comfortable purging a desktop machine and converting it into a single-purpose network sensor, those who can download source code and compile tools in multiple operating systems, those who will find it worthwhile to put their hands on and their hearts into a challenging and open-ended endeavor. But for those whom it suits, this will be an indispensable guide, the complete play-book of a fascinating new security specialty.

10 of 10 people found the following review helpful:

5.0 out of 5 stars Great book - covers topic in detail, January 19, 2005

By 

James E. Affeld (Seattle, WA United States) - See all my reviews
(REAL NAME)   

This review is from: The Tao of Network Security Monitoring: Beyond Intrusion Detection (Paperback)

This is a great book. With most geek books, I browse and grab what I need. With this one, I even read the apendices!

At first, the author's tone put me off. He spends the introductory chapters talking about the "Way" of Network Security Monitoring, (capitalized) and how it's much better than other approaches. It felt a little like, "My Burping Crane Kung-Fu will defeat your Shining Fist techniques!" I really didn't see much difference between what he was talking about and other approaches. I admit to being much newer to this discipline than the author, and he has an impressive appendix on the intellectual history of intrusion detection (uncapitalized). So it may be that the lessons he advocates have already been internalized; my exposure may have been to a field that has already moved up to his standard. But I have a hard time imagining that intrusion analysts have ever been satisfied with a single approach with no correlation. As I understand what he means by upper-case NSM, it's basically the efficient use of multiple techniques to detect intrusions. I can't see trying to argue the contrary position.

Ah, but then we get to the good stuff. He goes through the major types of indicators and the means of reviewing them. He covers the use of a number of important tools, but doesn't rehash what is better covered elsewhere. For example, he doesn't bother covering Snort, because there are plenty of books on Snort already. If you are reading the book, it's almost a certainty that you are familiar with Snort. Good call to skip over that. Instead, he covers some other tools that might be useful in the same area. He also refers to tons of other books. I made a lengthy wish-list based on his recommendations and they've been good. (He also reviews exhaustively here on Amazon). So this book is like the first stone in an avalanche- it triggers the acquisition of many other books.

The book provided many 'light bulb' moments. For example, he talks about giving up on source-based focus. In a world where a DDoS attack is currently using 23,000 separate bots, we may exhaust our resources tracking low-value drones. So focus on the targets they are after: light-bulb! In spite of my earlier resistance, I was soon going through it as eagerly as I did with the Patrick O'Brian Aubrey/Maturin novels. It's fun to read such clear, authoritative writing.

One quibble - he trashes the SANS intrusion detection course, which I took and thought was terrific. He has taught the class, and considered the course material out of date. Maybe they have updated, but his book didn't contradict anything in the course as I took it 1.5 years ago.