Exploiting Software: How to Break Code: How to Break Code 1st Edition

Greg Hoglund has been a pioneer in the area of software security. He is CEO of HBGary, Inc., a leading provider of software security verification services. After writing one of the first network vulnerability scanners (installed in over half of all Fortune 500 companies), he created and documented the first Windows NT-based rootkit, founding rootkit.com in the process. Greg is a frequent speaker at Black Hat, RSA, and other security conferences.

Gary McGraw, Cigital's CTO, is a leading authority on software security. Dr. McGraw is coauthor of the groundbreaking books Building Secure Software and Exploiting Software (both from Addison-Wesley). While consulting for major software producers and consumers, he has published over ninety peer-reviewed technical publications, and functions as principal investigator on grants from DARPA, the National Science Foundation, and NIST's Advanced Technology Program. He serves on the advisory boards of Authentica, Counterpane, and Fortify Software. He is also an advisor to the computer science departments at University of California, Davis, and the University of Virginia, as well as the School of Informatics at Indiana University.

Software security is gaining momentum as security professionals realize that computer security is really all about making software behave. The publication of Building Secure Software in 2001 (Viega and McGraw) unleashed a number of related books that have crystallized software security as a critical field. Already, security professionals, software developers, and business leaders are resonating with the message and asking for more.

Building Secure Software (co-authored by McGraw) is intended for software professionals ranging from developers to managers, and is aimed at helping people develop more secure code. Exploiting Software is useful to the same target audience, but is really intended for security professionals interested in how to find new flaws in software. This book should be of particular interest to security practitioners working to beef up their software security skills, including red teams and ethical hackers.

Exploiting Software is about how to break code. Our intention is to provide a realistic view of the technical issues faced by security professionals. This book is aimed directly toward software security as opposed to network security. As security professionals come to grips with the software security problem, they need to understand how software systems break.

Solutions to each of the problems discussed in Exploiting Software can be found in Building Secure Software. The two books are mirror images of each other.

We believe that software security and application security practitioners are in for a reality check. The problem is that simple and popular approaches being hawked by upstart "application security" vendors as solutions--such as canned black box testing tools--barely scratch the surface. This book aims to cut directly through the hype to the heart of the matter. We need to get real about what we're up against. This book describes exactly that.

What This Book Is About

This book closely examines many real-world software exploits, explaining how and why they work, the attack patterns they are based on, and in some cases how they were discovered. Along the way, this book also shows how to uncover new software vulnerabilities and how to use them to break machines.

Chapter 1 describes why software is the root of the computer security problem. We introduce the trinity of trouble--complexity, extensibility, and connectivity--and describe why the software security problem is growing. We also describe the future of software and its implications for software exploit.

Chapter 2 describes the difference between implementation bugs and architectural flaws. We discuss the problem of securing an open system, and explain why risk management is the only sane approach. Two real-world exploits are introduced: one very simple and one technically complex. At the heart of Chapter 2 is a description of attack patterns. We show how attack patterns fit into the classic network security paradigm and describe the role that attack patterns play in the rest of the book.

The subject of Chapter 3 is reverse engineering. Attackers disassemble, decompile, and deconstruct programs to understand how they work and how they can be made not to. Chapter 3 describes common gray box analysis techniques, including the idea of using a security patch as an attack map. We discuss Interactive Disassembler (IDA), the state-of-the-art tool used by hackers to understand programs. We also discuss in detail how real cracking tools are built and used.

In Chapters 4, 5, 6, and 7, we discuss particular attack examples that provide instances of attack patterns. These examples are marked with an asterisk.

Chapters 4 and 5 cover the two ends of the client-server model. Chapter 4 begins where the book Hacking Exposed McClure et al., 1999 leaves off, discussing trusted input, privilege escalation, injection, path tracing, exploiting trust, and other attack techniques specific to server software. Chapter 5 is about attacking client software using in-band signals, cross-site scripting, and mobile code. The problem of backwash attacks is also introduced. Both chapters are studded with attack patterns and examples of real attacks.

Chapter 6 is about crafting malicious input. It goes far beyond standard-issue "fuzzing" to discuss partition analysis, tracing code, and reversing parser code. Special attention is paid to crafting equivalent requests using alternate encoding techniques. Once again, both real-world example exploits and the attack patterns that inspire them are highlighted throughout.

The whipping boy of software security, the dreaded buffer overflow, is the subject of Chapter 7. This chapter is a highly technical treatment of buffer overflow attacks that leverages the fact that other texts supply the basics. We discuss buffer overflows in embedded systems, database buffer overflows, buffer overflow as targeted against Java, and content-based buffer overflows. Chapter 7 also describes how to find potential buffer overflows of all kinds, including stack overflows, arithmetic errors, format string vulnerabilities, heap overflows, C++ vtables, and multistage trampolines. Payload architecture is covered in detail for a number of platforms, including x86, MIPS, SPARC, and PA-RISC. Advanced techniques such as active armor and the use of trampolines to defeat weak security mechanisms are also covered. Chapter 7 includes a large number of attack patterns.

Chapter 8 is about rootkits--the ultimate apex of software exploit. This is what it means for a machine to be "owned." Chapter 8 centers around code for a real Windows XP rootkit. We cover call hooking, executable redirection, hiding files and processes, network support, and patching binary code. Hardware issues are also discussed in detail, including techniques used in the wild to hide rootkits in EEPROM. A number of advanced rootkit topics top off Chapter 8.

As you can see, Exploiting Software runs the gamut of software risk, from malicious input to stealthy rootkits. Using attack patterns, real code, and example exploits, we clearly demonstrate the techniques that are used every day by real malicious hackers against software.

How to Use This Book

This book is useful to many different kinds of people: network administrators, security consultants, information warriors, developers, and security programmers.

• If you are responsible for a network full of running software, you should read this book to learn the kinds of weaknesses that exist in your system and how they are likely to manifest.
• If you are a security consultant, you should read this book so you can effectively locate, understand, and measure security holes in customer systems.
• If you are involved in offensive information warfare, you should use this book to learn how to penetrate enemy systems through software.
• If you create software for a living, you should read this book to understand how attackers will approach your creation. Today, all developers should be security minded. The knowledge here will arm you with a real understanding of the software security problem.
• If you are a security programmer who knows your way around code, you will love this book.

The primary audience for this book is the security programmer, but there are important lessons here for all computer professionals.

But Isn't This Too Dangerous?

It's important to emphasize that none of the information we discuss here is news to the hacker community. Some of these techniques are as old as the hills. Our real objective is to provide some eye-opening information and up the level of discourse in software security.

Some security experts may worry that revealing the techniques described in this book will encourage more people to try them out. Perhaps this is true, but hackers have always had better lines of communication and information sharing than the good guys. This information needs to be understood and digested by security professionals so that they know the magnitude of the problem and they can begin to address it properly. Shall we grab the bull by the horns or put our head in the sand?

Perhaps this book will shock you. No matter what, it will educate you.

Software security is gaining momentum as security professionals realize that computer security is really all about making software behave. The publication of Building Secure Software in 2001 (Viega and McGraw) unleashed a number of related books that have crystallized software security as a critical field. Already, security professionals, software developers, and business leaders are resonating with the message and asking for more.

Building Secure Software (co-authored by McGraw) is intended for software professionals ranging from developers to managers, and is aimed at helping people develop more secure code. Exploiting Software is useful to the same target audience, but is really intended for security professionals interested in how to find new flaws in software. This book should be of particular interest to security practitioners working to beef up their software security skills, including red teams and ethical hackers.

Exploiting Software is about how to break code. Our intention is to provide a realistic view of the technical issues faced by security professionals. This book is aimed directly toward software security as opposed to network security. As security professionals come to grips with the software security problem, they need to understand how software systems break.

Solutions to each of the problems discussed in Exploiting Software can be found in Building Secure Software. The two books are mirror images of each other.

We believe that software security and application security practitioners are in for a reality check. The problem is that simple and popular approaches being hawked by upstart "application security" vendors as solutions--such as canned black box testing tools--barely scratch the surface. This book aims to cut directly through the hype to the heart of the matter. We need to get real about what we're up against. This book describes exactly that.

What This Book Is About

This book closely examines many real-world software exploits, explaining how and why they work, the attack patterns they are based on, and in some cases how they were discovered. Along the way, this book also shows how to uncover new software vulnerabilities and how to use them to break machines.

Chapter 1 describes why software is the root of the computer security problem. We introduce the trinity of trouble--complexity, extensibility, and connectivity--and describe why the software security problem is growing. We also describe the future of software and its implications for software exploit.

Chapter 2 describes the difference between implementation bugs and architectural flaws. We discuss the problem of securing an open system, and explain why risk management is the only sane approach. Two real-world exploits are introduced: one very simple and one technically complex. At the heart of Chapter 2 is a description of attack patterns. We show how attack patterns fit into the classic network security paradigm and describe the role that attack patterns play in the rest of the book.

The subject of Chapter 3 is reverse engineering. Attackers disassemble, decompile, and deconstruct programs to understand how they work and how they can be made not to. Chapter 3 describes common gray box analysis techniques, including the idea of using a security patch as an attack map. We discuss Interactive Disassembler (IDA), the state-of-the-art tool used by hackers to understand programs. We also discuss in detail how real cracking tools are built and used.

In Chapters 4, 5, 6, and 7, we discuss particular attack examples that provide instances of attack patterns. These examples are marked with an asterisk.

Chapters 4 and 5 cover the two ends of the client-server model. Chapter 4 begins where the book Hacking Exposed McClure et al., 1999 leaves off, discussing trusted input, privilege escalation, injection, path tracing, exploiting trust, and other attack techniques specific to server software. Chapter 5 is about attacking client software using in-band signals, cross-site scripting, and mobile code. The problem of backwash attacks is also introduced. Both chapters are studded with attack patterns and examples of real attacks.

Chapter 6 is about crafting malicious input. It goes far beyond standard-issue "fuzzing" to discuss partition analysis, tracing code, and reversing parser code. Special attention is paid to crafting equivalent requests using alternate encoding techniques. Once again, both real-world example exploits and the attack patterns that inspire them are highlighted throughout.

The whipping boy of software security, the dreaded buffer overflow, is the subject of Chapter 7. This chapter is a highly technical treatment of buffer overflow attacks that leverages the fact that other texts supply the basics. We discuss buffer overflows in embedded systems, database buffer overflows, buffer overflow as targeted against Java, and content-based buffer overflows. Chapter 7 also describes how to find potential buffer overflows of all kinds, including stack overflows, arithmetic errors, format string vulnerabilities, heap overflows, C++ vtables, and multistage trampolines. Payload architecture is covered in detail for a number of platforms, including x86, MIPS, SPARC, and PA-RISC. Advanced techniques such as active armor and the use of trampolines to defeat weak security mechanisms are also covered. Chapter 7 includes a large number of attack patterns.

Chapter 8 is about rootkits--the ultimate apex of software exploit. This is what it means for a machine to be "owned." Chapter 8 centers around code for a real Windows XP rootkit. We cover call hooking, executable redirection, hiding files and processes, network support, and patching binary code. Hardware issues are also discussed in detail, including techniques used in the wild to hide rootkits in EEPROM. A number of advanced rootkit topics top off Chapter 8.

As you can see, Exploiting Software runs the gamut of software risk, from malicious input to stealthy rootkits. Using attack patterns, real code, and example exploits, we clearly demonstrate the techniques that are used every day by real malicious hackers against software.

How to Use This Book

This book is useful to many different kinds of people: network administrators, security consultants, information warriors, developers, and security programmers.

• If you are responsible for a network full of running software, you should read this book to learn the kinds of weaknesses that exist in your system and how they are likely to manifest.
• If you are a security consultant, you should read this book so you can effectively locate, understand, and measure security holes in customer systems.
• If you are involved in offensive information warfare, you should use this book to learn how to penetrate enemy systems through software.
• If you create software for a living, you should read this book to understand how attackers will approach your creation. Today, all developers should be security minded. The knowledge here will arm you with a real understanding of the software security problem.
• If you are a security programmer who knows your way around code, you will love this book.

The primary audience for this book is the security programmer, but there are important lessons here for all computer professionals.

But Isn't This Too Dangerous?

It's important to emphasize that none of the information we discuss here is news to the hacker community. Some of these techniques are as old as the hills. Our real objective is to provide some eye-opening information and up the level of discourse in software security.

Some security experts may worry that revealing the techniques described in this book will encourage more people to try them out. Perhaps this is true, but hackers have always had better lines of communication and information sharing than the good guys. This information needs to be understood and digested by security professionals so that they know the magnitude of the problem and they can begin to address it properly. Shall we grab the bull by the horns or put our head in the sand?

Perhaps this book will shock you. No matter what, it will educate you.