How to Break Software: A Practical Guide to Testing W/CD [Paperback]

James A. Whittaker (Author)

Book Description

ISBN-10: 0201796198 | ISBN-13: 978-0201796193 | Publication Date: May 19, 2002

How to Break Software is a departure from conventional testing in which testers prepare a written test plan and then use it as a script when testing the software. The testing techniques in this book are as flexible as conventional testing is rigid. And flexibility is needed in software projects in which requirements can change, bugs can become features and schedule pressures often force plans to be reassessed. Software testing is not such an exact science that one can determine what to test in advance and then execute the plan and be done with it. Instead of a plan, intelligence, insight, experience and a "nose for where the bugs are hiding" should guide testers. This book helps testers develop this insight. The techniques presented in this book not only allow testers to go off-script, they encourage them to do so. Don't blindly follow a document that may be out of date and that was written before the product was even testable. Instead, use your head! Open your eyes! Think a little, test a little and then think a little more. This book does teach planning, but in an "on- the-fly while you are testing" way. It also encourages automation with many repetitive and complex tasks that require good tools (one such tool is shipped with this book on the companion CD). However, tools are never used as a replacement for intelligence. Testers do the thinking and use tools to collect data and help them explore applications more efficiently and effectively.

Editorial Reviews

From the Back Cover

Practical tutorial on how to actually do testing by presenting numerous "attacks" you can perform to test your software for bugs.

* Practical approach has little or no theory, but shows real ways effectively test software—accessible to beginners and seasoned testers.
* The author is well known and respected as an industry consultant and speaker.
* Uses market leading, and immediately identifiable, software applications as examples to show bugs and techniques.
How to Break Software is a departure from conventional testing in which testers prepare a written test plan and then use it as a script when testing the software. The testing techniques in this book are as flexible as conventional testing is rigid. And flexibility is needed in software projects in which requirements can change, bugs can become features and schedule pressures often force plans to be reassessed. Software testing is not such an exact science that one can determine what to test in advance and then execute the plan and be done with it. Instead of a plan, intelligence, insight, experience and a "nose for where the bugs are hiding" should guide testers. This book helps testers develop this insight. The techniques presented in this book not only allow testers to go off-script, they encourage them to do so. Don't blindly follow a document that may be out of date and that was written before the product was even testable. Instead, use your head! Open your eyes! Think a little, test a little and then think a little more. This book does teach planning, but in an "on- the-fly while you are testing" way. It also encourages automation with many repetitive and complex tasks that require good tools (one such tool is shipped with this book on the companion CD). However, tools are never used as a replacement for intelligence. Testers do the thinking and use tools to collect data and help them explore applications more efficiently and effectively.

James A. Whittaker is a well-known speaker and consultant, as well as seasoned professor.

About the Author

James A. Whittaker is a well-known speaker and consultant, as well as seasoned professor.

Product Details

• Paperback: 208 pages
• Publisher: Addison Wesley (May 19, 2002)
• Language: English
• ISBN-10: 0201796198
• ISBN-13: 978-0201796193
• Product Dimensions: 9.2 x 7.1 x 0.5 inches
• Shipping Weight: 12.8 ounces (View shipping rates and policies)
• Average Customer Review: 4.0 out of 5 stars  See all reviews (19 customer reviews)
• Amazon Best Sellers Rank: #262,832 in Books (See Top 100 in Books)
  • #99 in Books > Computers & Technology > Programming > Software Design, Testing & Engineering > Testing

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

73 of 76 people found the following review helpful:

5.0 out of 5 stars If you really want to learn testing, buy this book., June 22, 2002

By 

James Bach - See all my reviews

This review is from: How to Break Software: A Practical Guide to Testing W/CD (Paperback)

This book is part of the new wave of testing books that challenge not only the conventional wisdom about test process, but also challenge conventional wisdom about how to teach and write about testing. People who prefer testing textbooks that preach paperwork and process will be shocked, shocked, to discover that there are a lot of us who think it's a tester's job to find important bugs fast. We want books that give us strategies for actually finding problems. Paperwork and process help some, but not enough. We need something more. We need test-designer-sits-down-at-the-keyboard know-how.

As a test designer, myself (and a competitor of Whittaker's) I can certainly find things to nitpick about this book. But I won't do that here, because the big picture is far more important. That picture is simply this: if you are confused about what to do to uncover problems in software before it ships, EVEN IF you have no specifications to test from and EVEN IF no one listens when you rant about "quality assurance processes" they should follow, then there are only a few testing books yet published that will help you. This is one of them.

55 of 57 people found the following review helpful:

5.0 out of 5 stars More serious than the title implies - excellent book, May 20, 2002

By 

Mike Tarrani "www.tarrani.com" (Deltona, FL USA) - See all my reviews
(COMMUNITY FORUM 04)    (REAL NAME)   

This review is from: How to Break Software: A Practical Guide to Testing W/CD (Paperback)

Don't let the title or description fool you into thinking this is a book about ad hoc playing with applications with a goal to break them. In reality the book gives a structured approach to finding vulnerabilities in software. These vulnerabilities are weak points commonly found in software, and should be included in any test suite.

The vulnerabilities are classified by a fault model, then the book systematically walks you through the procedures used to attack and break the software. Each vulnerability type is addressed:
User Interface
- inputs and outputs, with 6 attacks for breaking common input flaws and 4 for output flaws.
- data and computation, with 3 attacks against stored data and 3 against computation and feature interaction.

System Interface
- 3 media-based and 3 file-based attacks against the file system.
- how to test the application/operating system interface.

The book also comes with a Windows application that helps you to create the hostile environment with which to 'attack' the software being tested. Therein lies the sophistication of the book, which employs fault injection as a technique. This technique is not commonly used in any but the most advanced testing environments, which raises this book's credibility from ad hoc to a serious approach to software engineering. More importantly, it provides test professionals, especially those who are testing Windows applications, a catalog of common vulnerabilities to address. More importantly, it teaches test professionals to approach parts of the testing process from an exploitation point of view - after all, their job is to break the software.

My initial misgivings about this book vanished as soon as I started reading it, and were replaced by enthusiasm by the time I was finished. This book addresses a niche topic, but deserves a place in every software testing library.

39 of 44 people found the following review helpful:

4.0 out of 5 stars A systematic process for rapid, basic testing of software, November 9, 2002

By 

Charles Ashbacher (Marion, Iowa United States) - See all my reviews
(TOP 500 REVIEWER)    (VINE VOICE)    (HALL OF FAME REVIEWER)   

This review is from: How to Break Software: A Practical Guide to Testing W/CD (Paperback)

If there is an area of software development that needs to be codified and formalized, it is the procedures for testing the software before release. With the exception of software that does only a few tasks, it is not possible to test all possible paths. The number of possible paths expands very quickly so that it is effectively infinite, which means that it is so large that it might as well be infinite. Furthermore, this problem will only get worse as software continues to increase in complexity. Finally, the testing phase of software is relegated to the last step and is often considered to be a menial task by developers. Given these conditions and the general pressure of meeting a release date, it follows that testing is often cut short.
With all of this as a background, it would appear that testing is a hopeless task. That is not the case if the testing is done in a systematic manner, which is what this book will teach you. Whittaker is a computer science professor whose area of expertise is that of testing software. He breaks the process into two broad categories: user interface attacks and system interface attacks. Each of these areas is then split into separate attacks, seventeen for user interface attacks and six for system interface attacks.
The attacks for user interface are:

* Apply inputs that force all the error messages to occur.
* Apply inputs that force the software to establish default values.
* Explore allowable character sets and data types.
* Overflow input buffers.
* Find inputs that may interact and test combinations of their values.
* Repeat the same input or series of inputs numerous times.
* Force different outputs to be generated for each input.
* Force invalid outputs to be generated.
* Force properties of an output to change.
* Force the screen to refresh.
* Apply inputs using a variety of initial conditions.
* Force a data structure to store too many or too few values.
* Investigate alternate ways to modify internal data constraints.
* Experiment with invalid operand and operator combinations.
* Force a function to call itself recursively.
* Force computation results to be too large or too small.
* Find features that share data or interact poorly.

The attacks for system interface are:

* Fill the file system to capacity.
* Force the media to be busy or unavailable.
* Damage the media.
* Assign an invalid file name.
* Vary file access permissions.
* Vary or corrupt file contents.

Each of the attacks is presented using the subsections:

* When to apply this attack.
* What software faults make this attack successful?
* How to determine if this attack exposes failures.
* How to conduct this attack.

This approach leads to a very thorough demonstration of how to perform rigorous software testing in a limited amount of time. If I ever teach a course in software testing, this is what I will use as a text.
The book includes a CD containing two software testing tools, one of which I wish was available when I was developing software. While it is running, you can move a slider to have it bind memory resources and learn the point of memory use where your software performance begins to suffer. This is very useful, and is much easier than trying to load up many other applications.
Software testing is a critical area of development that is still in the process of being codified into patterns for reuse. This book demonstrates many of the currently available strategies and should be read by all members of testing teams.