45 of 45 people found the following review helpful
on August 4, 2007
Let me start by stating the obvious: this is a patterns book about the organisation of tests and the workings of the xUnit family of unit testing frameworks. It is _not_ a book about Test Driven Development, although there is material that is pertinent to that. Given that the use of JUnit and TDD is pretty intertwined in the minds of many Java developers, it's worth making this distinction, so you know what sort of book you're getting. Speaking of JUnit, most of the code examples uses Java, although there are some examples in C#, VB and Ruby.
Like Martin Fowler's Patterns of Enterprise Application Architecture, the book is split into two main sections, a narrative that weaves together a lot of the patterns and strategies, and then a catalogue of individual patterns. Between the two, there is a catalogue of 'test smells', similar to the 'code smells' discussed by Fowler in Refactoring, which I would suggest can be read profitably with the narrative section, rather than used as reference material.
There are a lot of patterns here on the mechanics of xUnit, such as 'Test Runner', 'Garbage-Collected Teardown' and 'Named Test Suite'. I was a bit confused about who this material is aimed at -- maybe someone looking at porting xUnit to a new programming language would find it useful, but a lot of it is fairly obvious to anyone who's used an xUnit in a non-trivial fashion (and certainly, if you haven't done so, this book is not a format that makes for a good introduction), or requires playing against xUnit's strengths (e.g. not putting setup and teardown code in their eponymous methods), although there is good reason for doing so in some of the examples provided, such as databases.
Beyond this, there is some good stuff on design-for-testability patterns (e.g. dependency injection versus dependency lookup), value patterns to replace magic constants, custom assertions and custom creation and other utility methods to make the intent of tests more clear. This material, along with the test smells chapter, is where the real value of the book lies. It encourages the application of the same software engineering principles you would apply to your applications (encapsulation, intent-revealing names, Don't Repeat Yourself) as you would to your testing code, something that's surprisingly easy to overlook, at least in my experience.
Also, the material on 'Test Doubles' (mocks, stubs, dummies and their ilk) is extremely useful. It touches on designing with mocks only superficially, but it does provide a helpful taxonomy of what different classes of doubles do. Now, if only everyone would standardise on this nomenclature, it would make life a lot easier. I suggest we brandish this enormous book threateningly at anyone who refuses to toe the line, and that should do the trick.
Because, boy, this book is big (about 900 pages). To be honest, it's too big. I rarely complain about getting too much book for my money, but the likes of GoF, PoEAA and PoSA 1 manage to come in between 400-500ish pages, so there's no reason XTP couldn't. The advantage is that the patterns in the catalogue, which take up most of the space, stand alone, without requiring too much flicking backwards and forwards between patterns.
The disadvantage is that there is a lot of repetition, so unlike the three design patterns books I mentioned above, which I suspect most people read cover to cover (or maybe that was just me and I'm a complete freak), I would suggest only dipping into the catalogue as necessary. For instance, how much difference is there between the 'Testcase Class per Class', 'Testcase Class per Feature' and the 'Testcase Class per Fixture' patterns? Not a lot, as you might expect.
I definitely liked this book. I would have liked it even more if it came in at about half its size and I would have preferred more emphasis on test and assertion organisation than the mechanics of the xUnit framework, but maybe that would have been a different type of book to the one Gerard Meszaros intended. This is nonetheless a must buy for anyone who cares about unit testing.
15 of 16 people found the following review helpful
on June 16, 2007
We went over 2,000 unit tests this past week during Iteration 72 on our Agile project. Of course, over the course of the last 18-24 months we have removed some tests, and in many cases, refactored the existing tests many times. We also have been learning a whole lot about TDD and the actual domain that we are building and testing. As we were doing this, we were implicitly discovering Test Smells, and discovering test automation patterns. The value in establishing patterns, and more precisely a pattern language in a particular domain are substantial. It's not so much that the "collector" of patterns is defining something new (some often mistakenly criticize pattern books in that regard) that you didn't know, but defining a shared terminology of our practices that we keep doing over and over. To that end, the patterns themselves not only define a shared vocabulary but serve other functions, not the least of which is learning from others. An obvious example of this is Martin's PEAA collection of patterns that enables us to say things like PageController or Lazy Load or TableDataGateway and we all know what it means. In fact, when I am talking about Interaction versus State/Behavior type of testing on CB, and others here use much of this terminology, I am in fact, talking about patterns like TestDoubles and MockObjects, among others.
When I became aware that Gerard Meszaros ' xUnit Test Patterns book was going to ship Friday, I ordered it for overnight delivery on Saturday. I read well over 200 pages yesterday pretty much at one sitting, contented with a book that will change the face of the software industry, just as JUnit and all the other xUnit family have fundamentally altered software development for the better. Its definitely a big book at 944 pages, but it's not a book of excess, unnecessary pages. Rather it shows how hard it is to write defect-free software and the depth of the work that people are putting into this endeavor. The book uses Java as the language which obviously is no hardship to the C# programmer. Like most of the sound practices that have been evolving in the last ten years, this work has been evolving out of the terrific Java community.
Just like their are Code Smells, there are Test Smells, and writing good test code is just as hard and as worthy as writing good production code. Meszaros categorizes Test Smells into ProjectSmells, BehaviorSmells, and CodeSmells. Particularly interesting is his discussion in this regard to the commercial "record and playback" test automation products that have given test automation a bad name in many circles with their tendency to create FragileTests particularly with regard to Interface Sensitivity. Like many others, we were drawn in, and spent and wasted thousands of dollars with a vendor and exhibiting extreme Interface Sensitivity with the user interface. Their interface was not only unable to "pick up" most of the controls we use but even minor changes to the interface can cause tests to fail, even in circumstances in which a human user would say the test should pass. This only goes to support the notion many of us have talked about here about factoring a UI into MVP or MVC and not having logic in the "presentation."
Meszaros goes onto to provide very valuable discussions of Goals of Test Automation, Philosophies of Test Automation, and a Roadmap to Test Automation. We talk about things like Tests as Specification, also known as Executable Specification: "If we are doing test-driven development or test-first development, the tests give us a way to capture what the SUT should be doing before we start implementing it. They give us a way to specify the behavior in various scenarios captured in a form that we can then execute (essentially an "executable specification".) To ensure we are "building the right software", we must ensure that our tests reflect how the SUT will actually be used." We also talk about Tests as Documentation.
The main part of the book, of course, is the catalog of the patterns. Meszaros has provided a tremendous service to our community by not only cataloging and naming much of what we do, but also providing excellent discussions of why and how we do those things. I think, over time, this will be regarded as a seminal work in Software Development.
17 of 19 people found the following review helpful
on June 7, 2007
Writing good tests is not easy. Different developers have come up with many techniques to improve tests. You might have used Backdoor Manipulation to make tests faster or used Custom Assertions to make an Obscure Test more readable. Sometimes a failing test caused you to play Assertion Roulette while you were trying to figure out who the Mystery Guest in this test is and what role he plays.
Gerard documented these (and many more) smells and patterns to help you write better tests. If you have written tests, you have probably used some of them, but even then looking at the patterns described in this book will help you tune your technique. For example Custom Assertions should always take the actual and expected values as parameters, so assertMagicObjectsAreEqual(actual, expected) is good, assertEverythingIsAlright(actual) is bad. And now you have a name for this technique, which makes it easier to explain to your fellow developers what you are doing. At 800+ pages you are bound to find plenty of new techniques as well.
If you are worried that this is another work by the pattern weenies, rest assured. The book follows the very simple and pragmatic "How It Works" and "When to Use It" format also used by Martin Fowlers' Patterns of Enterprise Application Architecture.
6 of 6 people found the following review helpful
By now, the concept of "patterns" in program design is pretty well accepted. And the concept of test-driven development has a solid foundation also. But are there certain "patterns" to building and running those tests? The answer is yes, and the book that covers it is xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros. If you use any of the xUnit software in your development efforts, you need to have this book...
Part 1 - The Narratives: A Brief Tour; Test Smells; Goals of Test Automation; Philosophy of Test Automation; Principles of Test Automation; Test Automation Strategy; xUnit Basics; Transient Fixture Management; Persistent Fixture Management; Result Verification; Using Test Doubles; Organizing Our Tests; Testing with Databases; A Roadmap to Effective Test Automation
Part 2 - The Test Smells: Code Smells; Behavior Smells; Project Smells
Part 3 - The Patterns: Test Strategy Patterns; xUnit Basics Patterns; Fixture Setup Patterns; Result Verification Patterns; Fixture Teardown Patterns; Test Double Patterns; Test Organization Patterns; Database Patterns; Design-for-Testability Patterns; Value Patterns
Part 4 - Appendixes: Test Refactorings; xUnit Terminology; xUnit Family Members; Tools; Goals and Principles; Smells, Aliases, and Causes; Patterns, Aliases, and Variations
Glossary; References; Index
Most of the books that cover xUnit software do so from the perspective of a technical manual. Everything is geared to writing the actual code for the test. Meszaros takes a different tack. He covers more of the "why" behind test writing in xUnit, as well as the basic patterns and principles you should be aware of when you're putting together your tests. People new to xUnit will throw together tests without much thought as to the structure and robustness of that script. Meszaros maintains that much of the same care that goes into writing and designing programs should also go into the test scripts. Patterns such as In-line Setup, Chained Tests, State Verification, and many others can adjust your whole mindset towards what makes a solid and maintainable test script that will serve you well both now and down the road when you have to make changes to the program (and add more scripts to your test suite). The book is set up such that you can scan for basic ideas, and then go back to specific patterns for more information as the situations and needs arise. With the use of both actual code and UML diagrams, it's very easy to catch the gist of each pattern, as well as seeing how it would actually be implemented. Very good stuff here...
If you practice test-driven development (and you should), you have no doubt worked with your particular xUnit variant. This book is the next step in your learning, and it will make you a much better developer and tester...
5 of 5 people found the following review helpful
on June 15, 2007
Anyone who is serious about automated testing on their project should go out and read this book ASAP. Like Patterns of Enterprise Application Architecture [Fowler] and Design Patterns [GoF] before it, Gerard Meszaros has put together a very through (if not large... the book is twice the size of PoEAA)reference and catalog of test patterns to make testing more efficient.
The book is rather voluminous, divided up into 3 parts. The first part, "The Narratives", is a refreshing lecture on the philosophy of test automation, a brief on test smells, test organization, and tests with external resources. Essentially an overview of several concepts involved in test automation.
Part 2, aptly titled "The Test Smells," covers some of the common test smells, such as "Hard-to-Test Code", "Fragile Test", and "Slow Test" just to name a few. These are organized in the format Symptoms, Impact, Troubleshooting Advice, and Causes, with the Causes section broken down by cause, with Symptoms, Root Cause, and Possible Solution listed for each. I was impressed with this section as I already recognized some of the smells, as well as having already applied some of the solutions listed in the past.
Part 3 is "The Patterns", which follows the form of most patten catalogs out there. This is the largest section as it makes up more than half of the book, covering a lot of a ground. The patterns are given in a similar form to the ones in PoEAA, and provide easy to follow examples and in depth analysis. I was happy to find myself applying two of the patterns in the section already on a current project.
Overall, great book. Highly Recommended. ;)
4 of 4 people found the following review helpful
on December 23, 2007
I've been familiar with agile concepts of automated unit testing (AUT) and test-driven development (TDD) for awhile now. In the past few years I've made several attempts at incorporating AUT and TDD into my own personal workflow, but each attempt soon resulted in my abandoning the whole idea. The testing effort quickly outweighed the benefits. I've believed in the ideal of TDD, but I didn't see quite how to pull it off.
Then I bought XUnit Test Patterns by Gerard Meszaros. Wow! Finally the issues I've struggled with are being addressed. Okay, I must admit I'm not very plugged in to the online software development community, and I'm sure these issues have been discussed before. But this book looks special. I sense it's giving voice to these issues in a big way that's introducing many developers to these ideas for the first time. After all, it had to take time for this kind of book to be written. Time for the patterns to be developed through hard and frustrating work.
Rarely have I bought a thick book on software development and eagerly read every single word from cover to cover. But I have with this book. And I know I'll soon do it again. I'm even tempted to also purchase the PDF version of the book, just so I can reference it wherever I happen to be.
It's not the final word on AUT, but it has me embracing the ideal of TDD once more. The company I work for develops a huge OO-based enterprise software system with no automated tests. As Meszaros explains, this kind of legacy system is the most difficult for incorporating AUT (and daunting for those new to AUT). But at least now I feel like we have a good chance.
3 of 3 people found the following review helpful
on June 23, 2009
xUnit Test Patterns by Gerard Meszaros is a huge book. It is almost 900 pages of patterns to be used for test automation and for unit testing. This book contains a huge amount of useful knowledge for developers and is, without a doubt, the most thorough book on writing well-structured unit tests. The largest drawback of the book is its size and the amount of duplication.
The book consists of three parts. The first part is normal prose introducing the patterns and their contexts. The second part is a catalog of test smells and the last part (which is most of the book) describes the test patterns.
The first part of the book is excellent. It contains a couple of chapters related to test automation that are not specific to xUnit test automation. Chapter two introduces test smells, which I found very helpful. Chapter three contains the goals of test automation. This was one of the clearest descriptions I've seen which answer the question "why do we want to do this in the first place." The next chapters are about philosophy (e.g. TDD), principles and strategy. Chapter second introduces xUnit basics and the rest of the chapters of this part are narratives around the pattern sets (chapters) which are introduces further on in the book. The first part is about 180 pages of the book and is definitively worth reading completely. The individual chapters are short and easy to read.
The second part of the book covers test smells. The test smells are grouped intro three categories: code smells, behavior smells and project smells. These smells are also linked to each other, explaining what the "higher level smell" is caused by. I'm afraid I've seen all the smells in real life in the past and thus this part was very recognizable. Nowadays, I often send these smells to people I work with, since Gerard described them perhaps better than I ever could. The test smells are less than a hundred pages
The third part of the book are the test patterns. They are grouped in 'test pattern categories' with each category containing as little as four and as much as ten patterns. The pattern format Gerard uses is clear and easy to read. The largest drawback is that the pattern format causes quite a lot of duplication, especially across patterns. This is because the book is written so that each pattern can be read independently of others. This is good, but for people who read whole books (me) it causes the book to be somewhat boring to read (even though the content is interesting, just felt bored when reading again that there is one exception to object per test-case implementation of most xUnit frameworks).
The first chapter in part three is test strategy patterns, describing different approaches to testing (and this one is not specific to xUnit tests). The next chapter introduces xUnit frameworks written down in pattern form. Fixture setup is next in which the author describes the advantages and disadvantages for fresh vs shared fixtures and how to set them up. Chapter 21 then dives into the assertion parts of an xUnit framework describing the difference between state and behavior verification. Patterns such as custom assertions are frequently used in well-written test code. Next are the teardown patterns in which the authors somewhat promote the automated teardown (something I've not encountered very often in development). Chapter 23 is probably one of the best known chapters where Gerard categorizes test doubles (mocks/stubs/fakes) and describes when to use which term and why? This terminology is far from standardized and his effort to bring clarity in these terms is much appreciated. Next are test organization patterns such as how and where do you split your test case. Chapter 25 covers working with databases. Chapter 26 some general miscellaneous pattern in design which promote low coupling and thus increase testability. The final chapter relates to values within your tests.
The book also (for if this wasn't enough for you) contains over a hundred pages of appendixes. The refactorings, terminology mapping and glossary are useful.
Gerards book is huge. It contains useful insights related to unit testing and writing clean test code. From that perspective, this book is recommended for everyone serious about writing well-factored unit tests. However, the book also contains relative trivial things and, as mentioned, it contains a huge amount of duplication between different parts. This means that perhaps reading the book from cover to cover might not be the best approach :) Part one and two should definitively be read completely, but the patterns in chapter 3 are better browsed and read when needed or e.g. one per week. Another annoying part in the book, for me, was the way Gerard uses comments. Most comments in the code were not useful and made the code obscure and hard to read at times. Wished he had left them out (though, I guess this relates to coding style a lot).
Conclusion. Definitively worth reading or at least browsing for anyone serious about xUnit and test automation. Because of the duplication and the comments, I'd rate it four out of five stars. Personally I'd rather see a small 100 page book containing the summary and conclusions of what Gerard is talking about. It doesn't exist yet, so this book is, at this moment, the best on this particular subject.
2 of 2 people found the following review helpful
on July 12, 2010
I think the title is a bit misleading, because this book is about a lot more than just writing good test code.
I had to port xUnit to an obscure language, and this book was a constant and invaluable source of inspiration for features taken from various testing frameworks that I would otherwise not have known about.
It's a very authoritative book. It contains a lot of examples, but the format is more on the academic side. This means that, in line with any respectable pattern book, you get a theoretical description of what a certain pattern is about before you can actually see any code, or any examples, which may be pages later. This might put some readers off, and admittedly it could have been made less painful. However, to the author's merit, patterns start with a diagram that illustrates the concept at a glance, which I thought was very helpful.
The book is fairly verbose and there are some mildly irritating repetitions - you will be told about 100 times about nUnit's idiosyncratic definition (and implementation) of the concept of fixture and how the author once managed to make a test suite 50 (fifty! cinquenta! L!) times quicker by using an in-memory database.
To be fair, however, repetitions are probably inevitable in a reference book, especially considering that there is a chunky introductory first part (250 pages) that wraps the main patterns together in a more conversational style.
Presumably due to the author's background, the book is clearly biased towards Java, and so that's how the vast majority of example code is written in, plus you sometimes get rather local terminology or concepts. There is very good coverage of the various Java and .NET frameworks (not just xUnit, also Fit and others), with some Smalltalk and Ruby references as appropriate. C++ is hardly ever mentioned. If you want better C++ coverage, you may want to have a look at Michael Feather's book "Working Effectively with Legacy Code", which I see as complementary to this.
But I probably focus on the negatives. All in all, this book is choke-full of excellent expert advice that I simply wouldn't know where to get from otherwise, and therefore it deserves five stars.
1 of 1 people found the following review helpful
on April 28, 2010
If TDD or its kin survive, it will be largely due to adoption of manageable tactics for testing. This book is unprecedented in supplying principles and conceptual tools hopefully sufficient to permit pragmatic management of test automation. Perhaps Gerard Meszaros will one day be as frequently studied as the Gang of Four.
xUnit Test Patterns is a Martin Fowler Series Book in the pattern tradition, including useful book cover lists: patterns, problem-solution patterns, and even a list of smells. It is built to be a working book --- almost a 1000 page reference divided into 4 parts: Narratives, Test Smells, Patterns, and Appendices.
Although the pattern section constitutes over two-thirds of the book, the concepts of the narrative sections are fundamental. The author's relentless focus on SUT (System Under Test) is heart and soul of the clean definitions and crisp counsel present throughout. The persistence of this notion is key to avoiding confusions between tests for dependent subsystems, and system-subsystem tests. The author's sharp focus permits: (a) Common phrases like 'mock object' to be given a much needed preciseness, and (b) the believability of usefully explaining 'xUnit' patterns through use of JUnit as the main sample language.
The pattern section of the book is spine-marked for major pattern classes like 'Strategy Patterns', and all these pattern classes are clarified by UML diagrams (subcategories, dependency) which precede the forward. Those diagrams are very useful for understanding the conceptual structure of the books content but are easily overlooked. -- grc, 4/27/2010
1 of 1 people found the following review helpful
on March 27, 2009
I'm writing this review quite some time after the book was published, but I get a sense that there are people out there who haven't read or even heard of this book but who probably should.
Most software developers write automated unit tests these days, but how many can elaborate on their testing strategy beyond saying, "we just use JUnit" (for Java) or NUnit (for .NET)? Just like there is a gap between knowing the syntax of an object-oriented programming language and producing good object-oriented designs, there is a gap between knowing NUnit (there isn't really much to know about it) and creating clean, easy-to-maintain tests. This book will help you close that gap.
Many developers become frustrated at some point with their unit tests that break when something changes in the design of the system under test, or when the configuration changes, or when different data are loaded into the database. One of my colleagues says that many developers share the view that writing unit tests isn't fun, skip this activity whenever they can get away with it, and that is a real challenge in the adoption of test-driven development. (This colleague and I currently work for different companies - A.Z.) The answer to this challenge is to write better tests, and this book will teach you how.
The book is thick, but you don't need to read it end to end. It is very well organized and about two-thirds of it is the catalog of patterns, which you can use as a reference. It belongs on the desk of every serious software developer.