All large organizations have complex, heterogeneous IT systems. All of them need to integrate their applications to support faster, more accurate business processes and to provide meaningful, consistent management information. All organizations are struggling to achieve this.
One reason for their struggle is that they are caught in the crossfire of an IT vendor war. In one corner is Microsoft Corporation, whose strength is its consistent technical strategy based on the .NET set of software technologies and the Windows operating system. In the other corner, ranged against Microsoft, is a group that includes IBM Corporation, Sun Microsystems, Oracle Corporation, and BEA Systems. This group is focusing its resources on the Java environment. This is a battle about who will rule over middleware technology, a battle about how to implement distributed systems. Given the importance of the subject matter, it is a battle for the hearts and souls of IT for the next decade. Why? Because all large organizations have complex, heterogeneous IT systems that need to be brought together.
Vendor wars are only part of the problem. Scratch the surface of a large IT department and you will see many camps, for example, workstation/departmental server "decentralizers" and mainframe "centralizers." Look from another angle and you will see two kinds of people—"techies" and "modelers." A techy will start a project by deciding what platform and software to use and will eventually get around to the boring bit, which is writing application code. A modeler will design the application with a modeling tool, generate a few programs and a database, and will eventually confront the (to him or her) trivial question of what platform it will run on. Modeling to a techy seems abstract and disconnected from reality. Technical issues to a modeler are tedious, and surely, soon we will be able to generate the application from the model at the press of a button, won't we? One key to developing large distributed systems is to bring these people together.
Computer professionals generally are comfortable with developing applications on a single platform to a well-defined set of requirements. The reason is that the technology is well understood; the modelers know that what they design can be implemented, and the techies know they can make it work. Large distributed systems are not like that. A system designed without consideration for the distributed implementation will not work. Even worse, it will only become apparent that it doesn't work when scaling up to production capacity starts. To add to our woes, we are now considering integrating multiple systems, each of which was a challenge to develop in the first place, and each of which is changing at a different speed, driven ever faster by the business. The notion of a "well-defined set of requirements" is not realistic; requirements will always be changing.
It is our contention that modelers need to know something about technology, techies need to know something about modeling, and while we are about it, vendors, commentators, consultants, academics, and marketers need to know why their "solutions" aren't quite the panaceas they claim.
This book is about IT architecture. IT architecture provides a framework for discussing implementation design, and it is in these discussions where techies and modelers should meet. Anyone whose roles and responsibilities include IT architect should know everything in this book. (Note we said "know," not "agree with.")
Although IT architects are an important audience for this book, we have tried to write a book for IT management. We have assumed that the IT managers in our readership come fro m an IT background not a business background; this book is not an introduction to IT. So why do IT managers need a book about IT architecture? It is because here so many of their concerns come together—application flexibility, information quality, resiliency, scalability, and so on. One of our aims is to give IT management the knowledge to call the IT architects to account.
This book gives an overview of the whole subject of building and running large distributed systems. It is a deliberate attempt to step above the detail and the in-fighting to examine what is important, what isn't important, and what we need to do differently from the way we did it 10 years ago. Our contention is that the difference between then and now is much more than simply that there are new tools to play with. Building integrated systems is substantially different from building standalone applications, and it affects everything we do in IT.
A major theme of this book is enterprise computing. In the list of terms abused by the industry, "enterprise computing" has to be somewhere near the top. This book takes the view that enterprise computing is about being able to build systems that support the whole enterprise, which in large organizations means many thousands of users. It's obvious that systems that support thousands of users must have resiliency, scalability, security, and manageability as major concerns. The enterprise computing mentality is not prepared to compromise on these objectives. An old mainframe application written in COBOL that gives you resiliency, scalability, security, and manageability is far superior to any implementation that does not.
This is not to say that you cannot build enterprise-capable applications with modern tools, such as Web services. But to succeed you must understand the principles of building large, resilient systems. The principles that served us well for standalone applications do not all apply for distributed systems and vice versa.
Many people in IT find discussions of principles too abstract and dry, so we have tried to enliven the presentation with many examples.
Many organizations today are trying to avoid all these issues by buying third party application packages. This is partially successful. When you buy a package, you buy an IT architecture, albeit only in the context of the package's functionality. If you buy many packages, it is likely that you must lash them together somehow, and for this you need an IT architect. If the packages are from different vendors, integration is a challenge. In this book, we give you the principles that should help in this task, but we have chosen not to address the challenge directly. Because there are so many packages, to do the subject justice would require another book.
This book is not for everyone. You will find this book short on product detail. It does not tell you anything about installation, there are no proper coding examples, there is no survey of products, and there is little in the way of product comparisons. We make no apologies for any of these omissions. There are many books on coding, and product details change so fast the best place for comparisons is on the Internet. This book does not teach application design. There are many books for this as well. But we hope application designers will read this book because the discussion on the principles for building enterprise systems is vital for them also. Finally, this book is not an academic book. It contains little mathematics except for the back-of-the-envelope calculations to illustrate a few points. We offer a practical, wide-ranging discussion for IT professionals to help them understand what is going on so they can pick out the real issues from the imaginary issues and start building complex distributed systems with confidence.
Changes in the second edition
Changes that led to the second edition fall into three categories: new author, new technology, new approaches to design.
First, authorship has now become a team—Peter Bye has joined Chris. Peter brings a long-standing expertise in integration, networking, systems management, and all matters concerned with designing an IT architecture. Although Peter has concentrated on adding to the technical content in the first part of the book, all parts are mulled over by both of us and authorship is shared. Compare the new text with the old and you will notice small changes scattered everywhere. As befits Peter's additional expertise, where there was one chapter on systems management and security, there are now two.
Second, the book has been updated to take into account new technology, in particular Web services, and consequently new ways of thinking about IT architecture, in particular loosely coupled architectures.
Chris was in the final stages of writing the first edition of the book when Microsoft announced .NET and he had no time to digest the merits of the announcement and incorporate it into the text. When Web services moved toward center stage, it became clear that a fundamental driver was a desire for a more loosely coupled architecture or, more specifically, a loosely coupled integration of tightly coupled archipelagos of system. The integrated applications architecture described in the first edition of this book was a quintessentially tightly coupled approach. When many people said that they could never reach enough agreement across an organization to impose a tightly coupled architecture, we could see some truth to these complaints. But loosely coupled integration is impossible without some common standards, and Web services are now providing them.
However, "loosely coupled" is a fuzzy term, easily bandied about by salespeople and consultants, and you can't just apply Web services pixie dust to create loosely coupled integrated applications. One of our aims with this new edition is to identify when loosely coupled works and when it doesn't, and to lay out the advantages and disadvantages of the different approaches to architecture. In consequence, our approach to IT architecture is very flexible and we present a range of architectural options such as middleware bus architectures, hub and spoke architectures, and Web services architectures. The common theme throughout is the notion of service-oriented architectures.
In concrete terms, describing Web services and other new technology has led to...