PREFACE

UNIX for the Impatient is a UNIX handbook--a detailed, comprehensive guide to the UNIX system that can serve both as a ready reference and as a means of learning UNIX. We've consciously aimed the book at readers who are comfortable with technical material, presenting the subject as concisely as possible rather than in gentle but lengthy tutorial steps and covering in a section what other authors cover in a volume. We achieve that not by omitting details but by omitting long explanations and numerous examples that illustrate a single point. Though the tutorial approach may be less demanding, ours is faster; thus the title of our book.

The UNIX Operating System

UNIX has become the standard operating system for computers in scientific, engineering, and research environments. Designed to be used interactively, UNIX is not bound by inherent memory constraints, handles multiple processes and users easily and naturally, and is well suited as a platform for networking and graphical environments. As the only major operating system not irrevocably tied to a specific hardware architecture, UNIX is particularly well adapted to use in networks of heterogeneous computers. Most UNIX systems now support the graphical user interface provided by the X Window System and most UNIX users use that interface. The result is that UNIX is now as easy and comfortable to use as any of the other graphics-based systems such as Microsoft Windows that are available on workstations and personal computers.

Commercially supported versions of UNIX have long been available for most computers, but their price has usually been prohibitive for individual users. High-quality free systems that run on personal computers are now widely available as well. The advent of those free systems, Linux in particular, has enormously broadened the population of UNIX users.

The POSIX.1 standard for UNIX-like system calls has been widely accepted for several years and has enabled programmers to write portable UNIX programs. The newer POSIX.2 standard for user commands, the standard on which this book is based, has meant that UNIX systems from different sources and on different computers have become more consistent with one another. Additional POSIX standards under development promise to make it even easier to move among UNIX systems.

But UNIX is not an easy system to learn. In the early days of UNIX when it was still a research project, one person could master all its details and even a casual user could master most of them. That is true no longer. Today's UNIX user faces great obstacles: the profusion of commands and their options, variables, and subcommands; the differences among implementations; the rapid pace of change; the proliferation of related subsystems such as the Network File System and the X Window System; the sheer volume of material to be assimilated; the historical accretions; and the assumption all too prevalent in UNIX documentation that the reader is already familiar with what is being described.

Technically sophisticated people often prefer to learn new programs and computer systems by reading the reference manuals, but it's hard to learn UNIX that way. The traditional UNIX reference manual consists of an alphabetical list of command descriptions called "manual pages". Many manual pages are cryptic, misleading, poorly organized, or erroneous in subtle ways. UNIX-specific terminology is often used without being defined; terminology is often used inconsistently from one command to another. The customary alphabetical list of options makes it difficult to see how the options relate to one another. Few manuals provide an overview or explanation of the basic UNIX concepts, and those that do often rely on early technical papers that are now about two decades old and cannot reflect what has happened since they were written.

Overview of the Book

The central purpose of UNIX for the Impatient is to serve as a better manual for people who like to learn from manuals. We tell you about those undocumented, confusing, or subtle aspects of UNIX commands that a reading of the manual pages is unlikely to reveal. Rather than lead you by the hand through UNIX, we assist you in finding your own way. If you're seeking an easier but slower route to learning how to use UNIX, we suggest getting started with tutorial books (you may need several) and then turning to UNIX for the Impatient as you progress.

We describe UNIX from the standpoint of a user; that is, a person primarily interested in it as a tool rather than as an object of study for its own sake. Much of the book consists of command descriptions, usually as detailed as the ones in your system's manual. An important difference between our book and your manual is that we've organized the book and its parts, including the descriptions of the commands, logically by function rather than alphabetically. The functional arrangement enables you to see how different commands and options relate to one another, making it easier to find the right tools for a particular task. We include an alphabetical summary of commands and their options cross-referenced to the pages where you'll find the full descriptions. You can use the alphabetical summary both to get a quick reminder of what a particular command or option does and to retrieve the full description. The synopsis preceding the table of contents provides a fast overview of everything in the book, and we have paid particular attention to providing a thorough index.

In the second chapter of the book we cover the essential concepts of UNIX such as processes, files, shells, regular expressions, pipes, and redirection. These concepts underly the detailed descriptions in the rest of the book. We also provide a glossary containing concise definitions of UNIX terminology. When you encounter a technical term that you haven't seen before, you can easily find out what it means. We provide abundant cross-references to help you through the thicket of dependencies among different UNIX commands and concepts. To get you oriented, we provide a discussion of how to use this book in Section 1.3.

In keeping with our orientation of UNIX for the Impatient as a book primarily for users, we don't generally say much about UNIX internals--those aspects of UNIX mainly of interest to system programmers--although if you're a system programmer, this book can still be helpful for dealing with the external aspects of UNIX. Nor do we attempt to describe the programming facilities of UNIX such as the C compiler, lex,andyacc. How to use these tools, particularly C, can occupy a book by itself.

We do, however, describe awk, which is relatively easy to learn and which provides an easy way of transforming and filtering data while enabling you to carry out fairly complex programming tasks. In a few places, such as our discussions of awk and of shell scripts, we assume some familiarity with programming methods and terminology--we couldn't avoid it--but most of the book can be understood even if you aren't familiar with programming.

Why UNIX Is Hard to Describe

UNIX has become what it is today more through accretion than through conscious planning. There's a principle of programming language design called "orthogonality"--the idea that different aspects of the language interact as little as possible and can be treated independently. UNIX, unfortunately, violates that principle egregiously. Programs are sensitive to the conventions of other programs and interact with each other in complex ways. For example, the vi visual editor was built on top of the ex extended line editor, so-called because it extended the earlier ed editor. Thus to understand vi fully you also must know ex even though you have no interest in ex for its own sake.

Moreover, many UNIX facilities are someone's improvement or elaboration on an existing facility. The old version persists even when the new version is indisputably better. Programs with overlapping functions are common. Thus there are three data compression programs, pack, compress,and gzip; at least two different mail programs that call themselves mail; and several programs for splitting files into segments, head, tail, split, csplit, and dd. There are at least four widely used shells for managing interaction with your terminal: the KornShell ksh, the Bourne shell sh,the Cshell csh, and Bash, the GNU "Bourne-again Shell".

This profusion of similar facilities created a dilemma for us. How many of these programs should we try to cover? We have chosen to describe nearly all the utility commands despite their overlapping functions. On the other hand, we have based our discussion of shells in general on the KornShell, since that shell includes nearly all the facilities found in the others. We supplement that discussion by a description of the features of the other shells where they differ from the KornShell.

British lorries roll on tyres, while American trucks roll on tires. Just as there are many dialects of English, there are many versions of UNIX. Fortunately the POSIX.2 standard has brought a measure of consistency to the UNIX world; most implementors now adhere to it. But major differences among systems persist, particularly between System V-based systems and BSD-based systems. The POSIX specifications for some commands such as man and ps are deliberately skeletal, recognizing that the differences among existing implementations are too great to be bridged. Most systems provide additional commands beyond those specified by POSIX and additional options for the commands that are specified by POSIX. Pre-POSIX systems such as Solaris 1.x are still widely used, and pre-POSIX commands are still often found in otherwise POSIX-conforming systems.

The command descriptions in the Second Edition of UNIX for the Impatient follow the POSIX standard. However, we note many important variations and include a number of commands that are not part of POSIX. For example, we describe both the POSIX lp command for sending files to a line printer, derived from the System V command of the same name, and the BSD lpr command that does nearly the same thing but does it differently. You may find that some commands don't work on your system exactly as we describe them, but the differences will usually be minor.

Our Approach

You'll find topics covered in this book that don't make it into most UNIX books. Some examples are the X Window System, the emacs editor, communications programs such as telnet and ftp, the addressing conventions used for Internet and UUCP electronic mail, and popular but nonstandard utilities such as the gzip compression program and the shar archiver for sending files via electronic mail. Although we hope this book is the first one you take off the shelf, we realize it may not be the last, so we provide a list of printed and electronic resources you can use for additional information or for another view of the same material.

Our approach to UNIX is like Albert Einstein's approach to physics: make it as simple as possible, but no simpler. We explain UNIX as clearly as we can without neglecting difficult or complicated, but nevertheless essential, details. While some of these details may not seem important when you first see them, you're likely to find them indispensable as you read the manuals supplied with your system.

For those who notice such things, we mention that we've adopted the British (logical) treatment of punctuation at the end of a sentence: when a quotation ends a sentence, we place the period outside the quotation. We consider this policy necessary for any book whose subject matter demands great orthographic precision. We appreciate the indulgence of our editors.

Some people use UNIX by choice, others by necessity. We hope UNIX for the Impatient will be useful to you whatever your view of UNIX may be.

About the Second Edition

The Second Edition of UNIX for the Impatient reflects both the dramatic changes in the UNIX world from 1991 to 1995 and changes in our own understanding of how we as authors can best meet the needs of our readers. The world of commercial UNIX has been profoundly affected by AT&T's sale of UNIX Systems Laboratories (USL) to Novell, Inc., and by the refinement and general acceptance of the POSIX standards. High-quality free systems such as Linux and FreeBSD, complete with source code unaffected by USL licensing restrictions, have appeared, at last making UNIX affordable and available to individual users. The scope of the software components found in the GNU collection from the Free Software Foundation has broadened and the components themselves have matured; some of these components are now used in nearly every UNIX installation. The number of Internet users, the services available through the Internet, and the number of Internet service providers have all multiplied prodigiously to the point where "Internet" is now a household term and great numbers of people access the Internet as individuals, not as members of some organization.

In the First Edition we used System V as the base for our description of UNIX's facilities; in the Second Edition we use the POSIX.2 standard. This shift has led us to introduce descriptions of additional commands and has brought about many and pervasive textual revisions, including changes in terminology to be consistent with POSIX. Following the practice in the POSIX standard, we have replaced most generalized references to "options" in the syntax of commands with lists of the specific options each command accepts.

Because POSIX has already been widely adopted and POSIX conformance by UNIX vendors is increasing, you are now likely to find fewer differences between the features we describe and the ones you find in your particular version of UNIX. Nearly all the purely syntactic changes specified by POSIX are for the sake of greater uniformity and consistency. These changes have made the UNIX command interface much easier to use.

We have greatly expanded the number of commands we cover. The Second Edition describes all the POSIX user-level commands except for a few specifically intended for programmers: ctags and nm from the User Portability Utilities and ar, make,and strip from the Software Development Utilities. We have, however, retained our brief discussion of tag tables from the First Edition, and the material on software installation includes a discussion of makefiles from the user's viewpoint. We have also added some widely used non-POSIX commands such as gzip and shar.

It would take a separate book to provide anything like an adequate guide to the Internet, and indeed a number of such books have already been written. In this book we have expanded our Internet coverage to include the World Wide Web and other tools for exploring the riches of the Internet. We also include an explanation of USENET newsgroups and the newsreader programs used to access them.

Version 19 of Emacs is fully integrated with the X Window System and has many new and enhanced features. We have expanded our coverage of Emacs and have divided our treatment into two parts: a chapter on the editing facilities proper and another on Emacs's rich set of utilities, utilities such as the Directory Editor, the Mailer, and the Outliner.

The Bourne shell sh has largely been superseded as a user-level shell, even though it still provides a base set of facilities useful for writing shell scripts intended for general distribution. We have replaced its description by a description of the KornShell ksh, which provides a superset of the shell facilities specified by POSIX and also of those in the Bourne shell. We note those features of the KornShell that are not supported by the POSIX shell or the Bourne shell. We have folded our description of the C shell into the description of the KornShell and have added material on Bash.

As use of UNIX has spread beyond its American origins, internationalization has become essential to many UNIX users. The POSIX standard provides locales and other features that make UNIX suitable for international use. We have added a section on internationalization to the Concepts chapter and have modified the descriptions of UNIX facilities to account for the new POSIX features.

The brief section on "System Administration for Single-User Systems" has been moved and expanded into a full chapter, "Managing Your System". The new chapter covers important system files such as those usually found in the /etc directory and programs such as mount, lpd, and init that provide administrative services. It also explains how to install new software, particularly in the usual case where the software has been distributed in the form of source code. While intended primarily for readers who are running single-user systems for themselves, the new chapter should be helpful to users of larger systems who wish to understand what is going on within those systems or need to install software from an outside source. It will also assist anyone attempting to decode the frequent references to these files and programs in the UNIX literature.

About the CD-ROM

UNIX for the Hyper-Impatient is a hypertext version of UNIX for the Impatient, distributed on a CD-ROM. The idea of it came from Lisa Richards of QuickScan Electronic Publishing, who pointed out that a book such as this one is a natural candidate for a hypertext form because there are so many cross-references from one part to another. UNIX for the Hyper-Impatient is included in the CD-ROM version of this book and is also available separately.

UNIX for the Hyper-Impatient contains the entire text of UNIX for the Impatient, together with the DynaText browser developed by Electronic Book Technologies of Providence, Rhode Island. The primary view of the book contains two windows, one showing the Table of Contents and the other the text of the book. Using the Table of Contents, which you can expand or collapse, you can select any part of the book and scroll through the text starting there.

Wherever there's a cross-reference in the printed book, there's a hyperlink in the hypertext version. By clicking your mouse on that hyperlink, you can jump directly to the cross-referenced text, then easily return to where you were before. You can also check out all the index references to a topic by going to the index entry, then looking at the references one by one. Footnotes and figures don't interrupt the flow of the text; you view them by clicking on an icon that appears where the footnote or figure is referenced. You can also search either the entire book or just the headings, using a variety of search specifications.

Information about the features of the DynaText browser and how to use the CD-ROM is provided in the shaded pages at the end of the CD-ROM version of the printed book and in the booklet that accompanies the separately packaged version of UNIX for the Hyper-Impatient.

Acknowledgments

We wish to thank our colleagues at Addison-Wesley who made this book possible, especially Peter Gordon, our editor, who conceived the idea of the book and whose encouragement and helpful suggestions were invaluable, and Helen Goldstein, his associate editor, who patiently and unfailingly navigated the book past many obstacles. We are grateful to our production supervisors, Loren Hilgenhurst Stevens for the First Edition and Nancy Fenton for the Second Edition, whose skill and patience contributed so much to turning a manuscript into a book; and to Karen Wernholm and Patricia Unubun, the production supervisors of the CD-ROM version, for their help in dealing with all the novel problems of this type of production. We thank Lorraine Ferrier, the able proofreader of the First Edition, for an exceptionally thorough job performed under stringent time constraints. To Constance G. Burt, the proofreader of the Second Edition, we pay the ultimate compliment a fastidious author can offer: we agreed with practically all her corrections.

We also wish to thank George Lukas, who helped refine the original plan of the book; Bob Morris, Betty O'Neil, and Rick Martin of the Computer Science Department at the University of Massachusetts, Boston, who generously provided us with access to the Department's computing facilities; Karl Berry, who provided us with access to several UNIX systems and patiently answered many of our questions, both about UNIX and about T E X; Keith Bostic of UC Berkeley for his assistance in interpreting the POSIX standard and figuring out how the bugs in it might be fixed; Jack Dwyer of the Open Software Foundation for answering many questions about the role of OSF in UNIX development; Paul English, who provided us with access to a variety of other UNIX systems; Rob Kolstad of Berkeley Software Design for providing information about recent BSD systems and their derivatives; Michael Larson, who carefully checked and critiqued much of the First Edition and the Emacs chapters in the Second Edition; Don Libes for his valuable explanation of the mystical behavior of 'cat -u'; Jared P. Martin of the Shell Oil Company for his many small but essential corrections; Sandra O'Donnell of the Open Software Foundation for her enlightening explanation of the subtler aspects of POSIX locales; Frank and Lisa Richards of QuickScan Electronic Publishing for suggesting the CD-ROM version and for their skill and hard work in developing it; Hans Wenland for his tests of the CD-ROM version; Peter Weidner for his comments on the First Edition, which we've gratefully applied to the Second Edition; Dick Wood and Jeff Hanson of X/Open for clarifying to us the new role of X/Open as custodian of the UNIX trademark; and Jerry Nowlin of Iconic Software, Inc., John Norden of Amdahl, and John Chambers and Dick Muldoon of USL, who answered some of our more exotic questions.

We especially thank the reviewers of both editions, whose comments and feedback made the book far better than it would have been otherwise: Manuel Bermudez, Don Bolinger, Lyn Dupré, Jennifer Knuth, Alan Langerman, George Leach, Thomas F. Reid, Bjorn Satdeva, Steven Stepanek, Gregory Tucker, and Gerald Weiss. Of course, the responsibility for any improvements we should have made but didn't remains our own.

Then there are our individual debts of gratitude to those who provided us with support and encouragement and were so patient and understanding with us: from Bruce to his friends and colleagues Susan Kubany, Bob Heinmiller, and Dave Curado of Omnet, Inc., and Mike Gauthier and Henry McAvoy of Artis, Ltd., and to his wife, Cathy, and his children, Elsa and Eric; and from Paul to his daughter, Jodi, his sister, Nan Rubin, his parents, Al and Evelyn Abrahams, and his companion during the development of the CD-ROM version, Roberta B. Morris.

Bruce wishes to express his personal gratitude to Tom Wester and Ted Treibick of Vianet, Larry Conway and Larry Cudlitz of Epsilon, and Karl Berry for the support they gave him during the preparation of the Second Edition.