sed & awk (Nutshell Handbooks) [Paperback]

Dale Dougherty (Author)

Book Description

Publication Date: November 8, 1990 | Series: Nutshell Handbooks

For people who create and modify text files, sed and awk are power tools for editing. Most of the things that you can do with these programs can be done interactively with a text editor. However, using sed and awk can save many hours of repetitive work in achieving the same result.

To master sed and awk you must thoroughly understand UNIX regular expressions, and understand the syntax of sed and awk commands. This handbook treats regular expressions as a foundation for learning about sed and awk and contains a comprehensive treatment of sed and awk syntax. It emphasizes the kinds of practical problems that sed and awk can help users to solve, with many useful example scripts and programs.

Contents include:

• A tutorial showing basic operations of sed and awk
• Regular expression syntax and examples
• Basic and advanced sed commands
• Features of awk (plus gawk and nawk)
• Common programming constructs
• Advanced topics
• User-contributed scripts

Editorial Reviews

From the Publisher

sed & awk, one of the most popular books in O'Reilly & Associates' Nutshell series, describes two text processing programs that are mainstays of the UNIX programmer's toolbox. sed is a "stream editor" for editing streams of text that might be too large to edit as a single file, or that might be generated on the fly as part of a larger data processing step. The most common operation done with sed is substitution, replacing one block of text with another. awk is a complete programming language. Unlike many conventional languages, awk is "data driven" -- you specify what kind of data you are interested in and the operations to be performed when that data is found. awk does many things for you, including automatically opening and closing data files, reading records, breaking the records up into fields, and counting the records. While awk provides the features of most conventional programming languages, it also includes some unconventional features, such as extended regular expression matching and associative arrays. sed & awk describes both programs in detail and includes a chapter of example sed and awk scripts. This new edition covers features of sed and awk that are now mandated by the POSIX standard. This most notably affects awk, where POSIX standardized a new variable, CONVFMT, and new functions, toupper() and tolower(). The CONVFMT variable specifies the conversion format to use when converting numbers to strings (awk used to use OFMT for this purpose). The toupper() and tolower() functions each take a (presumably mixed case) string argument and return a new version of the string with all letters translated to the corresponding case. In addition, this edition covers GNU sed, newly available since the first edition. It also updates the first edition coverage of Bell Labs nawk and GNU awk (gawk), covers mawk, an additional freely available implementation of awk, and briefly discusses three commercial versions of awk, MKS awk, Thompson Automation awk (tawk), and Videosoft (VSAwk). --This text refers to an alternate Paperback edition.

From the Back Cover

The book begins with an overview and a tutorial that demonstrate a progression in functionality from grep to sed to awk. sed and awk share a similar command-line syntax, accepting user instructions in the form of a script. Because all three programs use UNIX regular expressions, an entire chapter is devoted to understanding UNIX regular expression syntax. Next, the book describes how to write sed scripts. After getting started by writing a few simple scripts, you'll learn other basic commands that parallel manual editing actions, as well as advanced commands that introduce simple programming constructs. Among the advanced commands are those that manipulate the hold space, a set-aside temporary buffer. The second part of the book has been extensively revised to include POSIX awk as well as coverage of three freely available and three commercial versions of awk. The book introduces the primary features of the awk language and how to write simple scripts. You'll also learn: common programming constructs; how to use awk's built-in functions; how to write user-defined functions; debugging techniques for awk programs; how to develop an application that processes an index, demonstrating much of the power of awk; and FTP and contact information for obtaining various versions of awk. Also included is a miscellany of user-contributed scripts that demonstrate a wide range of sed and awk scripting styles and techniques.

--This text refers to an alternate Paperback edition.

See all Editorial Reviews

Product Details

• Paperback: 414 pages
• Publisher: O'Reilly Media; 1 edition (November 8, 1990)
• Language: English
• ISBN-10: 0937175595
• ISBN-13: 978-0937175590
• Product Dimensions: 8.9 x 5.9 x 1 inches
• Shipping Weight: 1.2 pounds
• Average Customer Review: 4.4 out of 5 stars  See all reviews (45 customer reviews)
• Amazon Best Sellers Rank: #991,332 in Books (See Top 100 in Books)

Customer Reviews

Most Helpful Customer Reviews

52 of 55 people found the following review helpful:

5.0 out of 5 stars 2 cute little guys who will work hard for you, July 1, 2004

By 

Alexander E. Paulsen "AlexP" (Jacksonville, Fl United States) - See all my reviews
(REAL NAME)   

This review is from: sed & awk (2nd Edition) (Paperback)

I've had this book for 3-4 years now and find it to be my handiest reference. I even use ot more than my vi book.

The raw power of sed and awk will humble even the proudest VB programmer and this book will help you get the most out of both the "Aho, Weinberg and Kernigan" and the "stream editor".

The first chapter section is entitled "May You Solve Interesting Problems" a paraphrase of the old Chinese curse, but with sed and awk no problem seems insurmountable and will make all problems appear interesting.

This book is an outstanding reference and will get you up and running with both of these handy little programs in no time.

Just to give you an example - the first time I ever used these programs (sed and awk) I developed a cron script to query a database every day at midnight, sort the results, grabbed lines with tagged values within certain limits, added a few totals, did some averaging, "starred" the lines that departed from certain parameters, then formatted the results into a report with a header showing the dates, times and query results, statistics then packed them into a report file and e-mailed them to a dozen recipients and added a few recipients if the data was outside certain values.

I did this remarkable feat by copying a few examples from the book, changing a thing or two and in the end had a little shell script that was a few K-bytes at the most.

This VB guy from corporate worked on a VB version of the same function and ended up with six revisions, several megs in the executable and never was able to get it formatted right. Not to say that it couldn't be done but..... I wrote mine before lunch one day. He worked on his for weeks and weeks.

Such is the power contained within should you want to tap it Grasshopper.

Great book. Amaze your friends and colleagues. Have the office women ( or men) throw flowers at your feet as you enter every day - well maybe not, but this will make you more productive.

You real nerds will hardly get finished with a single box of Pop Tarts before you are writing really cool scripts when using this book.

Like all O'Reilly books this one is terse, practical and highly usable - just like UNIX.

32 of 33 people found the following review helpful:

4.0 out of 5 stars You'll not regret this one!, September 25, 2001

By 

Ales Kavsek (Ljubljana, Slovenia) - See all my reviews

This review is from: sed & awk (2nd Edition) (Paperback)

My primarily profession is database administration on Windows NT/2000 platform so I'm pretty new to Unix tools. Several months ago I needed some inexpensive, yet powerful tools to manipulate some large text files for our data warehouse project -- basically I was looking for a way to avoid expensive commercial ETL tools. PERL was my first choice, but it seemed to me that learning curve is too steep for what I needed, so I ended with GNU sed and awk.
I was happy with the GNU version of Guide for Awk "Effective AWK Programming" written by one of the author of this book, Mr. Arnold D. Robbins -- but I needed some good reference for sed as well and this book was logical choice for me. Not only that it helped me to learn SED, now I'm using the book mostly as reference and user guide for awk. It's well written and will certainly satisfy both the advanced and the novice users.
The only objections that I have are examples at the end of the book. Not many readers will find them interesting and useful, especially chapter 12 Full-Featured Applications.
Overall good choice if you want to learn either of the two tools!

30 of 32 people found the following review helpful:

4.0 out of 5 stars Powerful tools for text manipulation, April 15, 2000

By 

Primoz Peterlin - See all my reviews

This review is from: sed & awk (2nd Edition) (Paperback)

Unix has earned itself quite a reputation for its potent tools, used for batch editing of text files (like program output). Sed and Awk are two of these tools. Sed is a direct descendent of Ed, the original Unix line editor, which employs regular expressions, a powerful method for description of patterns in text, for operations like substitute, append or delete. Awk is a complete scripting language with programming structures like conditionals, loops, functions etc., developed in 1970's by Alfred Aho, Brian Kernighan and Peter Weinberger (hence A-W-K). The trio has also written a book on Awk.

Dale Dougherty (in the 2nd edition with Arnold Robbins, maintainer of GNU Awk and author of several more books on Awk programming language) have made a good job in making a thoroughly readable tutorial on Sed and Awk. However, it remains a mystery to me how they succeeded to fill no less than 407 pages with it. Mind you, Sed and Awk are not really some big monsters. There exist something like two dozens of operators in Sed (most of them you will probably never use), and the syntax of Awk mimics those of C programming language, so it is likely that you know it already. Once you grok the idea of regular expressions, you should become a proficient user of Awk in about 30 minutes.

In conclusion, go buy the book if your need to manipulate text files on Unix and you think you need a lengthy tutorial with a gentle learning curve. Otherwise, short references on Awk and Sed, like the ones in "Unix Power Tools" and a bunch of examples showing some tricks you might not think of, will probably be more useful. In addition, it is good to know that during the nineties, much of the focus has drifted from Awk to Perl, so you might consider a book on Perl as well.