Perl & LWP (Paperback)

by Sean M. Burke (Author)
Key Phrases: token sequences, file uploads, user agent object, Fresh Air, Walter Kirn, Joanna Merlin (more...)

Editorial Reviews

Amazon.com Review
Perl & LWP sets out to unwrap the Library for the Web in Perl (LWP), which is a collection of modules that make it easier to access and pick apart Web pages (and FTP-accessible files, and outgoing e-mail messages) from within your Perl programs. The book succeeds wonderfully, not only in conveying the technical aspects of LWP programming, but in making clear the fun of doing work that's very well suited to Perl. Sean Burke assumes that his readers know something about Perl, albeit not much, and a similar amount about HTML. He does a great job of explaining how LWP functions fit into Perl programs, and how you can use them to make reference to Internet resources far more easily than before.

Burke's narrative takes the form of a guided tour in which he introduces his readers to aspects of the LWP modules one by one. His tone is generally straightforward (sharp commentary alternates with brief code listings, with occasional passages of reference material), but there's sometimes an undercurrent of exuberance that makes the reader want to get going with his or her own programming right away. Overall, the emphasis is on teaching both LWP and Perl itself to the extent necessary to do LWP work. Because of the concise and nicely indexed code modules, though, you'll find this book useful as a reference after you're under way with LWP. --David Wall

Topics covered: How to program with LWP and Perl itself. All of LWP's strong points--including HTML parsing (with tokens and trees as well as with regular expressions), HTML generation and modification, manipulation of HTML forms, and the operation of spiders--are covered. This book has more of a tutorial tone than any similar reference material on the Internet.

Product Description
With the LWP (Library for WWW in Perl) suite of modules, your programs can download and extract information from the Web. Written by a contributor to LWP, and with a foreword by one of LWP's creators, Perl & LWP is the authoritative guide to using this powerful and popular toolkit.You'll learn how to navigate the Web with LWP, fetching web pages, submitting forms, managing cookies, and accessing authenticated web sites. The book also demonstrates using three different techniques, regular expressions, tokens, and tress, to extract information from HTML. The many detailed case studies show you how to apply this knowledge to real world situations, from downloading news stories to e-commerce. Perl & LWP is a must have for Perl programmers who want to automate and mine the Web.

See all Editorial Reviews

Product Details

• Paperback: 260 pages
• Publisher: O'Reilly Media, Inc.; 1st edition (June 20, 2002)
• Language: English
• ISBN-10: 0596001789
• ISBN-13: 978-0596001780
• Product Dimensions: 9.2 x 7 x 0.8 inches
• Shipping Weight: 1.1 pounds (View shipping rates and policies)
• Average Customer Review: 4.6 out of 5 stars See all reviews (10 customer reviews)
• Amazon.com Sales Rank: #113,212 in Books (See Bestsellers in Books)

  Popular in this category: (What's this?)

  #85 in

  Books > Computers & Internet > Web Development > Web Services

Customer Reviews

Most Helpful Customer Reviews

18 of 18 people found the following review helpful:

5.0 out of 5 stars Excellent coverage of LWP, packed full of useful examples, July 16, 2002

By	Gavin Estey (Burlington, VT, USA) - See all my reviews

I was definitely interested when I first heard that O'Reilly were publishing a book on LWP. LWP is a definitive collection of perl modules covering everything you could think of doing with URIs, HTML, and HTTP. While 'web services' are the buzzword friendly technology of the day, sometimes you need to roll your sleeves up and get a bit dirty scraping screens and hacking at HTML. For such a deep subject, this book weighs in at a slim 242 pages. This is a very good thing. I'm far too busy to read these massive shelf-destroying tomes that seem to be churned out recently.

It covers everything you need to know with concise examples, which is what makes this book really shine. You start with the basics using LWP::Simple through to more advanced topics using LWP::UserAgent, HTTP::Cookies, and WWW::RobotRules. Sean shows finger saving tips and shortcuts that take you more than a couple notches above what you can learn from the lwpcook manpage, with enough depth to satisfy somebody who is an experienced LWP hacker.

This book is a great reference, just flick through and you'll find a relevant chapter with an example to save the day. Chapters include filling in forms and extracting data from HTML using regular expressions, then more advanced topics using HTML::TokeParser, and then my preferred tool, the author's own HTML::TreeBuilder. The book ends with a chapter on spidering, with excellent coverage of design and warnings to get your started on your web trawling.

14 of 14 people found the following review helpful:

5.0 out of 5 stars This book can teach you expert-level web scraping/munging., July 12, 2003

By	"mwickline" (Portland, Oregon USA) - See all my reviews

If you aren't yet comfortable using object-oriented Perl modules, the multitude of examples will at least allow you see how it's done even if you're a bit fuzzy on what's happening 'underneath' when you call object methods. If you're comfortable learning how to do something without knowing exactly why it works, then the author's clear step-by-step explantions and numerous progressively more powerful examples should make this book accessible even to relatively innexperienced Perl programmers.

More experienced programmers will understand better why things work, but any Perl programmer will set this book down feeling empowered to turn the web into their own valet. No longer do you need to check multiple sites looking for interesting information. Instead, you can readily author code to do that for you and alert you when items of interest are found. You can use these tools to free up personal time, to harvest information to inform business decisions, to automate tedious web application testing, and a zillion other things.

The author's clear exploration of the relevant Perl modules leaves the reader with a good depth of understanding of what these modules do, when you might want to use which module, and how to use them for real world tasks. Before reading the book, I knew of these modules, but they were a rather intimidating pile. I'd used a few of them on occasion for rather limited projects, but was reluctant to invest the time required to read all of the documentation from the whole collection. Mountains of method-level documentation do not a tutorial make. This book takes all of that information, selects the most important parts, and ensures that those parts are covered in progressively more powerful and/or flexible examples.

If you know Perl and you're sick of 'working the web' to get information and you want the web to work for you instead, then you need this book. I had a personal project that was on the back burner for a couple of years because it just sounded too hard. The weekend after I finished this book, I wrote what I had previously thought to be the hard part of that project and it was both easy and fun. This book makes hard things not just possible, but actually easy.

-matt

14 of 14 people found the following review helpful:

5.0 out of 5 stars Very Informative and useful, August 7, 2002

By	"sherzodr" (East Lansing, MI USA) - See all my reviews

As a web programmer, I had dealt with several such projects dealing with web automation and writing simple crawlers even before I read "Perl & LWP". The book was the first book I've read on the subject, and I'm by no means disappointed. The book is very well organized, very informative and nails the subject in the head. I am pleased.

I noticed some inaccuracies in the discussions, some chopped off paragraphs and sentences. But this doesn't affect the usability of the book much. Author Sean Burke does a great job in walking one through the most of the aspects of web automation and data extraction in the web using Perl and LWP (libwww in Perl ).

The codes the book gives are very well organized, well written and easily debugable. The steps are pretty consistent across all the examples:
a) Inspect the HTML source code of the page;
b) Determine the tokens and patterns of interest;
c) Write the first code;
d) Fine tune the code;

As usual, I'll be commenting on individual chapters to give you an idea of the
coverage of the book in more details...