Industrial-Sized Deals Best Books of the Month Shop Women's Handbags Learn more nav_sap_plcc_6M_fly_beacon Andra Day Storm Fire TV Stick Subscribe & Save Find the Best Purina Pro Plan for Your Pet Shop Popular Services Home Theater Setup Plumbing Services Assembly Services Shop all tmnt tmnt tmnt  Amazon Echo Fire HD 6 Kindle Voyage The Walking Dead\ Shop Back to School with Amazon Back to School with Amazon Outdoor Recreation Deal of the Day
Programming Amazon Web Services and over one million other books are available for Amazon Kindle. Learn more

Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB 1st Edition

9 customer reviews
ISBN-13: 978-0596515812
ISBN-10: 0596515812
Why is ISBN important?
This bar-code number lets you verify that you're getting exactly the right version or edition of a book. The 13-digit and 10-digit formats both work.
Scan an ISBN with your phone
Use the Amazon App to scan ISBNs and compare prices.
Have one to sell? Sell on Amazon
Buy used
Buy new
More Buying Choices
31 New from $24.95 29 Used from $4.91
Free Two-Day Shipping for College Students with Amazon Student Free%20Two-Day%20Shipping%20for%20College%20Students%20with%20Amazon%20Student

InterDesign Brand Store Awareness Rent Textbooks
$31.23 FREE Shipping on orders over $35. In Stock. Ships from and sold by Gift-wrap available.

Frequently Bought Together

Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB + Programming Amazon EC2 + Resilience and Reliability on AWS
Price for all three: $86.50

Buy the selected items together

Editorial Reviews Review

Product Description
Building on the success of its storefront and fulfillment services, Amazon now allows businesses to "rent" computing power, data storage and bandwidth on its vast network platform. This book demonstrates how developers working with small- to mid-sized companies can take advantage of Amazon Web Services (AWS) such as the Simple Storage Service (S3), Elastic Compute Cloud (EC2), Simple Queue Service (SQS), Flexible Payments Service (FPS), and SimpleDB to build web-scale business applications. With AWS, Amazon offers a new paradigm for IT infrastructure: use what you need, as you need it, and pay as you go. Programming Web Services explains how you can access Amazon's open APIs to store and run applications, rather than spend precious time and resources building your own. With this book, you'll learn all the technical details you need to: Store and retrieve any amount of data using application servers, unlimited data storage, and bandwidth with the Amazon S3 service Buy computing time using Amazon EC2's interface to requisition machines, load them with an application environment, manage access permissions, and run your image using as many or few systems as needed Use Amazon's web-scale messaging infrastructure to store messages as they travel between computers with Amazon SQS Leverage the Amazon FPS service to structure payment instructions and allow the movement of money between any two entities, humans or computers Create and store multiple data sets, query your data easily, and return the results using Amazon SimpleDB. Scale up or down at a moment's notice, using these services to employ as much time and space as you need Whether you're starting a new online business, need to ramp upexisting services, or require an offsite backup for your home, Programming Web Services gives you the background and the practical knowledge you need to start using AWS. Other books explain how to build web services. This book teaches businesses how to take make use of existing services from an established technology leader.

Create HTML POST Forms That Allow Your Web Site Visitors to Upload Files Into Your S3 Account Using a Standard Web Browser
By James Murty, creator of the JetS3t Java S3 library and author of Programming Amazon Web Services

Amazon’s Simple Storage Service (S3) provides cheap and unlimited online data storage for anyone with a credit card and an Amazon Web Service (AWS) account. If you have an AWS account, you can interact with the S3 service using specialized tools to upload and manage your files. It is very convenient to have access to this online storage resource for yourself, but there may be situations where you would like to allow others to upload files into your account.

For this purpose, S3 accepts uploads via specially-crafted and pre-authorized HTML POST forms. You can include these forms in any web page to allow your web site visitors to send you files using nothing more than a standard web browser.

In this article, I will demonstrate how to build simple S3 POST forms. I will assume that you have already signed up for the S3 service, and that you have an S3 client program for creating buckets and viewing files in your account. Before you proceed, create your own bucket to store uploaded files — in the examples below I will use a bucket named s3-bucket.

POST Form Web Page

Here is a web page with an S3 POST Form that you can use as a template for your own forms:

<html>    <head>     <title>S3 POST Form</title>      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />   </head>    <body>      <form action="" method="post" enctype="multipart/form-data">        <input type="hidden" name="key" value="uploads/${filename}">       <input type="hidden" name="AWSAccessKeyId" value="YOUR_AWS_ACCESS_KEY">        <input type="hidden" name="acl" value="private">        <input type="hidden" name="success_action_redirect" value="http://localhost/">       <input type="hidden" name="policy" value="YOUR_POLICY_DOCUMENT_BASE64_ENCODED">       <input type="hidden" name="signature" value="YOUR_CALCULATED_SIGNATURE">       <input type="hidden" name="Content-Type" value="image/jpeg">                 File to upload to S3:        <input name="file" type="file">        <br>        <input type="submit" value="Upload File to S3">      </form>    </body> </html> 

This template demonstrates some important features of an S3 POST form, and the web page that contains it:

  • The web page that contains the form has a meta tag in the head section that tells web browsers to use the UTF-8 unicode character encoding.
  • The form’s action parameter specifies an S3 URL that includes the name of your destination bucket, in this case the bucket called s3-bucket.
  • The form contains a number of input fields with parameter names and values that will be sent to the S3 service. If any required input fields are missing, or if a field has an incorrect value, the service will not accept uploads from the form.

The S3 service uses information from the form’s input fields to authorize uploads, and to set the properties of uploaded file objects. Here is a description of the most common input fields:

Field Name Description
key A name for the S3 object that will store the uploaded file’s data. This name can be set in advance when you know what information the user will upload, for example: uploads/monthly_report.txt.

If you do not know the name of the file a user will upload, the key value can include the special variable ${filename} which will be replaced with the name of the uploaded file. For example, the key value uploads/${filename} will become the object name uploads/Birthday Cake.jpg if the user uploads a file called Birthday Cake.jpg.

AWSAccessKeyId The Access Key Identifier credential for your Amazon Web Service account.
acl The access control policy to apply to the uploaded file. If you do not want the uploaded file to be made available to the general public, you should use the value private. To make the uploaded file publicly available, use the value public-read.
success_action_redirect The URL address to which the user’s web browser will be redirected after the file is uploaded. This URL should point to a “Successful Upload” page on your web site, so you can inform your users that their files have been accepted. S3 will add bucket, key and etag parameters to this URL value to inform your web application of the location and hash value of the uploaded file.
policy A Base64-encoded policy document that applies rules to file uploads sent by the S3 POST form. This document is used to authorize the form, and to impose conditions on the files that can be uploaded. Policy documents will be described in more detail below.
signature A signature value that authorizes the form and proves that only you could have created it. This value is calculated by signing the Base64-encoded policy document with your AWS Secret Key, a process that I will demonstrate below.
Content-Type The content type (mime type) that will be applied to the uploaded file, for example image/jpeg for JPEG picture files. If you do not know what type of file a user will upload, you can either prompt the user to provide the appropriate content type, or write browser scripting code that will automatically set this value based on the file’s name.

If you do not set the content type with this field, S3 will use the default value application/octet-stream which may prevent some web browsers from being able to display the file properly.

file The input field that allows a user to select a file to upload. This field must be the last one in the form, as any fields below it are ignored by S3.

This overview of the form’s input fields should help you to modify the template POST form to suit your own purposes. At a minimum, you will need to edit the form’s action parameter to point to your own S3 bucket, and set the value of the AWSAccessKeyId field to your AWS Access Key credential.

To complete the form and make it acceptable to the S3 service, you will also need to generate a policy document and signature value.

Policy Document

S3 POST forms include a policy document that authorizes the form and imposes limits on the files that can be uploaded. When S3 receives a file via a POST form, it will check the policy document and signature to confirm that the form was created by someone who is allowed to store files in the target S3 account.

A policy document is a collection of properties expressed in JavaScript Object Notation, which simply means that the document’s structure and content must conform to a certain format. Every policy document contains two top-level items:

  • expiration - A Greenwich Mean Time (GMT) timestamp that specifies when the policy document will expire. Once a policy document has expired, the upload form will no longer work.
  • conditions - A set of rules to define the values that may be included in the form’s input fields, and to impose size limits for file uploads.

Here is a policy document corresponding to the POST form template above. This policy has an expiration date of January 1st 2009:

{"expiration": "2009-01-01T00:00:00Z",   "conditions": [      {"bucket": "s3-bucket"},      ["starts-with", "$key", "uploads/"],     {"acl": "private"},     {"success_action_redirect": "http://localhost/"},     ["starts-with", "$Content-Type", ""],     ["content-length-range", 0, 1048576]   ] } 

To create a valid S3 POST form, you must include a policy document whose conditions section contains a rule for almost every input field in the form. At a minimum, this document must include rules for the bucket and key values of the uploaded file object. In addition to these two rules, you will need to include a rule for every other input field in the form except for AWSAccessKeyId, signature, policy and file.

Because our template POST form includes the input fields acl, success_action_redirect, and Content-Type, our policy document includes rules corresponding to these fields. Our policy document also includes an extra content-length-range rule that limits the size of files that can be uploaded.

There are three kinds of rules you can apply in your policy document:

  1. Equality rule, which checks that an input field’s value is set to a given string. An equality rule is expressed as a name and value pair within brace characters, for example: {"acl": "private"}
  2. Starts-With rule, which checks that an input field’s value begins with a given string. If the given string is empty, S3 will check only that the field is present in the form and will not care what value it contains. A starts-with rule is expressed as a three-element array that contains the term starts-with, followed by the name of the input field preceded by a $ symbol, then the prefix string value for comparison.
    In the policy document above, we use starts-with rules for the key and Content-Type fields because we do not know in advance the name of the file a user will upload, or what type of file it will be. The rule for the Content-Type field uses an empty string for comparison, which means it will permit any content type value. The rule for the object’s key name uses the prefix string “upload/”, which means that the key value must always start with the upload/ subdirectory path.
  3. Content length rule, which checks that the size of an uploaded file is between a given minimum and maximum value. If this rule is not included in a policy document, users will be able to upload files of any size up to the 5GB limit imposed by S3.
    A content length rule is expressed as a three-element array that contains the term content-length-range, followed by integer values to set the minimum and maximum file size. The policy document above includes a content length rule that will prevent the form from uploading files larger than 1MB in size (1,048,576 bytes).

It is important to make sure that your policy document corresponds exactly to your S3 POST form. If there are any discrepancies between the input field values in your form and the rule values in your policy document, or if your form contains input fields that do not have corresponding rules in your policy, the S3 service will reject the form and return an incomprehensible XML error message to your users.

Sign Your S3 POST Form

To complete your S3 POST form, you must sign it to prove to S3 that you actually created the form. If you do not sign the form properly, or if someone else tries to modify your form after it has been signed, the service will be unable to authorize it and will reject the upload.

To sign your form you need to perform two steps:

  1. Base64-encode the policy document, and include it in the form’s policy input field.
  2. Calculate a signature value (SHA-1 HMAC) from the encoded policy document using your AWS Secret Key credential as a password. Include this value in the form’s signature input field after Base64-encoding it.

Almost all programming languages include libraries for performing these two steps. Here are some example code fragments to do the job with different languages, assuming you have already defined the variables policy_document and aws_secret_key.


require 'base64' require 'openssl' require 'digest/sha1'  policy = Base64.encode64(policy_document).gsub("\n","")  signature = Base64.encode64(     OpenSSL::HMAC.digest('sha1'),          aws_secret_key, policy)     ).gsub("\n","") 


import sun.misc.BASE64Encoder; import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec;  String policy = (new BASE64Encoder()).encode(     policy_document.getBytes("UTF-8")).replaceAll("\n","");  Mac hmac = Mac.getInstance("HmacSHA1"); hmac.init(new SecretKeySpec(     aws_secret_key.getBytes("UTF-8"), "HmacSHA1")); String signature = (new BASE64Encoder()).encode(     hmac.doFinal(policy.getBytes("UTF-8")))     .replaceAll("\n", ""); 


import base64 import hmac, sha  policy = base64.b64encode(policy_document)  signature = base64.b64encode(, policy, sha).digest()) 

Once you have calculated the values for the policy and signature input fields and included these values in your form, the form should be complete. Save the web page and form document as an .html file, open it in your favorite web browser, and test it by uploading some files to your S3 bucket.


The form web page and policy document templates in this article should give you a starting point for creating your own upload forms. With some minor modifications to the template documents and a little coding, you will be able to create authorized forms that make it easy for your web site visitors to upload files to your S3 account.

The S3 service’s POST support is a powerful feature with many potential uses. You could create a single upload form to allow your friends and colleagues to send you files that are too large for email, or you could modify your web applications to generate forms on-demand so your users can store their data in S3 rather than on your own server. Just remember that you will be liable for any S3 data transfer and storage fees incurred by the people who use your forms.

About the Author

James Murty is a software developer with extensive experience creating web-based applications and architectures using Java. With a working background spanning a research institute, a small software house and various corporations he has a broad perspective on both the promise and the difficulties inherent in networked applications.

Most recently James has been excited to see the rise of webapplications and services that provide compelling new tools and new ways of approaching old problems. While experimenting in this area he created JetS3t, an open source library and application suite that is the leading Java implementation available for Amazon's S3 data storageservice.


Best Books of the Month
Best Books of the Month
Want to know our Editors' picks for the best books of the month? Browse Best Books of the Month, featuring our favorite new books in more than a dozen categories.

Product Details

  • Paperback: 604 pages
  • Publisher: O'Reilly Media; 1 edition (April 4, 2008)
  • Language: English
  • ISBN-10: 0596515812
  • ISBN-13: 978-0596515812
  • Product Dimensions: 7 x 1.3 x 9.2 inches
  • Shipping Weight: 1.9 pounds (View shipping rates and policies)
  • Average Customer Review: 3.3 out of 5 stars  See all reviews (9 customer reviews)
  • Amazon Best Sellers Rank: #189,740 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

44 of 44 people found the following review helpful By Felix Sheng on May 17, 2008
Format: Paperback
This is a good overview of the suite of services that comprise Amazon Web Services (AWS), I'd have given it a 3.5 star rating if I could. It talks about all of them, but it spends the bulk of its time, very reasonably, discussing S3 (the persistent storage system) and EC2 (the compute cloud - basically Amazon's Rackspace in the clouds) - each getting about 100 pages devoted to it.

As others have noted it is out of date - but any book would have the same problem due to the moving target that AWS is. The biggest news is that EC2 is going to be getting persistent storage, which I believe will change the game completely when it is rolled out to the public. Instead of needing some elaborate connection with S3, now instances will behave much more like a typical physical machine with real disk drive. The book, on the other hand, provides almost no real advice on how to deal with the problem of non-persistence of EC2's current storage mechanism. This is a signifcant problem that everyone will have to deal with and glossing over it is a failing of the book.

This is also a Ruby book, which I found fairly annoying. Nowhere in the description does it suggest that it is done in Ruby. And while Ruby certainly is trendy these days, the actual number of Ruby developers is small - it gets undue weight in computer texts. At the end of the day, though, it generally provides the actual request strings and XML requests and responses for non-ruby folk to come to their own conclusions.

This is a worthwhile book to get if you're interested in quickly getting a good and broad idea on how to work with AWS. It will give a good foundation to get more out of the documentation and forums found on Amazon's AWS site itself.
1 Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again
11 of 12 people found the following review helpful By Dan McKinnon VINE VOICE on June 10, 2008
Format: Paperback
'Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB' is a good resource for anyone that is using the Amazon suite of web products and need to learn more about how to get the most out of these powerful set of web 2.0 tools.

For anyone that doesn't know what these tools are, here's a quick one-liner about each:

S3 - online storage to store and retrieve data

EC2 - online computing to be able to run jobs on a farm of machines

SQS - web messaging infrastructure for computer-computer communication

FPS - flexible payment system for moving money online

SimpleDB - store and retrieve datasets online

I like the content of this book and feel that it plays an important part in this niche market but my major qualm is that the code is written solely in Ruby in this book. While that might appeal to a certain market, to only have this communication in Ruby and/or not use a more traditional language of the day I feel is a major mistake. For this reason alone I knock a star off but still recommend it to anyone looking to learn or use these incredibly cool technologies provided by amazon.

1 Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again
7 of 8 people found the following review helpful By Sundar Raman on July 2, 2008
Format: Paperback
Excellent resource, but a bit droll. The content is laid out well, there are plenty of (working) examples, and there's pretty much no fluff to the book at all (in contrast to many O'Reilly books which add a fair amount of humor and distraction).
My chief worry when I received the book was that the title on the spine said "Programming Amazon Web Servcies [sic]". Yes, really the spine has a typo! The cover page does *not* have the typo. Obviously I was worried that the content might have similar brazen errors. But so far not so.
I'd recommend this book for anyone who needs an EC2/S3/AWS reference.
Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again
2 of 2 people found the following review helpful By Thomas J. Quinlan on May 11, 2008
Format: Paperback Verified Purchase
I'd have rated this a 4.5 if I could have.

This is an excellent book covering a very new subject matter. My only major complaint is that it seems a little rushed - I've found several typos, and even one section where a couple of lines of (important for that section) code are missing. (I figured out what was missing as I'm sure most people will.)

Also, the book is out of date. However, that is not the fault of the author or the publisher! It is that Amazon's service changes so quickly. The author and the publisher have made every attempt to mention the most recent changes to the service as of the time of writing, including pointing to places on the web to find out more information.

The material it covers is spot on. It goes through the different services that Amazon offers - including their storage, elastic computing, payment systems, and database systems. It clearly explains the disadvantages and advantages of each system, and provides -useful- code examples (in ruby) of how one can take advantage of the services Amazon provides. (There are examples in other languages, like Python, that the author makes available on the book's website.) Each section is devoted to a service for the most part, and the book is very readable.

As I said, I'd have rated this book a 4.5 if I could have. Outside of the errors due to rushing, it's quite useful and quite informative. The code is easy to follow, and I've found it very handy for working with the Amazon Web Services.
Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again
7 of 10 people found the following review helpful By Reader from Texas on April 4, 2008
Format: Paperback
I am reading this book on Safari, where it appeared before the hard copy was available. It also avoided snail mail, and this allowed me to be the first reviewer.

The book is about one of the hottest offerings in the software and surely will be read by many people, if only because it is the first book on this subject. It goes through detailed examples and explains the code. You could, potentially, get most of the same information from the tutorials, excepting maybe the bugs and workarounds which are not for the polished company publications.

My only wish is that the examples were not written in Ruby, which today ranks 10th on the TIOBE rating. The explanation that "it is easy to read" seems weak. I would venture to say the author felt more comfortable in Ruby. The upshot was that I learned to read a new language and found out about some new techniques that are cross-language, such as XPath.

I would recommend this book if only because reading it will make you feel more cozy with AWS, before you go to the tutorials and code samples.

Chapter 3 is a good overview of pros and cons of AWS, which you can show to your management if they are uncomfortable with the cloud.

And good luck on the cloud adventure.
Comment Was this review helpful to you? Yes No Sending feedback...
Thank you for your feedback. If this review is inappropriate, please let us know.
Sorry, we failed to record your vote. Please try again

Set up an Amazon Giveaway

Amazon Giveaway allows you to run promotional giveaways in order to create buzz, reward your audience, and attract new followers and customers. Learn more
Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB
This item: Programming Amazon Web Services: S3, EC2, SQS, FPS, and SimpleDB
Price: $31.23
Ships from and sold by

What Other Items Do Customers Buy After Viewing This Item?