Amazon Simple Storage Service Developer Guide API Version 2006-03-01 AmazonSimpleStorageServiceDeveloperGuide AmazonSimpleStorageService:DeveloperGuide Copyright©2008AmazonWebServicesLLCoritsaffiliates.Allrightsreserved. AmazonSimpleStorageServiceDeveloperGuide Table of Contents What'sNew......................................................................................................................................1 WelcometoAmazonS3..................................................................................................................3 IntroductiontoAmazonS3..............................................................................................................6 CoreConcepts..................................................................................................................................7 ComponentsofAmazonS3..........................................................................................7 Operations.....................................................................................................................8 AmazonS3ApplicationProgrammingInterfaces(API)..............................................8 AmazonS3DataConsistencyModel...........................................................................9 PayingforAmazonS3................................................................................................10 UsingAmazonS3..........................................................................................................................11 WorkingwithAmazonS3Components.....................................................................11 WorkingwithAmazonS3Buckets...................................................................11 BucketRestrictionsandLimitations........................................................12 BucketConfigurationOptions.................................................................13 BucketsandAccessControl....................................................................14 BillingandReportingofBuckets............................................................14 WorkingwithAmazonS3Objects....................................................................14 Keys.........................................................................................................15 ListingKeys...................................................................................15 CommonListRequestParameters.................................................16 CommonListResponseElements.................................................16 IteratingThroughMulti-PageResults............................................18 ListingKeysHierarchicallyusingPrefixandDelimiter................19 Metadata..................................................................................................19 GettingObjects........................................................................................20 StandardDownloads......................................................................20 ChunkedandResumableDownloads............................................21 AuthenticationandAccessControl............................................................................21 Authentication...................................................................................................22 AccessControlLists..........................................................................................25 QueryStringAuthentication.............................................................................30 RequestRouting..........................................................................................................31 RequestRedirectionandtheRESTAPI...........................................................31 DNSConsiderations..........................................................................................36 PerformanceOptimization..........................................................................................36 TCPWindowScaling........................................................................................36 TCPSelectiveAcknowledgement.....................................................................37 UsingAmazonDevPaywithAmazonS3...................................................................37 AmazonS3CustomerDataIsolation................................................................37 AmazonDevPayTokenMechanism.................................................................38 AmazonS3andAmazonDevPayAuthentication............................................39 AmazonS3BucketLimitation..........................................................................39 AmazonS3andAmazonDevPayProcess........................................................40 AdditionalInformation......................................................................................40 WorkingwithErrors...................................................................................................40 AmazonS3ErrorBestPractices.......................................................................40 ErrorResponse..................................................................................................41 ErrorCode...............................................................................................41 ErrorMessage..........................................................................................42 FurtherDetails.........................................................................................42 ListofErrorCodes..................................................................................42 ServerAccessLogging...............................................................................................46 ServerAccessLoggingConfigurationAPI.......................................................47 AmazonSimpleStorageServiceDeveloperGuide DeliveryofServerAccessLogs........................................................................49 ServerAccessLogFormat................................................................................50 SettingUpServerAccessLogging...................................................................53 UsingtheRESTAPI......................................................................................................................57 CommonRESTAPIElements...................................................................................58 TheRESTErrorResponse..........................................................................................59 AuthenticatingRESTRequests..................................................................................60 SettingAccessPolicywithREST...............................................................................70 VirtualHostingofBuckets.........................................................................................71 RequestRedirectionandtheRESTAPI.....................................................................74 Browser-BasedUploadsUsingPOST........................................................................77 Introduction.......................................................................................................77 HTMLForms....................................................................................................78 Examples...........................................................................................................86 POSTwithAdobeFlash....................................................................................94 OperationsontheService...........................................................................................95 GETOperation..................................................................................................95 OperationsonBuckets................................................................................................96 PUTBucket.......................................................................................................97 GETBucket.......................................................................................................98 GETBucketLocation......................................................................................100 DELETEBucket.............................................................................................100 POSTObject...................................................................................................101 OperationsonObjects...............................................................................................107 PUTObject......................................................................................................107 GETObject.....................................................................................................109 HEADObject..................................................................................................114 DELETEObject..............................................................................................114 UsingtheSOAPAPI....................................................................................................................116 CommonSOAPAPIElements.................................................................................116 TheSOAPErrorResponse.......................................................................................117 AuthenticatingSOAPRequests................................................................................117 SettingAccessPolicywithSOAP............................................................................118 OperationsontheService.........................................................................................119 ListAllMyBuckets...........................................................................................119 OperationsonBuckets..............................................................................................120 CreateBucket...................................................................................................121 DeleteBucket...................................................................................................121 ListBucket.......................................................................................................122 GetBucketAccessControlPolicy......................................................................124 SetBucketAccessControlPolicy.......................................................................124 GetBucketLoggingStatus................................................................................125 SetBucketLoggingStatus.................................................................................126 OperationsonObjects...............................................................................................127 PutObjectInline................................................................................................127 PutObject.........................................................................................................129 GetObject........................................................................................................131 GetObjectExtended.........................................................................................136 DeleteObject....................................................................................................136 GetObjectAccessControlPolicy.......................................................................137 SetObjectAccessControlPolicy.......................................................................138 UsingBitTorrent™withAmazonS3..........................................................................................140 HowYouareChargedforBitTorrentDelivery........................................................140 UsingBitTorrenttoRetrieveObjectsStoredinAmazonS3....................................141 PublishingContentUsingAmazonS3andBitTorrent.............................................141 Glossary.......................................................................................................................................143 DocumentConventions................................................................................................................144 Index.............................................................................................................................................147 AmazonSimpleStorageServiceDeveloperGuide What's New ThisWhat'sNewisassociatedwiththe2006-03-01releaseofAmazonS3.Thisguidewaslastupdated onApril09,2008. ThefollowingtabledescribestheimportantchangessincethelastreleaseoftheAmazonS3Developer Guide. Change Description ReleaseDate LoggingChanges AmazonS3nowenablesyoutoautomaticallygrantaccessto 7April2008 logswithinabuckettousersotherthanthebucketowner.For moreinformation,seeSettingUpServerAccessLogging. TCPWindowScaling AmazonS3nowsupportsTCPwindowscalingandTCP 3March2008 selectiveacknowledgementwhichenablesyoutooptimize networkperformance.Formoreinformation,seePerformance Optimization. Chunkedand Theguidewasupdatedtodescribehowtoperformchunked 11January Resumable andresumabledownloads.Formoreinformation,see 2008 Downloads ChunkedandResumableDownloads. HTTPPOSTchanges Theredirectfieldwaschangedtosuccess_action_redirectand 31December thesuccess_action_statusfieldwasadded.Formore 2007 information,seeBrowser-BasedUploadsUsingPOST. DevPay AmazonDevPayenablesyoutochargecustomersforusing 18December yourAmazonS3productthroughAmazon'sauthentication 2007 andbillinginfrastructure.Youcanchargeanyamountfor yourproductincludingusagecharges(storage,transactions, andbandwidth),monthlyfixedcharges,andaone-time charge.Formoreinformation,seeUsingAmazonDevPay withAmazonS3. HTTPPOST AmazonS3nowsupportsbrowser-baseduploadsusing 17December POST,whichallowsyouruserstouploadcontentdirectlyto 2007 AmazonS3.Formoreinformation,seeBrowser-Based UploadsUsingPOST. Restructuringand Theintroductorysectionsofthedocumentwererestructured 17December VariousEdits andnumerouseditsweremadebasedoncustomerinputfrom 2007 APIVersion2006-03-01 1 AmazonSimpleStorageServiceDeveloperGuide Change Description ReleaseDate theFeedbacklinkandforums. LocationConstraints AmazonS3nowsupportslocationconstraints,whichallow 15October youtospecifywheretostoredata.Formoreinformation,see 2007 LocationSelection. SupportforRedirects IfDNSinformationforabucketisnotpropagatedthroughout 15October theInternet,clientswillreceivea307redirect.Ifyouattempt 2007 useapath-stylerequesttoaccessanobjectwithinabucket thatwascreatedusing<CreateBucketConfiguration>, youwillreceiveapermanent301redirect.Formore informationonredirects,soyoucanoptimizeyourcode,see LocationSelection. BucketLocation AmazonS3supportsanewoperationforgettingthelocation 15October ofabucket.Formoreinformation,seeGETBucketLocation. 2007 NewAuthentication Theauthenticationsectionwasrewrittentoclarifyquestions 10September Section thatappearedintheforums.Formoreinformation,see 2007 AuthenticationandAccessControl. Feedback Youcannowprovidefeedbackcommentsonanytopicinthe 10September HTMLversionofthisguide.Toprovidefeedback,simply 2007 clicktheFeedbacklinkatthetopofthepage. MinorEdits Minoreditsweremadethroughoutthedocumenttoclarify 10September issuesthatappearedintheforumsandtoimproveoverall 2007 documentquality. AmazonDevPay AmazonDevPayisanewAmazonservicethatenablesyouto 10September chargecustomersforuseofyourAmazonS3productthrough 2007 theAmazonauthenticationandbillinginfrastructure.For moreinformation,seeUsingAmazonDevPaywith AmazonS3. NewBucketLimit Inadditiontothe100bucketlimitassociatedwithyourAWS 10September account,eachofyourcustomerscanhaveupto100buckets 2007 foreachAmazonDevPayproductthatyousell.Formore information,seeUsingAmazonDevPaywithAmazonS3. APIVersion2006-03-01 2 AmazonSimpleStorageServiceDeveloperGuide Audience Welcome to Amazon S3 Topics • Audience • HowThisGuideIsOrganized • RelatedResources ThankyouforyourinterestinAmazonS3. Thissectiondescribeswhoshouldreadthisguide,howtheguideisorganized,andotherresources relatedtoAmazonS3. AmazonS3willoccasionallybereferredtowithinthisguideassimply"S3";allcopyrightsandlegal protectionsstillapply. Wehopeyoufindtheservicetobeeasy-to-use,reliable,andinexpensive.Ifyouwanttoprovide feedbacktotheAmazonS3developmentteam,pleasepostamessagetotheAmazonS3Developer Forum. Audience ThisguidedescribestheAmazonS3interfacesandfunctionalityindetailandisintendedfordevelopers whoarebuildingapplicationandservicesthatneedtostoreandretrieveanyamountofdata,atanytime, fromanywhereontheweb. Required Knowledge and Skills Useofthisguideassumesyouarefamiliarwiththefollowing: • XML(SeeW3SchoolsXMLTutorial) • Basicunderstandingofwebservices(SeeW3SchoolsWebServicesTutorial)) • Aprogramminglanguageforconsumingawebserviceandanyrelatedtools YoushouldalsohavereadtheAmazonS3GettingStartedGuide. APIVersion2006-03-01 3 AmazonSimpleStorageServiceDeveloperGuide RelatedResources How This Guide Is Organized Thisguideisorganizedintoseveralmajorsectionsdescribedinthetablebelow. Information RelevantSections GeneralinformationaboutAmazonS3 • IntroductiontoAmazonS3 Conceptualinformationabout • CoreConcepts AmazonS3 InformationaboutusingAmazonS3 • UsingAmazonS3 APIInformation • UsingtheRESTAPI • UsingtheSOAPAPI InformationaboutBitTorrent • UsingBitTorrent™withAmazonS3 Typographicandsymbolconventions • DocumentConventions Eachsectioniswrittentostandonitsown,soyoushouldbeabletolookuptheinformationyouneed andgobacktowork.However,youcanalsoreadthroughthemajorsectionssequentiallytogetin-depth knowledgeabouttheAmazonS3. Related Resources Thetablebelowlistsrelatedresourcesthatyou'llfindusefulasyouworkwiththisservice. Resource Description AmazonS3GettingStartedGuide TheGettingStartedGuideprovidesaquicktutorialofthe servicebasedonasimpleusecase.Examplesand instructionsforJava,Perl,PHP,C#,Python,andRubyare included. AmazonS3DeveloperGuide TheDeveloperGuide(whichyouarereading)providesa detaileddiscussionoftheservice.Itincludesanoverview, programmingreference,andAPIreference. AmazonS3ReleaseNotes TheReleaseNotesgiveahigh-leveloverviewofthecurrent release.Theyspecificallynoteanynewfeatures,corrections, andknownissues. AWSDeveloperResourceCenter Acentralstartingpointtofinddocumentation,code samples,releasenotes,andotherinformationtohelpyou buildinnovativeapplicationswithAWS. APIVersion2006-03-01 4 AmazonSimpleStorageServiceDeveloperGuide RelatedResources Resource Description AmazonS3productinformation TheprimarywebpageforinformationaboutAmazonS3. DiscussionForums Acommunity-basedforumfordeveloperstodiscuss technicalquestionsrelatedtoAmazonWebServices. E-mailaddressforquestionsrelatedto Thise-mailaddressisonlyforaccountquestions.For yourAWSaccount: technicalquestions,usetheDiscussionForums. <[email protected]> APIVersion2006-03-01 5 AmazonSimpleStorageServiceDeveloperGuide OverviewofAmazonS3 Introduction to Amazon S3 Topics • OverviewofAmazonS3 • ServiceFeatures ThisintroductiontoAmazonS3isintendedtogiveyouadetailedsummaryofthiswebservice.After readingthissection,youshouldhaveagoodideaofwhatitoffersandhowitcanfitinwithyour business. Overview of Amazon S3 AmazonS3isstoragefortheInternet.Itisdesignedtomakeweb-scalecomputingeasierfordevelopers. AmazonS3hasasimplewebservicesinterfacethatcanbeusedtostoreandretrieveanyamountof data,atanytime,fromanywhereontheweb.Itgivesanydeveloperaccesstothesamehighlyscalable, reliable,fast,inexpensivedatastorageinfrastructurethatAmazonusestorunitsownglobalnetworkof websites.Theserviceaimstomaximizebenefitsofscaleandtopassthosebenefitstodevelopers. Service Features AmazonS3isintentionallybuiltwithaminimalfeaturesetthatfocusesonsimplicityand robustness.ThefollowingaresomeoffeaturesoftheAmazonS3service: • Write,read,anddeleteobjectsfrom1byteto5gigabytesinsizewithaccompanyingmetadata.There isnofixedlimitonthenumberofobjectsyoucanstore. • Astraightforwardflatobjectstoremodel,whereeachobjectisstoredandretrievedusingaunique developer-assignedkey • Authenticationmechanismsareprovidedtoensurethatdataiskeptsecurefromunauthorizedaccess. Objectscanbemadeprivateorpublicandrightscanbegrantedtospecificusers. • Standards-basedRESTandSOAPinterfacesdesignedtoworkwithanyInternet-developmenttoolkit. APIVersion2006-03-01 6