Amazon Elastic Compute Cloud User Guide for Linux API Version 2014-10-01 Amazon Elastic Compute Cloud User Guide for Linux Amazon Elastic Compute Cloud: User Guide for Linux Copyright © 2014 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. The following are trademarks of Amazon Web Services, Inc.: Amazon, Amazon Web Services Design, AWS, Amazon CloudFront, Cloudfront, CloudTrail, Amazon DevPay, DynamoDB, ElastiCache, Amazon EC2, Amazon Elastic Compute Cloud, Amazon Glacier, Kinesis, Kindle, Kindle Fire, AWS Marketplace Design, Mechanical Turk, Amazon Redshift, Amazon Route 53, Amazon S3, Amazon VPC. In addition, Amazon.com graphics, logos, page headers, button icons, scripts, and service names are trademarks, or trade dress of Amazon in the U.S. and/or other countries. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Amazon Elastic Compute Cloud User Guide for Linux Table of Contents What Is Amazon EC2?................................................................................................................... 1 Features of Amazon EC2........................................................................................................ 1 How to Get Started with Amazon EC2....................................................................................... 2 Related Services................................................................................................................... 2 Accessing Amazon EC2 ......................................................................................................... 3 Pricing for Amazon EC2 ......................................................................................................... 4 Instances and AMIs ............................................................................................................... 4 Instances ..................................................................................................................... 5 AMIs ........................................................................................................................... 6 Regions and Availability Zones ................................................................................................ 7 Region and Availability Zone Concepts.............................................................................. 7 Describing Your Regions and Availability Zones.................................................................. 9 Specifying the Region for a Resource.............................................................................. 11 Launching Instances in an Availability Zone...................................................................... 12 Migrating an Instance to Another Availability Zone............................................................. 13 Root Device Volume............................................................................................................. 14 Root Device Storage Concepts ...................................................................................... 14 Choosing an AMI by Root Device Type............................................................................ 16 Determining the Root Device Type of Your Instance............................................................ 17 Changing the Root Device Volume to Persist.................................................................... 17 Setting Up .................................................................................................................................. 20 Sign Up for AWS ................................................................................................................. 20 Create an IAM User ............................................................................................................. 21 Create a Key Pair ................................................................................................................ 22 Create a Virtual Private Cloud (VPC)....................................................................................... 24 Create a Security Group ....................................................................................................... 24 Getting Started............................................................................................................................ 26 Step 1: Launch an Instance................................................................................................... 27 Step 2: Connect to Your Instance............................................................................................ 28 Option 1: Connect Using Your Browser............................................................................ 29 Option 2: Connect from Windows Using PuTTY................................................................. 30 Option 3: Connect from Mac or Linux Using an SSH Client.................................................. 31 Step 3: Add a Volume........................................................................................................... 31 Step 4: Clean Up................................................................................................................. 34 Best Practices............................................................................................................................. 36 Tutorial: Installing a LAMP Web Server............................................................................................ 38 Tutorial: Hosting a WordPress Blog................................................................................................. 45 Amazon Machine Images.............................................................................................................. 53 Using an AMI...................................................................................................................... 53 Creating Your Own AMI......................................................................................................... 54 Buying, Sharing, and Selling AMIs.......................................................................................... 54 Deregistering Your AMI ......................................................................................................... 54 Amazon Linux ..................................................................................................................... 54 AMI Types .......................................................................................................................... 55 Launch Permissions..................................................................................................... 55 Storage for the Root Device........................................................................................... 55 Virtualization Types .............................................................................................................. 58 Finding a Linux AMI ............................................................................................................. 58 Finding a Linux AMI Using the Amazon EC2 Console......................................................... 59 Finding an AMI Using the Command Line........................................................................ 59 Shared AMIs....................................................................................................................... 60 Finding Shared AMIs.................................................................................................... 60 Making an AMI Public................................................................................................... 63 Sharing an AMI with Specific AWS Accounts.................................................................... 64 Using Bookmarks ........................................................................................................ 66 API Version 2014-10-01 iii Amazon Elastic Compute Cloud User Guide for Linux Guidelines for Shared Linux AMIs................................................................................... 66 Paid AMIs........................................................................................................................... 70 Selling Your AMI .......................................................................................................... 70 Finding a Paid AMI....................................................................................................... 71 Purchase a Paid AMI.................................................................................................... 71 Getting the Product Code for Your Instance...................................................................... 72 Using Paid Support...................................................................................................... 72 Bills for Paid and Supported AMIs................................................................................... 73 Managing Your AWS Marketplace Subscriptions................................................................ 73 Creating an Amazon EBS-Backed Linux AMI............................................................................ 74 Overview of the Creation Process for Amazon EBS-Backed AMIs........................................ 74 Creating the AMI from an Instance.................................................................................. 75 Creating an AMI from a Snapshot................................................................................... 76 Creating an Instance Store-Backed Linux AMI.......................................................................... 77 Overview of the Creation Process for Instance Store-Backed AMIs....................................... 77 Prerequisites .............................................................................................................. 78 Creating an AMI from an Instance Store-Backed Linux Instance........................................... 78 Converting your Instance Store-Backed AMI to an Amazon EBS-Backed AMI......................... 82 Copying an AMI................................................................................................................... 85 AMI Copy ................................................................................................................... 85 Copying an Amazon EC2 AMI........................................................................................ 86 Copying an Amazon EC2 AMI with Encrypted Volumes...................................................... 87 Stopping a Pending AMI Copy Operation......................................................................... 89 Deregistering Your AMI ......................................................................................................... 90 Cleaning Up Your Amazon EBS-Backed AMI.................................................................... 90 Cleaning Up Your Instance Store-Backed AMI................................................................... 91 Amazon Linux ..................................................................................................................... 91 Finding the Amazon Linux AMI....................................................................................... 92 Launching and Connecting to an Amazon Linux Instance.................................................... 92 Identifying Amazon Linux AMI Images............................................................................. 92 Included AWS Command Line Tools................................................................................ 93 cloud-init .............................................................................................................. 94 Repository Configuration............................................................................................... 95 Adding Packages......................................................................................................... 96 Accessing Source Packages for Reference ...................................................................... 96 Developing Applications................................................................................................ 97 Instance Store Access.................................................................................................. 97 Product Life Cycle........................................................................................................ 97 Security Updates ......................................................................................................... 97 Support...................................................................................................................... 98 PV-GRUB........................................................................................................................... 98 Limitations of PV-GRUB................................................................................................ 99 Configuring GRUB ....................................................................................................... 99 Amazon PV-GRUB Kernel Image IDs............................................................................. 100 Updating PV-GRUB .................................................................................................... 102 Instances.................................................................................................................................. 103 Instance Types .................................................................................................................. 103 Available Instance Types ............................................................................................. 104 Hardware Specifications.............................................................................................. 105 T2 Instances ............................................................................................................. 105 I2 Instances .............................................................................................................. 109 HI1 Instances............................................................................................................ 110 HS1 Instances........................................................................................................... 112 R3 Instances............................................................................................................. 113 GPU Instances .......................................................................................................... 115 T1 Micro Instances..................................................................................................... 118 EBS-Optimized Instances............................................................................................ 125 Placement Groups ..................................................................................................... 126 API Version 2014-10-01 iv Amazon Elastic Compute Cloud User Guide for Linux Resizing Instances ..................................................................................................... 128 Spot Instances .................................................................................................................. 132 Quick Look: Getting Started with Spot Instances Video..................................................... 132 Checklist for Getting Started with Spot Instances............................................................. 132 Getting Started with Spot Instances .............................................................................. 133 Fundamentals of Spot Instances................................................................................... 145 Walkthroughs: Using Spot Instances with AWS Services................................................... 161 Advanced Tasks......................................................................................................... 180 Reserved Instances............................................................................................................ 185 Reserved Instance Overview........................................................................................ 185 Getting Started with Reserved Instances........................................................................ 186 Reserved Instance Fundamentals................................................................................. 191 Buying Reserved Instances ......................................................................................... 202 Obtaining Information About Your Reserved Instances...................................................... 213 Modifying Your Reserved Instances............................................................................... 218 Selling in the Reserved Instance Marketplace................................................................. 228 Requirements Checklist for Reserved Instances.............................................................. 254 Instance Metadata and User Data......................................................................................... 256 Retrieving Instance Metadata....................................................................................... 257 Retrieving User Data .................................................................................................. 259 Retrieving Dynamic Data............................................................................................. 260 Example: AMI Launch Index Value................................................................................ 260 Instance Metadata Categories...................................................................................... 262 Importing and Exporting Instances........................................................................................ 267 Prerequisites............................................................................................................. 268 Importing a VM into Amazon EC2................................................................................. 270 Exporting Amazon EC2 Instances................................................................................. 279 Troubleshooting ......................................................................................................... 280 Instance Lifecycle ...................................................................................................................... 286 Instance Launch ................................................................................................................ 286 Instance Stop and Start (Amazon EBS-backed instances only).................................................. 287 Instance Reboot ................................................................................................................ 287 Instance Retirement ........................................................................................................... 287 Instance Termination .......................................................................................................... 288 Differences Between Reboot, Stop, and Terminate................................................................... 288 Launch............................................................................................................................. 289 Launching an Instance................................................................................................ 290 Launching an Instance From an Existing Instance............................................................ 295 Launching a Linux Instance from a Backup..................................................................... 296 Launching an AWS Marketplace Instance....................................................................... 297 Connect ........................................................................................................................... 299 Connect Using SSH ................................................................................................... 299 Connect Using PuTTY ................................................................................................ 303 Connect Using MindTerm ............................................................................................ 308 Stop and Start................................................................................................................... 309 Overview .................................................................................................................. 309 Stopping and Starting Your Instances ............................................................................ 310 Modifying a Stopped Instance ...................................................................................... 311 Troubleshooting ......................................................................................................... 312 Reboot............................................................................................................................. 312 Retire............................................................................................................................... 312 Identifying Instances Scheduled for Retirement............................................................... 313 Working with Instances Scheduled for Retirement............................................................ 313 Terminate ......................................................................................................................... 315 Instance Termination .................................................................................................. 315 Terminating an Instance.............................................................................................. 315 Enabling Termination Protection.................................................................................... 316 Changing the Shutdown Behavior................................................................................. 317 API Version 2014-10-01 v Amazon Elastic Compute Cloud User Guide for Linux Preserving Amazon EBS Volumes on Instance Termination............................................... 318 Troubleshooting ......................................................................................................... 320 Configure Instances ................................................................................................................... 321 Common Configuration Scenarios......................................................................................... 321 Managing Software ............................................................................................................ 322 Updating Instance Software......................................................................................... 322 Adding Repositories ................................................................................................... 326 Finding Software Packages.......................................................................................... 327 Installing Software Packages........................................................................................ 328 Preparing to Compile Software..................................................................................... 329 Managing Users ................................................................................................................ 330 Setting the Time................................................................................................................. 332 Changing the Time Zone............................................................................................. 332 Configuring Network Time Protocol (NTP)...................................................................... 333 Changing the Hostname...................................................................................................... 335 Changing the System Hostname................................................................................... 335 Changing the Shell Prompt Without Affecting the Hostname.............................................. 336 Setting Up Dynamic DNS .................................................................................................... 337 Running Commands at Launch ............................................................................................ 339 Prerequisites............................................................................................................. 340 User Data and Shell Scripts......................................................................................... 340 User Data and cloud-init Directives................................................................................ 341 Monitoring ................................................................................................................................ 343 Automated and Manual Monitoring........................................................................................ 344 Automated Monitoring Tools......................................................................................... 344 Manual Monitoring Tools.............................................................................................. 345 Best Practices for Monitoring................................................................................................ 346 Monitoring the Status of Your Instances.................................................................................. 346 Monitoring Instances with Status Checks ....................................................................... 346 Monitoring Events for Your Instances............................................................................. 351 Monitoring Your Instances with CloudWatch............................................................................ 354 Enabling or Disabling Detailed Monitoring on an Amazon EC2 Instance............................... 355 View Amazon EC2 Metrics .......................................................................................... 358 Get Statistics for Metrics ............................................................................................. 364 Graphing Metrics ....................................................................................................... 380 Create a CloudWatch Alarm......................................................................................... 384 Create Alarms That Stop or Terminate an Instance........................................................... 391 Monitoring Scripts for Amazon EC2 Instances......................................................................... 405 Amazon CloudWatch Monitoring Scripts for Linux............................................................ 405 Network and Security ................................................................................................................. 412 Key Pairs.......................................................................................................................... 413 Creating Your Key Pair Using Amazon EC2..................................................................... 413 Importing Your Own Key Pair to Amazon EC2.................................................................. 415 Retrieving the Public Key for Your Key Pair...................................................................... 416 Verifying Your Key Pair's Fingerprint .............................................................................. 417 Deleting Your Key Pair................................................................................................. 418 Connecting to Your Linux Instance if You Lose Your Private Key.......................................... 418 Security Groups................................................................................................................. 421 Security Groups for EC2-Classic................................................................................... 422 Security Groups for EC2-VPC...................................................................................... 422 Security Group Rules ................................................................................................. 423 Default Security Groups .............................................................................................. 424 Custom Security Groups ............................................................................................. 424 Creating a Security Group........................................................................................... 425 Describing Your Security Groups................................................................................... 426 Adding Rules to a Security Group................................................................................. 426 Deleting Rules from a Security Group............................................................................ 427 Deleting a Security Group............................................................................................ 428 API Version 2014-10-01 vi Amazon Elastic Compute Cloud User Guide for Linux API and Command Overview....................................................................................... 428 Controlling Access ............................................................................................................. 429 Network Access to Your Instance.................................................................................. 429 Amazon EC2 Permission Attributes............................................................................... 429 IAM and Amazon EC2 ................................................................................................ 430 IAM Policies .............................................................................................................. 431 IAM Roles................................................................................................................. 461 Network Access......................................................................................................... 467 Amazon VPC .................................................................................................................... 468 Benefits of Using a VPC.............................................................................................. 468 Differences Between EC2-Classic and EC2-VPC............................................................. 482 Amazon VPC Documentation....................................................................................... 470 Supported Platforms................................................................................................... 471 Migrating from EC2-Classic to a VPC............................................................................ 473 Instance IP Addressing ....................................................................................................... 480 Private IP Addresses and Internal DNS Hostnames......................................................... 480 Public IP Addresses and External DNS Hostnames......................................................... 481 Elastic IP Addresses................................................................................................... 482 Differences Between EC2-Classic and EC2-VPC............................................................. 482 Determining Your Public, Private, and Elastic IP Addresses............................................... 482 Assigning a Public IP Address...................................................................................... 484 Multiple Private IP Addresses....................................................................................... 485 Elastic IP Addresses........................................................................................................... 489 Elastic IP Addresses in EC2-Classic.............................................................................. 490 Elastic IP Addresses in a VPC...................................................................................... 490 Differences Between EC2-Classic and EC2-VPC............................................................. 491 Allocating an Elastic IP Address ................................................................................... 491 Describing Your Elastic IP Addresses............................................................................. 492 Associating an Elastic IP Address with a Running Instance............................................... 492 Associating an Elastic IP Address with a Different Running Instance................................... 492 Releasing an Elastic IP Address................................................................................... 493 Using Reverse DNS for Email Applications..................................................................... 494 Elastic IP Address Limit .............................................................................................. 494 Elastic Network Interfaces ................................................................................................... 494 Private IP Addresses Per ENI Per Instance Type.............................................................. 495 Creating a Management Network.................................................................................. 497 Use Network and Security Appliances in Your VPC.......................................................... 497 Creating Dual-homed Instances with Workloads/Roles on Distinct Subnets.......................... 497 Create a Low Budget High Availability Solution................................................................ 498 Best Practices for Configuring Network Interfaces............................................................ 498 Configuring Your Network Interface Using ec2-net-utils..................................................... 498 Creating a Network Interface........................................................................................ 499 Deleting a Network Interface........................................................................................ 500 Viewing Details about a Network Interface...................................................................... 500 Attaching a Network Interface When Launching an Instance.............................................. 501 Attaching a Network Interface to a Stopped or Running Instance........................................ 502 Detaching a Network Interface from an Instance.............................................................. 502 Changing the Security Group of a Network Interface........................................................ 503 Changing the Source/Destination Checking of a Network Interface..................................... 504 Associating an Elastic IP Address with a Network Interface............................................... 504 Disassociating an Elastic IP Address from a Network Interface........................................... 505 Changing Termination Behavior for a Network Interface.................................................... 505 Adding or Editing a Description for a Network Interface..................................................... 506 Adding or Editing Tags for a Network Interface................................................................ 506 Enhanced Networking......................................................................................................... 507 Requirements............................................................................................................ 507 Testing Whether Enhanced Networking Is Enabled........................................................... 508 Enabling Enhanced Networking on Amazon Linux........................................................... 510 API Version 2014-10-01 vii Amazon Elastic Compute Cloud User Guide for Linux Enabling Enhanced Networking on Ubuntu..................................................................... 512 Enabling Enhanced Networking on Other Linux Distributions............................................. 515 Troubleshooting Connectivity Issues.............................................................................. 517 Storage .................................................................................................................................... 518 Amazon EBS .................................................................................................................... 519 Features of Amazon EBS............................................................................................ 520 EBS Volumes ............................................................................................................ 521 EBS Snapshots ......................................................................................................... 559 EBS Encryption ......................................................................................................... 564 EBS Performance ...................................................................................................... 567 API and Command Overview....................................................................................... 578 Instance Store................................................................................................................... 581 Instance Storage Concepts ......................................................................................... 581 Instance Stores Available on Instance Types................................................................... 582 Instance Store Swap Volumes...................................................................................... 584 Instance Store Device Names...................................................................................... 586 Instance Store Usage Scenarios .................................................................................. 586 Adding Instance Store Volumes to an AMI...................................................................... 588 Optimizing Disk Performance ....................................................................................... 589 Amazon S3....................................................................................................................... 589 Amazon S3 and Amazon EC2...................................................................................... 590 Block Device Mapping ........................................................................................................ 591 Block Device Mapping Concepts................................................................................... 592 AMI Block Device Mapping.......................................................................................... 595 Instance Block Device Mapping.................................................................................... 597 Using Public Data Sets ....................................................................................................... 602 Public Data Set Concepts............................................................................................ 602 Finding Public Data Sets............................................................................................. 602 Creating a Public Data Set Volume from a Snapshot........................................................ 603 Attaching and Mounting the Public Data Set Volume......................................................... 604 Resources and Tags................................................................................................................... 605 Resource Locations............................................................................................................ 605 Listing and Filtering Your Resources...................................................................................... 606 Advanced Search....................................................................................................... 607 Listing Resources Using the Console ............................................................................ 608 Filtering Resources Using the Console.......................................................................... 608 Listing and Filtering Using the CLI and API..................................................................... 609 Tagging Your Resources...................................................................................................... 610 Tag Basics................................................................................................................ 610 Tag Restrictions......................................................................................................... 611 Tagging Your Resources for Billing ................................................................................ 612 Working with Tags in the Console.................................................................................. 612 API and CLI Overview................................................................................................. 617 Service Limits.................................................................................................................... 618 Viewing Your Current Limits ......................................................................................... 618 Requesting a Limit Increase......................................................................................... 619 Usage Reports .................................................................................................................. 619 Available Reports....................................................................................................... 619 Getting Set Up for Usage Reports................................................................................. 619 Granting IAM Users Access to the Amazon EC2 Usage Reports........................................ 621 Instance Usage ......................................................................................................... 621 Reserved Instance Utilization....................................................................................... 625 Troubleshooting ......................................................................................................................... 631 Launching Your Instance ..................................................................................................... 631 Getting the Reason for Instance Termination................................................................... 632 Connecting to Your Instance................................................................................................. 632 Error connecting to your instance: Connection timed out................................................... 633 Error: User key not recognized by server........................................................................ 634 API Version 2014-10-01 viii Amazon Elastic Compute Cloud User Guide for Linux Error: Host key not found, Permission denied (publickey), or Authentication failed, permission denied ..................................................................................................................... 635 Error: Unprotected Private Key File ............................................................................... 636 Error: Server refused our key or No supported authentication methods available................... 636 Error using MindTerm on Safari Browser........................................................................ 637 Error Using Mac OS X RDP Client................................................................................ 637 Stopping Your Instance........................................................................................................ 637 Terminating Your Instance.................................................................................................... 638 Delayed Instance Termination....................................................................................... 638 Automatically Launch or Terminate Instances.................................................................. 639 Failed Status Checks.......................................................................................................... 639 Initial Steps You Can Take............................................................................................ 639 Troubleshooting Instance Status Checks for Linux-Based Instances.................................... 640 Out of memory: kill process.......................................................................................... 641 ERROR: mmu_update failed (Memory management update failed)..................................... 641 I/O error (Block device failure)...................................................................................... 642 IO ERROR: neither local nor remote disk (Broken distributed block device).......................... 643 request_module: runaway loop modprobe (Looping legacy kernel modprobe on older Linux versions) .................................................................................................................. 644 "FATAL: kernel too old" and "fsck: No such file or directory while trying to open /dev" (Kernel and AMI mismatch) ................................................................................................... 645 "FATAL: Could not load /lib/modules" or "BusyBox" (Missing kernel modules)........................ 646 ERROR Invalid kernel (EC2 incompatible kernel)............................................................. 647 request_module: runaway loop modprobe (Looping legacy kernel modprobe on older Linux versions) .................................................................................................................. 648 fsck: No such file or directory while trying to open... (File system not found).......................... 649 General error mounting filesystems (Failed mount)........................................................... 651 VFS: Unable to mount root fs on unknown-block (Root filesystem mismatch)........................ 653 Error: Unable to determine major/minor number of root device... (Root file system/device mis- match) .................................................................................................................... 654 XENBUS: Device with no driver..................................................................................... 655 ... days without being checked, check forced (File system check required)............................ 656 fsck died with exit status... (Missing device)..................................................................... 657 GRUB prompt (grubdom>)........................................................................................... 658 Bringing up interface eth0: Device eth0 has different MAC address than expected, ignoring. (Hard-coded MAC address) ........................................................................................ 660 Unable to load SELinux Policy. Machine is in enforcing mode. Halting now. (SELinux miscon- figuration) ................................................................................................................. 661 XENBUS:Timeout connecting to devices (Xenbus timeout)............................................... 661 Instance Capacity .............................................................................................................. 662 Error: InsufficientInstanceCapacity ................................................................................ 662 Error: InstanceLimitExceeded....................................................................................... 662 General ............................................................................................................................ 663 Instance Reboot ........................................................................................................ 663 Instance Console Output............................................................................................. 663 Instance Recovery When its Host Computer Fails............................................................ 664 Making API Requests ................................................................................................................. 665 Required Knowledge .......................................................................................................... 665 Available APIs for Amazon EC2............................................................................................ 665 Query Requests ................................................................................................................ 666 Structure of a GET Request......................................................................................... 667 Endpoints ................................................................................................................. 668 Query Parameters...................................................................................................... 668 Query API Authentication ............................................................................................ 668 Query Response Structures......................................................................................... 669 Troubleshooting API Request Errors...................................................................................... 670 Query API Request Rate............................................................................................. 670 Eventual Consistency ................................................................................................. 671 API Version 2014-10-01 ix Amazon Elastic Compute Cloud User Guide for Linux Unauthorized Operation .............................................................................................. 671 Ensuring Idempotency ........................................................................................................ 672 Idempotency Support ................................................................................................. 673 Example Idempotent Command.................................................................................... 673 Example Idempotent Query ......................................................................................... 674 SOAP Requests ................................................................................................................ 674 Logging API Calls .............................................................................................................. 674 Amazon EC2 Information in CloudTrail........................................................................... 675 Understanding Amazon EC2 Log File Entries.................................................................. 675 Document History ...................................................................................................................... 677 AWS Glossary........................................................................................................................... 688 API Version 2014-10-01 x