Amazon Simple Workflow Service Developer Guide API Version 2012-01-25 Amazon Simple Workflow Service Developer Guide Amazon Simple Workflow Service: Developer Guide Copyright © 2022 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Amazon Simple Workflow Service Developer Guide Table of Contents What is Amazon SWF? ........................................................................................................................ 1 Development Options................................................................................................................. 1 AWS SDKs......................................................................................................................... 1 AWS Flow Framework ......................................................................................................... 2 HTTP Service API ............................................................................................................... 2 Development Environments................................................................................................. 2 Introduction to Amazon SWF ............................................................................................................... 4 Concepts................................................................................................................................... 4 Workflow Execution.................................................................................................................... 5 Setting Up........................................................................................................................................ 7 AWS Account and Access Keys ..................................................................................................... 7 Endpoints.................................................................................................................................. 8 Getting Started .................................................................................................................................. 9 About the Workflow................................................................................................................... 9 Prerequisites............................................................................................................................ 10 Tutorial Steps.......................................................................................................................... 10 Part 1: Using Amazon SWF with the SDK for Ruby ........................................................................ 10 Include the AWS SDK for Ruby ........................................................................................... 11 Configuring the AWS Session ............................................................................................. 11 Registering an Amazon SWF Domain .................................................................................. 12 Next Steps....................................................................................................................... 13 Part 2: Implementing the Workflow............................................................................................ 13 Designing the Workflow .................................................................................................... 13 Setting up our Workflow Code ........................................................................................... 14 Registering the Workflow .................................................................................................. 14 Polling for Decisions ......................................................................................................... 15 Starting the Workflow Execution ........................................................................................ 17 Next Steps....................................................................................................................... 19 Part 3: Implementing the Activities ............................................................................................ 19 Defining a Basic Activity Type ............................................................................................ 19 Defining GetContactActivity............................................................................................... 20 Defining SubscribeTopicActivity.......................................................................................... 22 Defining WaitForConfirmationActivity................................................................................. 24 Defining SendResultActivity............................................................................................... 26 Next Steps....................................................................................................................... 27 Part 4: Implementing the Activities Task Poller ............................................................................ 27 Running the Workflow.............................................................................................................. 29 Where Do I Go from Here? ................................................................................................ 32 Tutorials.......................................................................................................................................... 33 Registering a Domain ................................................................................................................ 33 Registering a Workflow Type ..................................................................................................... 35 Registering an Activity Type ...................................................................................................... 39 Starting a Workflow Execution ................................................................................................... 42 To start a workflow execution using the console .................................................................. 42 Viewing Pending Tasks.............................................................................................................. 45 Managing Your Workflow Executions .......................................................................................... 45 Managing Amazon SWF Domains............................................................................................... 51 How Amazon SWF Works .................................................................................................................. 52 Workflows............................................................................................................................... 52 What is a Workflow? ......................................................................................................... 52 A Simple Workflow Example: an E-Commerce Application ...................................................... 52 Workflow Registration and Execution .................................................................................. 53 See Also.......................................................................................................................... 54 Workflow History..................................................................................................................... 54 API Version 2012-01-25 iii Amazon Simple Workflow Service Developer Guide Actors..................................................................................................................................... 57 What is an Actor in Amazon SWF? ...................................................................................... 57 Workflow Starters............................................................................................................ 58 Deciders.......................................................................................................................... 58 Activity Workers............................................................................................................... 59 Data Exchange Between Actors .......................................................................................... 59 Tasks...................................................................................................................................... 59 Domains.................................................................................................................................. 60 Object Identifiers...................................................................................................................... 61 Task Lists................................................................................................................................ 61 Decision Task Lists ............................................................................................................ 61 Activity Task Lists ............................................................................................................. 62 Task Routing.................................................................................................................... 62 Workflow Execution Closure ...................................................................................................... 62 Life Cycle of an Amazon SWF Workflow Execution ........................................................................ 63 Workflow Execution Life Cycle ........................................................................................... 63 Polling for Tasks ...................................................................................................................... 67 Advanced Concepts.................................................................................................................. 67 Versioning....................................................................................................................... 67 Signals............................................................................................................................ 68 Child Workflows............................................................................................................... 68 Markers........................................................................................................................... 69 Tags................................................................................................................................ 70 Implementing Exclusive Choice .......................................................................................... 72 Timers............................................................................................................................. 74 Signals............................................................................................................................ 74 Activity Task Cancellation .................................................................................................. 75 Markers........................................................................................................................... 77 Security........................................................................................................................................... 79 Data Protection........................................................................................................................ 79 Encryption....................................................................................................................... 80 Identity and Access Management ............................................................................................... 80 Authentication................................................................................................................. 80 Access Control ................................................................................................................. 81 Basic Principles................................................................................................................ 82 Amazon SWF IAM Policies ................................................................................................. 82 API Summary................................................................................................................... 88 Tag-based Policies............................................................................................................ 93 Logging and Monitoring in Amazon Simple Workflow Service ........................................................ 93 Amazon SWF Metrics for CloudWatch ................................................................................. 94 Viewing Amazon SWF Metrics .......................................................................................... 100 Logging Amazon SWF API Calls with CloudTrail .................................................................. 103 EventBridge for Amazon SWF .......................................................................................... 108 Compliance Validation............................................................................................................. 114 Resilience.............................................................................................................................. 114 Infrastructure Security............................................................................................................. 115 Configuration and Vulnerability Analysis .................................................................................... 115 Using the AWS CLI ......................................................................................................................... 116 Working with APIs.......................................................................................................................... 117 Making HTTP Requests ............................................................................................................ 117 HTTP Header Contents .................................................................................................... 117 HTTP Body Content ........................................................................................................ 119 Sample JSON Request and Response ................................................................................ 119 Calculating the HMAC-SHA Signature ................................................................................ 120 List of Amazon SWF Actions .................................................................................................... 121 Actions Related to Activities ............................................................................................ 122 Actions Related to Deciders ............................................................................................. 122 API Version 2012-01-25 iv Amazon Simple Workflow Service Developer Guide Actions Related to Workflow Executions ............................................................................ 122 Actions Related to Administration ..................................................................................... 122 Visibility Actions............................................................................................................. 123 Creating a Basic Workflow ....................................................................................................... 124 Modeling Your Workflow and Its Activities ......................................................................... 124 Registering a Domain .............................................................................................................. 124 See Also........................................................................................................................ 125 Setting Timeout Values ........................................................................................................... 125 Quotas on Timeout Values ............................................................................................... 125 Workflow Execution and Decision Task Timeouts ................................................................ 125 Activity Task Timeouts .................................................................................................... 126 See Also........................................................................................................................ 126 Registering a Workflow Type ................................................................................................... 126 See Also........................................................................................................................ 127 Registering an Activity Type ..................................................................................................... 127 See Also........................................................................................................................ 127 Lambda Tasks........................................................................................................................ 127 About AWS Lambda ........................................................................................................ 128 Benefits and limitations of using Lambda tasks.................................................................. 128 Using Lambda tasks in your workflows .............................................................................. 128 Developing an Activity Worker ................................................................................................. 132 Polling for Activity Tasks ................................................................................................. 132 Performing the Activity Task ............................................................................................ 133 Reporting Activity Task Heartbeats ................................................................................... 133 Completing or Failing an Activity Task ............................................................................... 134 Launching Activity Workers .............................................................................................. 135 Developing Deciders............................................................................................................... 135 Defining Coordination Logic ............................................................................................. 136 Polling for Decision Tasks ................................................................................................ 136 Applying the Coordination Logic ...................................................................................... 137 Responding with Decisions .............................................................................................. 138 Closing a Workflow Execution .......................................................................................... 139 Launching Deciders......................................................................................................... 139 Starting Workflow Executions .................................................................................................. 140 Setting Task Priority ............................................................................................................... 141 Setting Task Priority for Workflows ................................................................................... 141 Setting Task Priority for Activities ..................................................................................... 143 Actions that Return Task Priority Information ..................................................................... 143 Handling Errors...................................................................................................................... 144 Validation Errors............................................................................................................. 144 Errors in Enacting Actions or Decisions .............................................................................. 144 Timeouts....................................................................................................................... 145 Errors raised by user code ............................................................................................... 145 Errors related to closing a workflow execution ................................................................... 145 Quotas.......................................................................................................................................... 146 General Account Quotas for Amazon SWF ................................................................................. 146 Quotas on Workflow Executions ............................................................................................... 147 Quotas on Task Executions ...................................................................................................... 147 Amazon SWF throttling quotas ................................................................................................ 148 Throttling quotas for all Regions ...................................................................................... 148 Decision quotas for all Regions ........................................................................................ 149 Workflow-level quotas.................................................................................................... 150 Requesting a Quota Increase .................................................................................................... 150 Resources...................................................................................................................................... 151 Timeout Types....................................................................................................................... 151 Timeouts in Workflow and Decision Tasks .......................................................................... 151 Timeouts in Activity Tasks ............................................................................................... 152 API Version 2012-01-25 v Amazon Simple Workflow Service Developer Guide Endpoints.............................................................................................................................. 154 Additional Documentation....................................................................................................... 154 Amazon Simple Workflow Service API Reference ................................................................ 154 AWS Flow Framework Documentation ............................................................................... 154 AWS SDK Documentation ................................................................................................ 154 AWS CLI Documentation .................................................................................................. 156 Web Resources....................................................................................................................... 156 Amazon SWF Forum ....................................................................................................... 156 Amazon SWF FAQ ........................................................................................................... 156 Amazon SWF Videos ....................................................................................................... 156 Ruby Flow Options......................................................................................................................... 157 Continue to use the Ruby Flow Framework ................................................................................ 157 Migrate to the Java Flow Framework ........................................................................................ 157 Migrate to Step Functions ....................................................................................................... 158 Use the Amazon SWF API directly ............................................................................................ 158 Document History.......................................................................................................................... 159 AWS glossary................................................................................................................................. 162 API Version 2012-01-25 vi Amazon Simple Workflow Service Developer Guide Development Options What is Amazon Simple Workflow Service? The Amazon Simple Workflow Service (Amazon SWF) makes it easy to build applications that coordinate work across distributed components. In Amazon SWF, a task represents a logical unit of work that is performed by a component of your application. Coordinating tasks across the application involves managing intertask dependencies, scheduling, and concurrency in accordance with the logical flow of the application. Amazon SWF gives you full control over implementing tasks and coordinating them without worrying about underlying complexities such as tracking their progress and maintaining their state. When using Amazon SWF, you implement workers to perform tasks. These workers can run either on cloud infrastructure, such as Amazon Elastic Compute Cloud (Amazon EC2), or on your own premises. You can create tasks that are long-running, or that may fail, time out, or require restarts—or that may complete with varying throughput and latency. Amazon SWF stores tasks and assigns them to workers when they are ready, tracks their progress, and maintains their state, including details on their completion. To coordinate tasks, you write a program that gets the latest state of each task from Amazon SWF and uses it to initiate subsequent tasks. Amazon SWF maintains an application's execution state durably so that the application is resilient to failures in individual components. With Amazon SWF, you can implement, deploy, scale, and modify these application components independently. Amazon SWF offers capabilities to support a variety of application requirements. It is suitable for a range of use cases that require coordination of tasks, including media processing, web application back-ends, business process workflows, and analytics pipelines. Development Options You have a number of options for implementing your workflow solutions with the Amazon Simple Workflow Service. Topics • AWS SDKs (p. 1) • AWS Flow Framework (p. 2) • HTTP Service API (p. 2) • Development Environments (p. 2) AWS SDKs Amazon SWF is supported by the AWS SDKs for Java, .NET, Node.js, PHP, PHP version 2, Python and Ruby, providing a convenient way to use the Amazon SWF HTTP API in the programming language of your choice. You can develop deciders, activities, or workflow starters using the API exposed by these libraries. Additionally, you can access visibility operations through these libraries so you can develop your own Amazon SWF monitoring and reporting tools. To download any of the AWS SDKs, go to https://aws.amazon.com/code. For detailed information about the Amazon SWF methods in each SDK, refer to the language-specific reference documentation for the SDK you are using. API Version 2012-01-25 1 Amazon Simple Workflow Service Developer Guide AWS Flow Framework Here is a list of the available AWS SDK documentation. • AWS Java Developer Guide | Amazon SWF • AWS SDK for Java API Reference • AWS SDK for .NET API Reference • AWS SDK for JavaScript in Node.js API Reference • AWS SDK for PHP API Reference • AWS SDK for Python (Boto) API Reference • AWS SDK for Ruby API Reference AWS Flow Framework The AWS Flow Framework is an enhanced SDK for writing distributed, asynchronous programs that can run as workflows on Amazon SWF. It is available for the Java progamming language, and it provides classes that simplify writing complex distributed programs. With the AWS Flow Framework, you use preconfigured types to map the definition of your workflow directly to methods in your program. The AWS Flow Framework supports standard object-oriented concepts, such as exception-based error handling, which makes it easier to implement complex workflows. Programs written with the AWS Flow Framework can be created, executed, and debugged entirely within your preferred editor or IDE. For more information, see the AWS Flow Framework website. Here are links to the AWS Flow Framework documentation: • AWS Flow Framework for Java Developer Guide • AWS Flow Framework for Java Reference AWS Flow Framework Sample Code In addition to the code snippets that appear in the AWS Flow Framework documentation, you can obtain full, downloadable code samples at the following locations: • AWS Flow Framework Recipes • AWS Flow Framework Samples for Amazon SWF HTTP Service API Amazon SWF provides service operations that are accessible through HTTP requests. You can use these operations to communicate directly with Amazon SWF, and you can use them to develop your own libraries in any language that can communicate with Amazon SWF through HTTP. You can develop deciders, activity workers, or workflow starters by using the service API. You can also access visibility operations through the API to develop your own monitoring and reporting tools. For information about how to use the API, see Making HTTP Requests to Amazon SWF (p. 117). For detailed information about API operations, go to the Amazon Simple Workflow Service API Reference. Development Environments You will need to set up a development environment appropriate to the programming language that you will use. For example, if you intend to develop for Amazon SWF with Java, you will need to install a Java API Version 2012-01-25 2 Amazon Simple Workflow Service Developer Guide Development Environments development environment, such as the AWS SDK for Java, on each of your development workstations. If you use the Eclipse IDE for Java development, you might consider also installing the AWS Toolkit for Eclipse. The Toolkit is an Eclipse plug-in that adds features that are helpful for AWS development. If your programming language requires a run-time environment, you need to set up that environment on each computer on which these processes run. API Version 2012-01-25 3 Amazon Simple Workflow Service Developer Guide Concepts Introduction to Amazon SWF A growing number of applications are relying on asynchronous and distributed processing. The scalability of such applications is the primary motivation for using this approach. By designing autonomous distributed components, developers have the flexibility to deploy and scale out parts of the application independently if the load on the application increases. Another motivation is the availability of cloud services. As application developers start taking advantage of cloud computing, they have a need to combine their existing on-premises assets with additional cloud-based assets. Yet another motivation for the asynchronous and distributed approach is the inherent distributed nature of the process being modeled by the application; for example, the automation of an order-fulfillment business process may span several systems and human tasks. Developing such applications can be complicated. It requires that you coordinate the execution of multiple distributed components and deal with the increased latencies and unreliability inherent in remote communication. To accomplish this, you would typically need to write complicated infrastructure involving message queues and databases, along with the complex logic to synchronize them. The Amazon Simple Workflow Service (Amazon SWF) makes it easier to develop asynchronous and distributed applications by providing a programming model and infrastructure for coordinating distributed components and maintaining their execution state in a reliable way. By relying on Amazon SWF, you are freed to focus on building the aspects of your application that differentiate it. Simple Workflow Concepts The basic concepts necessary for understanding Amazon SWF workflows are introduced below and are explained further in the subsequent sections of this guide. The following discussion is a high-level overview of the structure and components of a workflow. The fundamental concept in Amazon SWF is the workflow. A workflow is a set of activities that carry out some objective, together with logic that coordinates the activities. For example, a workflow could receive a customer order and take whatever actions are necessary to fulfill it. Each workflow runs in an AWS resource called a domain, which controls the workflow's scope. An AWS account can have multiple domains, each of which can contain multiple workflows, but workflows in different domains can't interact. When designing an Amazon SWF workflow, you precisely define each of the required activities. You then register each activity with Amazon SWF as an activity type. When you register the activity, you provide information such as a name and version, and some timeout values based on how long you expect the activity to take. For example, a customer may have an expectation that an order will ship within 24 hours. Such expectations would inform the timeout values that you specify when registering your activities. In the process of carrying out the workflow, some activities may need to be performed more than once, perhaps with varying inputs. For example, in a customer-order workflow, you might have an activity that handles purchased items. If the customer purchases multiple items, then this activity would have to run multiple times. Amazon SWF has the concept of an activity task that represents one invocation of an activity. In our example, the processing of each item would be represented by a single activity task. An activity worker is a program that receives activity tasks, performs them, and provides results back. Note that the task itself might actually be performed by a person, in which case the person would use the activity worker software for the receipt and disposition of the task. An example might be a statistical analyst, who receives sets of data, analyzes them, and then sends back the analysis. API Version 2012-01-25 4