Map & Parallel States in AWS Step Functions

Introduction

AWS Step Functions is a serverless orchestration service that allows you to coordinate components of distributed applications using visual workflows. Among its features are Map and Parallel states, which enable handling multiple tasks at once.

Key Concepts

Definitions

Map State: A state that allows you to run a set of steps for each item of a list in parallel.
Parallel State: A state that allows you to execute multiple branches of execution at the same time.

Map State

The Map state is designed for scenarios where you need to process each item in an array independently and possibly in parallel. It can be used to iterate through a list and applies the same set of states to each item.

Example of Map State

{
              "Comment": "A Map State Example",
              "StartAt": "ProcessItems",
              "States": {
                "ProcessItems": {
                  "Type": "Map",
                  "ItemsPath": "$.items",
                  "Iterator": {
                    "StartAt": "ProcessItem",
                    "States": {
                      "ProcessItem": {
                        "Type": "Task",
                        "Resource": "arn:aws:lambda:us-east-1:123456789012:function:ProcessItem",
                        "End": true
                      }
                    }
                  },
                  "End": true
                }
              }
            }

Parallel State

Parallel states allow you to execute multiple states simultaneously, all in one state. This is useful for dividing work into multiple branches that can be completed independently.

Example of Parallel State

{
              "Comment": "A Parallel State Example",
              "StartAt": "RunInParallel",
              "States": {
                "RunInParallel": {
                  "Type": "Parallel",
                  "Branches": [
                    {
                      "StartAt": "TaskA",
                      "States": {
                        "TaskA": {
                          "Type": "Task",
                          "Resource": "arn:aws:lambda:us-east-1:123456789012:function:TaskA",
                          "End": true
                        }
                      }
                    },
                    {
                      "StartAt": "TaskB",
                      "States": {
                        "TaskB": {
                          "Type": "Task",
                          "Resource": "arn:aws:lambda:us-east-1:123456789012:function:TaskB",
                          "End": true
                        }
                      }
                    }
                  ],
                  "End": true
                }
              }
            }

Best Practices

Use Map states when you need to iterate over a collection of items.
Utilize Parallel states to execute independent tasks that do not depend on each other.
Monitor and log the performance of each state for troubleshooting and optimization.
Limit the number of branches in a Parallel state to prevent hitting service limits.
Consider error handling strategies (e.g., Retry and Catch) to manage failures gracefully.

FAQ

What is the maximum number of iterations for a Map state?

The maximum number of iterations for a Map state is 1,000, and the maximum payload size is 256 KB.

Can I nest Map and Parallel states?

Yes, you can nest Map and Parallel states to create more complex workflows.

Are there any limits on the number of parallel branches?

The maximum number of parallel branches is 40, and the total execution time for all branches must not exceed 1 year.

Flowchart


            graph TD;
                A[Start] --> B{Use Map State?};
                B -- Yes --> C[Process Each Item];
                B -- No --> D{Use Parallel State?};
                D -- Yes --> E[Execute Branches];
                D -- No --> F[End];
                C --> F;
                E --> F;