Process Amazon MQ messages with Lambda - Amazon Lambda
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Process Amazon MQ messages with Lambda

Configure network access

To give Lambda full access to your broker through your event source mapping, either your broker must use a public endpoint (public IP address), or you must provide access to the Amazon VPC you created the broker in.

By default, when you create an Amazon MQ broker, the PubliclyAccessible flag is set to false. For your broker to receive a public IP address, you must set the PubliclyAccessible flag to true.

Best practice for using Amazon MQ with Lambda is to use Amazon PrivateLink VPC endpoints and to give your Lambda function access to your broker's VPC. Deploy an endpoint for Lambda, and, for ActiveMQ only, an endpoint for Amazon Security Token Service (Amazon STS). If your broker uses authentication, also deploy an endpoint for Amazon Secrets Manager. To learn more, see Set up Amazon VPC endpoints.

Alternatively, configure a NAT gateway on each public subnet in the VPC containing your Amazon MQ broker. For more information, see Enable internet access for VPC-connected Lambda functions.

When you create an event source mapping for an Amazon MQ broker, Lambda checks whether Elastic Network Interfaces (ENIs) are already present for the subnets and security groups of your broker’s VPC. If Lambda finds existing ENIs, it attempts to re-use them. Otherwise, Lambda creates new ENIs to connect to the event source and invoke your function.

Note

Lambda functions always run inside VPCs owned by the Lambda service. These VPCs are maintained automatically by the service and are not visible to customers. You can also connect your function to an Amazon VPC. In either case, your function’s VPC configuration doesn’t affect the event source mapping. Only the configuration of the event source’s VPC determines how Lambda connects to your event source.

Configure security group rules

Configure the security groups for the Amazon VPC containing your Amazon MQ.

Note

By default, Amazon MQ for ActiveMQ uses port 61617, and Amazon MQ for RabbitMQ port 5671.

  • Inbound rules – Allow all traffic on the broker port for the security group associated with your event source from within its own security group.

  • Outbound rules – Allow all traffic on port 443 for all destinations. Allow all traffic on the broker port from within its own security group.

  • Amazon VPC endpoint inbound rules — For the security group associated with your Amazon VPC endpoint, allow inbound traffic on port 443 from the broker's security group

Set up Amazon VPC endpoints

When you use Amazon VPC endpoints, Amazon routes your API calls to invoke a function using the endpoint's Elastic Network Interface (ENI). The Lambda service principal needs to call lambda:InvokeFunction on any functions that use those ENIs. Additionally, for Amazon MQ for ActiveMQ, the Lambda service principal must perform sts:AssumeRole on roles that use the ENIs.

By default, Amazon VPC endpoints have open IAM policies that allow broad access to resources. To use Amazon MQ with Lambda in production, you might restrict these policies to only allow specific principals to only access specifc roles and functions.

The following example VPC endpoint policies show how to grant the required access for Amazon STS and Lambda endpoints.

Example Endpoint policy — Amazon STS endpoint (ActiveMQ only)
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": [ "lambda.amazonaws.com" ] }, "Resource": "arn:aws-cn::iam::123456789012:role/my-role" } ] }
Example Endpoint policy — Lambda endpoint
{ "Statement": [ { "Action": "lambda:InvokeFunction", "Effect": "Allow", "Principal": { "Service": [ "lambda.amazonaws.com" ] }, "Resource": "arn:aws-cn::lambda:us-west-2:123456789012:function:my-function" } ] }

If your Amazon MQ broker uses authentication, you can also restrict the endpoint policy for the Secrets Manager endpoint. To call the Secrets Manager API, Lambda uses your function role, not the Lambda service principal. The following example shows a Secrets Manager endpoint policy.

Example VPC endpoint policy — Secrets Manager endpoint
{ "Statement": [ { "Action": "secretsmanager:GetSecretValue", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws-cn::iam::123456789012:role/my-role" ] }, "Resource": "arn:aws-cn::secretsmanager:us-west-2:123456789012:secret:my-secret" } ] }
Warning

Restricting your endpoint policies to only allow API calls originating within your organization prevents the event source mapping from functioning properly.

Create the event source mapping

Create an event source mapping to tell Lambda to send records from an Amazon MQ broker to a Lambda function. You can create multiple event source mappings to process the same data with multiple functions, or to process items from multiple sources with a single function.

To configure your function to read from Amazon MQ, add the required permissions and create an MQ trigger in the Lambda console.

To read records from an Amazon MQ broker, your Lambda function needs the following permissions. You grant Lambda permission to interact with your Amazon MQ broker and its underlying resouces by adding permission statements to your function execution role:

Note

When using an encrypted customer managed key, add the kms:Decrypt permission as well.

To add permissions and create a trigger
  1. Open the Functions page of the Lambda console.

  2. Choose the name of a function.

  3. Choose the Configuration tab, and then choose Permissions.

  4. Under Role name, choose the link to your execution role. This link opens the role in the IAM console.

    Link to execution role
  5. Choose Add permissions, and then choose Create inline policy.

    Create inline policy in IAM console
  6. In the Policy editor, choose JSON. Enter the following policy. Your function needs these permissions to read from an Amazon MQ broker.

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "mq:DescribeBroker", "secretsmanager:GetSecretValue", "ec2:CreateNetworkInterface", "ec2:DeleteNetworkInterface", "ec2:DescribeNetworkInterfaces", "ec2:DescribeSecurityGroups", "ec2:DescribeSubnets", "ec2:DescribeVpcs", "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "*" } ] }
    Note

    When using an encrypted customer managed key, you must also add the kms:Decrypt permission.

  7. Choose Next. Enter a policy name and then choose Create policy.

  8. Go back to your function in the Lambda console. Under Function overview, choose Add trigger.

    Function overview section of the Lambda console
  9. Choose the MQ trigger type.

  10. Configure the required options, and then choose Add.

Lambda supports the following options for Amazon MQ event sources:

  • MQ broker – Select an Amazon MQ broker.

  • Batch size – Set the maximum number of messages to retrieve in a single batch.

  • Queue name – Enter the Amazon MQ queue to consume.

  • Source access configuration – Enter virtual host information and the Secrets Manager secret that stores your broker credentials.

  • Enable trigger – Disable the trigger to stop processing records.

To enable or disable the trigger (or delete it), choose the MQ trigger in the designer. To reconfigure the trigger, use the event source mapping API operations.