Configuring resource data sync for Inventory
This topic describes how to set up and configure resource data sync for Amazon Systems Manager Inventory. For information about resource data sync for Systems Manager Explorer, see Setting up Systems Manager Explorer to display data from multiple accounts and Regions.
About resource data sync
You can use Systems Manager resource data sync to send inventory data collected from all of your managed nodes to a single Amazon Simple Storage Service (Amazon S3) bucket. Resource data sync then automatically updates the centralized data when new inventory data is collected. With all inventory data stored in a target Amazon S3 bucket, you can use services like Amazon Athena and Amazon QuickSight to query and analyze the aggregated data.
For example, say that you've configured inventory to collect data about the operating system (OS) and applications running on a fleet of 150 managed nodes. Some of these nodes are located in an on-premises data center, and others are running in Amazon Elastic Compute Cloud (Amazon EC2) across multiple Amazon Web Services Regions. If you have not configured resource data sync, you either need to manually gather the collected inventory data for each managed node, or you have to create scripts to gather this information. You would then need to port the data into an application so that you can run queries and analyze it.
With resource data sync, you perform a one-time operation that synchronizes all inventory data from all of your managed nodes. After the sync is successfully created, Systems Manager creates a baseline of all inventory data and saves it in the target Amazon S3 bucket. When new inventory data is collected, Systems Manager automatically updates the data in the Amazon S3 bucket. You can then quickly and cost-effectively port the data to Amazon Athena and Amazon QuickSight.
Diagram 1 shows how resource data sync aggregates inventory data from Amazon EC2 and other machine types in a hybrid and multicloud environment to a target Amazon S3 bucket. This diagram also shows how resource data sync works with multiple Amazon Web Services accounts and Amazon Web Services Regions.
Diagram 1: Resource data sync with multiple Amazon Web Services accounts and Amazon Web Services Regions
If you delete a managed node, resource data sync preserves the inventory file
for the deleted node. For running nodes, however, resource data sync
automatically overwrites old inventory files when new files are created and
written to the Amazon S3 bucket. If you want to track inventory changes over time,
you can use the Amazon Config service to track the
SSM:ManagedInstanceInventory
resource type. For more
information, see Getting Started
with Amazon Config.
Use the procedures in this section to create a resource data sync for Inventory by
using the Amazon S3 and Amazon Systems Manager consoles. You can also use Amazon CloudFormation to create or
delete a resource data sync. To use Amazon CloudFormation, add the AWS::SSM::ResourceDataSync
resource to your Amazon CloudFormation
template. For information, see one of the following documentation resources:
-
Amazon CloudFormation resource for resource data sync in Amazon Systems Manager
(blog) -
Working with Amazon CloudFormation Templates in the Amazon CloudFormation User Guide
Note
You can use Amazon Key Management Service (Amazon KMS) to encrypt inventory data in the Amazon S3 bucket. For an example of how to create an encrypted sync by using the Amazon Command Line Interface (Amazon CLI) and how to work with the centralized data in Amazon Athena and Amazon QuickSight, see Walkthrough: Use resource data sync to aggregate inventory data.
Before you begin
Before you create a resource data sync, use the following procedure to create a central Amazon S3 bucket to store aggregated inventory data. The procedure describes how to assign a bucket policy that allows Systems Manager to write inventory data to the bucket from multiple accounts. If you already have an Amazon S3 bucket that you want to use to aggregate inventory data for resource data sync, then you must configure the bucket to use the policy in the following procedure.
Note
Systems Manager Inventory can't add data to a specified Amazon S3 bucket if that bucket is configured to use Object Lock. Verify that the Amazon S3 bucket you create or choose for resource data sync isn't configured to use Amazon S3 Object Lock. For more information, see How Amazon S3 Object Lock works in the Amazon Simple Storage Service User Guide.
To create and configure an Amazon S3 bucket for resource data sync
Open the Amazon S3 console at https://console.amazonaws.cn/s3/
. -
Create a bucket to store your aggregated Inventory data. For more information, see Create a Bucket in the Amazon Simple Storage Service User Guide. Make a note of the bucket name and the Amazon Web Services Region where you created it.
-
Choose the Permissions tab, and then choose Bucket Policy.
-
Copy and paste the following bucket policy into the policy editor. Replace
DOC-EXAMPLE-BUCKET
andaccount-id
with the name of the S3 bucket you created and a valid Amazon Web Services account ID.To allow multiple Amazon Web Services accounts to send inventory data to the central Amazon S3 bucket, specify each account in the policy as shown in the following
Resource
sample:"Resource": [ "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=123456789012/*", "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=444455556666/*", "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=777788889999/*" ], "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control", "aws:SourceAccount": [ "123456789012", "444455556666", "777788889999" ] }, "ArnLike": { "aws:SourceArn": [ "arn:aws-cn:ssm:*:123456789012:resource-data-sync/*", "arn:aws-cn:ssm:*:444455556666:resource-data-sync/*", "arn:aws-cn:ssm:*:777788889999:resource-data-sync/*" ] } }
Note
For information about viewing your Amazon Web Services account ID, see Your Amazon Web Services Account ID and Its Alias in the IAM User Guide.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "SSMBucketPermissionsCheck", "Effect": "Allow", "Principal": { "Service": "ssm.amazonaws.com.cn" }, "Action": "s3:GetBucketAcl", "Resource": "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET" }, { "Sid": " SSMBucketDelivery", "Effect": "Allow", "Principal": { "Service": "ssm.amazonaws.com.cn" }, "Action": "s3:PutObject", "Resource": [ "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=
ID_number
/*", "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=ID_number
/*", "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=ID_number
/*", "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET/*/accountid=ID_number
/*" ], "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control", "aws:SourceAccount": "ID_number
" }, "ArnLike": { "aws:SourceArn": "arn:aws-cn:ssm:*:ID_number
:resource-data-sync/*" } } } ] }
Create a resource data sync for Inventory
Use the following procedure to create a resource data sync for Systems Manager Inventory by using the Systems Manager console. For information about how to create a resource data sync by using the Amazon CLI, see Walkthrough: Configure your managed nodes for Inventory by using the CLI.
To create a resource data sync
Open the Amazon Systems Manager console at https://console.amazonaws.cn/systems-manager/
. In the navigation pane, choose Fleet Manager.
-or-
If the Amazon Systems Manager home page opens first, choose the menu icon ( ) to open the navigation pane, and then choose Fleet Manager in the navigation pane.
-
In the Account management menu, choose Resource data sync.
-
Choose Create resource data sync.
-
In the Sync name field, enter a name for the sync configuration.
-
In the Bucket name field, enter the name of the Amazon S3 bucket you created using the To create and configure an Amazon S3 bucket for resource data sync procedure.
-
(Optional) In the Bucket prefix field, enter the name of an Amazon S3 bucket prefix (subdirectory).
-
In the Bucket region field, choose This region if the Amazon S3 bucket you created is located in the current Amazon Web Services Region. If the bucket is located in a different Amazon Web Services Region, choose Another region, and enter the name of the Region.
Note
If the sync and the target Amazon S3 bucket are located in different regions, you might be subject to data transfer pricing. For more information, see Amazon S3 Pricing
. -
(Optional) In the KMS Key ARN field, type or paste a KMS Key ARN to encrypt inventory data in Amazon S3.
-
Choose Create.
To synchronize inventory data from multiple Amazon Web Services Regions, you must create a resource data sync in each Region. Repeat this procedure in each Amazon Web Services Region where you want to collect inventory data and send it to the central Amazon S3 bucket. When you create the sync in each Region, specify the central Amazon S3 bucket in the Bucket name field. Then use the Bucket region option to choose the Region where you created the central Amazon S3 bucket, as shown in the following screen shot. The next time the association runs to collect inventory data, Systems Manager stores the data in the central Amazon S3 bucket.
Creating an inventory resource data sync for accounts defined in Amazon Organizations
You can synchronize inventory data from Amazon Web Services accounts defined in Amazon Organizations to a central Amazon S3 bucket. After you complete the following procedures, inventory data is synchronized to individual Amazon S3 key prefixes in the central bucket. Each key prefix represents a different Amazon Web Services account ID.
Before you begin
Before you begin, verify that you set up and configured Amazon Web Services accounts in Amazon Organizations. For more information, see in the Amazon Organizations User Guide.
Also, be aware that you must create the organization-based resource data sync for each Amazon Web Services Region and Amazon Web Services account defined in Amazon Organizations.
Creating a central Amazon S3 bucket
Use the following procedure to create a central Amazon S3 bucket to store aggregated inventory data. The procedure describes how to assign a bucket policy that allows Systems Manager to write inventory data to the bucket from your Amazon Organizations account ID. If you already have an Amazon S3 bucket that you want to use to aggregate inventory data for resource data sync, then you must configure the bucket to use the policy in the following procedure.
To create and configure an Amazon S3 bucket for resource data sync for multiple accounts defined in Amazon Organizations
Open the Amazon S3 console at https://console.amazonaws.cn/s3/
. -
Create a bucket to store your aggregated inventory data. For more information, see Create a Bucket in the Amazon Simple Storage Service User Guide. Make a note of the bucket name and the Amazon Web Services Region where you created it.
-
Choose the Permissions tab, and then choose Bucket Policy.
-
Copy and paste the following bucket policy into the policy editor. Replace
DOC-EXAMPLE-BUCKET
andorganization-id
with the name of the Amazon S3 bucket you created and a valid Amazon Organizations account ID.Optionally, replace
bucket-prefix
with the name of an Amazon S3 prefix (subdirectory). If you didn't create a prefix, removebucket-prefix
/ from the ARN in the following policy.{ "Version": "2012-10-17", "Statement": [ { "Sid": "SSMBucketPermissionsCheck", "Effect": "Allow", "Principal": { "Service": "ssm.amazonaws.com.cn" }, "Action": "s3:GetBucketAcl", "Resource": "arn:aws-cn:s3:::
S3_bucket_name
" }, { "Sid": " SSMBucketDelivery", "Effect": "Allow", "Principal": { "Service": "ssm.amazonaws.com.cn" }, "Action": "s3:PutObject", "Resource": [ "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET
/bucket-prefix
/*/accountid=*/*" ], "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control", "aws:SourceOrgID": "organization-id
" } } }, { "Sid": " SSMBucketDeliveryTagging", "Effect": "Allow", "Principal": { "Service": "ssm.amazonaws.com.cn" }, "Action": "s3:PutObjectTagging", "Resource": [ "arn:aws-cn:s3:::DOC-EXAMPLE-BUCKET
/bucket-prefix
/*/accountid=*/*" ] } ] }
Create an inventory resource data sync for accounts defined in Amazon Organizations
The following procedure describes how to use the Amazon CLI to create a resource data sync for accounts that are defined in Amazon Organizations. You must use the Amazon CLI to perform this task. You must also perform this procedure for each Amazon Web Services Region and Amazon Web Services account defined in Amazon Organizations.
To create a resource data sync for an account defined in Amazon Organizations (Amazon CLI)
Install and configure the Amazon Command Line Interface (Amazon CLI), if you haven't already.
For information, see Installing or updating the latest version of the Amazon CLI.
-
Run the following command to verify that you don't have any other resource data syncs. You can only have one organization-based resource data sync.
aws ssm list-resource-data-sync
If the command returns another resource data sync, you must delete it or choose not to create a new one.
-
Run the following command to create a resource data sync for an account defined in Amazon Organizations. For
DOC-EXAMPLE-BUCKET
, specify the name of the Amazon S3 bucket you created earlier in this topic. If you created a prefix (subdirectory) for your bucket, then specify this information forprefix-name
.aws ssm create-resource-data-sync --sync-name
name
--s3-destination "BucketName=DOC-EXAMPLE-BUCKET
,Prefix=prefix-name
,SyncFormat=JsonSerDe,Region=Amazon Web Services Region, for example us-east-2
,DestinationDataSharing={DestinationDataSharingType=Organization}" -
Repeat Steps 2 and 3 for every Amazon Web Services Region and Amazon Web Services account where you want to synchronize data to the central Amazon S3 bucket.
Managing resource data syncs
Each Amazon Web Services account can have 5 resource data syncs per Amazon Web Services Region. You can use the Amazon Systems Manager Fleet Manager console to manage your resource data syncs.
To view resource data syncs
Open the Amazon Systems Manager console at https://console.amazonaws.cn/systems-manager/
. In the navigation pane, choose Fleet Manager.
-or-
If the Amazon Systems Manager home page opens first, choose the menu icon ( ) to open the navigation pane, and then choose Fleet Manager in the navigation pane.
-
In the Account management dropdown, choose Resource data syncs.
-
Select a resource data sync from the table, and then choose View details to view information about your resource data sync.
To delete a resource data sync
Open the Amazon Systems Manager console at https://console.amazonaws.cn/systems-manager/
. In the navigation pane, choose Fleet Manager.
-or-
If the Amazon Systems Manager home page opens first, choose the menu icon ( ) to open the navigation pane, and then choose Fleet Manager in the navigation pane.
-
In the Account management dropdown, choose Resource data syncs.
-
Select a resource data sync from the table, and then choose Delete.