Using secrets to access Amazon Database Migration Service endpoints
For Amazon DMS, a secret is an encrypted key that you can use to represent a set of user credentials to authenticate, through secret authentication, the database connection for a supported Amazon DMS source or target endpoint. For an Oracle endpoint that also uses Oracle Automatic Storage Management (ASM), Amazon DMS requires an additional secret that represents the user credentials to access Oracle ASM.
You can create the secret or secrets that Amazon DMS requires for secret
authentication using Amazon Secrets Manager, a service for securely creating,
storing, and retrieving credentials to access applications, services, and IT resources in
the cloud and on premise. This includes support for automatic periodic rotation of the
encrypted secret value without your intervention, providing an extra level of security for
your credentials. Enabling secret value rotation in Amazon Secrets Manager also
ensures that this secret value rotation happens without any effect on any database
migration that relies on the secret. For secretly authenticating an endpoint database
connection, create a secret whose identity or ARN you assign to
SecretsManagerSecretId
, which you include in your endpoint settings. For
secretly authenticating Oracle ASM as part of an Oracle endpoint, create a secret whose
identity or ARN you assign to SecretsManagerOracleAsmSecretId
, which you also
include in your endpoint settings.
Note
You can't use master credentials managed by Amazon RDS Aurora. These credentials don't include host or port information, which Amazon DMS needs to establish connections. Instead, create a new user and secret. For information about creating a user and secret, see Using the Amazon Web Services Management Console to create a secret and secret access role following.
For more information on Amazon Secrets Manager, see What Is Amazon Secrets Manager? in the Amazon Secrets Manager User Guide.
Amazon DMS supports secret authentication for the following on-premise or Amazon-managed databases on supported source and target endpoints:
-
Amazon DocumentDB
-
IBM Db2 LUW
-
Microsoft SQL Server
-
MongoDB
-
MySQL
-
Oracle
-
PostgreSQL
-
Amazon Redshift
-
SAP ASE
For connection to any of these databases, you have the choice of entering one of the following sets of values, but not both, as part of your endpoint settings:
-
Clear-text values to authenticate the database connection using the
UserName
,Password
,ServerName
, andPort
settings. For an Oracle endpoint that also uses Oracle ASM, include additional clear-text values to authenticate ASM using theAsmUserName
,AsmPassword
, andAsmServerName
settings. -
Secret authentication using values for the
SecretsManagerSecretId
andSecretsManagerAccessRoleArn
settings. For an Oracle endpoint using Oracle ASM, include additional values for theSecretsManagerOracleAsmSecretId
andSecretsManagerOracleAsmAccessRoleArn
settings. The secret values for these settings can include the following for:-
SecretsManagerSecretId
– The full Amazon Resource Name (ARN), partial ARN, or friendly name of a secret that you have created for endpoint database access in the Amazon Secrets Manager. -
SecretsManagerAccessRoleArn
– The ARN of a secret access role that you have created in IAM to provide Amazon DMS access to thisSecretsManagerSecretId
secret on your behalf. -
SecretsManagerOracleAsmSecretId
– The full Amazon Resource Name (ARN), partial ARN, or friendly name of a secret that you have created for Oracle ASM access in the Amazon Secrets Manager. -
SecretsManagerOracleAsmAccessRoleArn
– The ARN of a secret access role that you have created in IAM to provide Amazon DMS access to thisSecretsManagerOracleAsmSecretId
secret on your behalf.
Note
You can also use a single secret access role to provide Amazon DMS access to both the
SecretsManagerSecretId
secret and theSecretsManagerOracleAsmSecretId
secret. If you create this single secret access role for both secrets, ensure that you assign the same ARN for this access role to bothSecretsManagerAccessRoleArn
andSecretsManagerOracleAsmAccessRoleArn
. For example, if your secret access role for both secrets has its ARN assigned to the variable,ARN2xsecrets
, you can set these ARN settings as follows:SecretsManagerAccessRoleArn = ARN2xsecrets; SecretsManagerOracleAsmAccessRoleArn = ARN2xsecrets;
For more information on creating these values, see Using the Amazon Web Services Management Console to create a secret and secret access role.
-
After you have created and specified the required secret and secret access-role endpoint
settings for your endpoints, update the permissions on the user accounts that will run the
CreateEndpoint
or ModifyEndpoint
API request with these secret
details. Ensure that these account permissions include the IAM:GetRole
permission on the secret access role and the SecretsManager:DescribeSecret
permission on the secret. Amazon DMS requires these permissions to validate both the
access role and its secret.
To provide and verify required user permissions
-
Sign in to the Amazon Web Services Management Console and open the Amazon Identity and Access Management console at https://console.amazonaws.cn/iam/
. -
Choose Users, then select the User ID used for making
CreateEndpoint
andModifyEndpoint
API calls. -
From the Permissions tab, choose {} JSON.
-
Make sure the user has the permissions shown following.
{ "Statement": [{ "Effect": "Allow", "Action": [ "iam:GetRole", "iam:PassRole" ], "Resource": "
SECRET_ACCESS_ROLE_ARN
" }, { "Effect": "Allow", "Action": "secretsmanager:DescribeSecret", "Resource": "SECRET_ARN
" } ] } -
If the user doesn't have those permission, add the permissions.
-
If you're using an IAM Role for making DMS API calls, repeat the steps above for the respective role.
-
Open a terminal and use the Amazon CLI to validate that permissions are given correctly by assuming the Role or User used above.
-
Validate user’s permission on the SecretAccessRole using the IAM
get-role
command.aws iam get-role --role-name
ROLE_NAME
Replace
ROLE_NAME
with the name ofSecretsManagerAccessRole
.If the command returns an error message, make sure the permissions were given correctly.
-
Validate user’s permission on the secret using the Secrets Manager
describe-secret
command.aws secretsmanager describe-secret --secret-id
SECRET_NAME OR SECRET_ARN
--region=REGION_NAME
User can be the friendly name, partial ARN or the full ARN. For more information, see describe-secret.
If the command returns an error message, make sure the permissions were given correctly.
-
Using the Amazon Web Services Management Console to create a secret and secret access role
You can use the Amazon Web Services Management Console to create a secret for endpoint authentication and to create the policy and role to allow Amazon DMS to access the secret on your behalf.
To create a secret using the Amazon Web Services Management Console that Amazon DMS can use to authenticate a database for source and target endpoint connections
-
Sign in to the Amazon Web Services Management Console and open the Amazon Secrets Manager console at https://console.amazonaws.cn/secretsmanager/
. -
Choose Store a new secret.
-
Under Select secret type on the Store a new secret page, choose Other type of secrets, then choose Plaintext.
Note
This is the only place that you need to enter clear text credentials to connect to your endpoint database from this point forward.
-
In the Plaintext field:
-
For a secret whose identity you assign to
SecretsManagerSecretId
, enter the following JSON structure.{ "username":
db_username
, "password":db_user_password
, "port":db_port_number
, "host":db_server_name
}Note
This is the minimum list of JSON members required to authenticate the endpoint database. You can add any additional JSON endpoint settings as JSON members in all lower case that you want. However, Amazon DMS ignores any additional JSON members for endpoint authentication.
Here,
is the name of the user accessing the database,db_username
is the password of the database user,db_user_password
is the port number to access the database, anddb_port_number
is the database server name (address) on the web, as in the following example.db_server_name
{ "username": "admin", "password": "some_password", "port": "8190", "host": "oracle101.abcdefghij.us-east-1.rds.amazonaws.com" }
-
For a secret whose identity you assign to
SecretsManagerOracleAsmSecretId
, enter the following JSON structure.{ "asm_user":
asm_username
, "asm_password":asm_user_password
, "asm_server":asm_server_name
}Note
This is the minimum list of JSON members required to authenticate Oracle ASM for an Oracle endpoint. It is also the complete list that you can specify based on the available Oracle ASM endpoint settings.
Here,
is the name of the user accessing Oracle ASM,asm_username
is the password of the Oracle ASM user, andasm_user_password
is the Oracle ASM server name (address) on the web, including the port, as in the following example.asm_server_name
{ "asm_user": "oracle_asm_user", "asm_password": "oracle_asm_password", "asm_server": "oracle101.abcdefghij.us-east-1.rds.amazonaws.com:8190/+ASM" }
-
-
Select an Amazon KMS encryption key to encrypt the secret. You can accept the default encryption key created for your service by Amazon Secrets Manager or select a Amazon KMS key that you create.
-
Specify a name to reference this secret and an optional description. This is the friendly name that you use as the value for
SecretsManagerSecretId
orSecretsManagerOracleAsmSecretId
. -
If you want to enable automatic rotation in the secret, you need to select or create an Amazon Lambda function with permission to rotate the credentials for the secret as described. However, before setting automatic rotation to use your Lambda function, ensure that the configuration settings for the function add the following four characters to the value of the
EXCLUDE_CHARACTERS
environment variable.;.:+{}
Amazon DMS doesn't allow these characters in passwords used for endpoint credentials. Configuring your Lambda function to exclude them prevents Amazon Secrets Manager from generating these characters as part of its rotated password values. After you set automatic rotation to use your Lambda function, Amazon Secrets Manager immediately rotates the secret to validate your secret configuration.
Note
Depending on your database engine configuration, your database might not fetch the rotated credentials. In this case, you need to manually restart the task to refresh the credentials.
-
Review and store your secret in Amazon Secrets Manager. You can then look up each secret by its friendly name in Amazon Secrets Manager, then retrieve the secret ARN as the value for
SecretsManagerSecretId
orSecretsManagerOracleAsmSecretId
as appropriate to authenticate access to your endpoint database connection and Oracle ASM (if used).
To create the secret access policy and role to set your
SecretsManagerAccessRoleArn
or
SecretsManagerOracleAsmAccessRoleArn
, which allows Amazon DMS
to access Amazon Secrets Manager to access your appropriate secret
-
Sign in to the Amazon Web Services Management Console and open the Amazon Identity and Access Management (IAM) console at https://console.amazonaws.cn/iam/
. -
Choose Policies, then choose Create policy.
-
Choose JSON and enter the following policy to enable access to and decryption of your secret.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource":
secret_arn
, }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:DescribeKey" ], "Resource":kms_key_arn
, } ] }Here,
is the ARN of your secret, which you can get from eithersecret_arn
SecretsManagerSecretId
orSecretsManagerOracleAsmSecretId
as appropriate, and
is the ARN of the Amazon KMS key that you are using to encrypt your secret, as in the following example.kms_key_arn
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": "arn:aws:secretsmanager:us-east-2:123456789012:secret:MySQLTestSecret-qeHamH" }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:DescribeKey" ], "Resource": "arn:aws:kms:us-east-2:123456789012:key/761138dc-0542-4e58-947f-4a3a8458d0fd" } ] }
Note
If you use the default encryption key created by Amazon Secrets Manager, you do not have to specify the Amazon KMS permissions for
.kms_key_arn
If you want your policy to provide access to both secrets, simply specify an additional JSON resource object for the other
secret_arn
.If your secret is in a different account, then the
SecretsManagerAccessRoleArn
role needs an additional policy to verify the cross account secret. For such use cases, add the actionsecretsmanager:DescribeSecret
to the policy. For more details on setting up a cross-account secret, see Permissions to Amazon Secrets Manager secrets for users in a different account. -
Review and create the policy with a friendly name and optional description.
-
Choose Roles, then choose Create role.
-
Choose Amazon service as the type of trusted entity.
-
Choose DMS from the list of services as the trusted service, then choose Next: Permissions.
-
Look up and attach the policy you created in step 4, then proceed through adding any tags and review your role. At this point, edit the trust relationships for the role to use your Amazon DMS regional service principal as the trusted entity. This principal has the following format.
dms.
region-name
.amazonaws.comHere,
is the name of your region, such asregion-name
us-east-1
. Thus, an Amazon DMS regional service principal for this region follows.dms.us-east-1.amazonaws.com
-
After editing the trusted entity for the role, create the role with a friendly name and optional description. You can now look up your new role by its friendly name in IAM, then retrieve the role ARN as the
SecretsManagerAccessRoleArn
orSecretsManagerOracleAsmAccessRoleArn
value to authenticate your endpoint database connection.
To use secrets manager with a replication instance in a private subnet
-
Create a secret manager VPC endpoint and note the DNS for the endpoint. For more information about creating a secrets manager VPC endpoint, see Connecting to Secrets Manager through a VPC endpoint in the Amazon Secrets Manager User Guide.
-
Attach the replication instance security group to the secret manager VPC endpoint.
-
For the replication instance security group egress rules, allow all traffic for destination
0.0.0.0/0
. -
Set the endpoint extra connection attribute,
secretsManagerEndpointOverride=
to provide the secret manager VPC endpoint DNS, as shown in the following example.secretsManager endpoint DNS
secretsManagerEndpointOverride=vpce-1234a5678b9012c-12345678.secretsmanager.eu-west-1.vpce.amazonaws.com