Namespace Amazon.CDK.IntegTests.Alpha
integ-tests
---The APIs of higher level constructs in this module are experimental and under active development.
They are subject to non-backward compatible changes or removal in any future version. These are
not subject to the <a href="https://semver.org/">Semantic Versioning</a> model and breaking changes will be
announced in the release notes. This means that while you may use them, you may need to update
your source code when upgrading to a newer version of this package.
Overview
This library is meant to be used in combination with the integ-runner CLI to enable users to write and execute integration tests for AWS CDK Constructs.
An integration test should be defined as a CDK application, and there should be a 1:1 relationship between an integration test and a CDK application.
So for example, in order to create an integration test called my-function
we would need to create a file to contain our integration test application.
test/integ.my-function.ts
var app = new App();
var stack = new Stack();
new Function(stack, "MyFunction", new FunctionProps {
Runtime = Runtime.NODEJS_LATEST,
Handler = "index.handler",
Code = Code.FromAsset(Join(__dirname, "lambda-handler"))
});
This is a self contained CDK application which we could deploy by running
cdk deploy --app 'node test/integ.my-function.js'
In order to turn this into an integration test, all that is needed is to
use the IntegTest
construct.
App app;
Stack stack;
new IntegTest(app, "Integ", new IntegTestProps { TestCases = new [] { stack } });
You will notice that the stack
is registered to the IntegTest
as a test case.
Each integration test can contain multiple test cases, which are just instances
of a stack. See the Usage section for more details.
Usage
IntegTest
Suppose you have a simple stack, that only encapsulates a Lambda function with a certain handler:
class StackUnderTestProps : StackProps
{
public Architecture? Architecture { get; set; }
}
class StackUnderTest : Stack
{
public StackUnderTest(Construct scope, string id, StackUnderTestProps props) : base(scope, id, props)
{
new Function(this, "Handler", new FunctionProps {
Runtime = Runtime.NODEJS_LATEST,
Handler = "index.handler",
Code = Code.FromAsset(Join(__dirname, "lambda-handler")),
Architecture = props.Architecture
});
}
}
You may want to test this stack under different conditions. For example, we want
this stack to be deployed correctly, regardless of the architecture we choose
for the Lambda function. In particular, it should work for both ARM_64
and
X86_64
. So you can create an IntegTestCase
that exercises both scenarios:
class StackUnderTestProps : StackProps
{
public Architecture? Architecture { get; set; }
}
class StackUnderTest : Stack
{
public StackUnderTest(Construct scope, string id, StackUnderTestProps props) : base(scope, id, props)
{
new Function(this, "Handler", new FunctionProps {
Runtime = Runtime.NODEJS_LATEST,
Handler = "index.handler",
Code = Code.FromAsset(Join(__dirname, "lambda-handler")),
Architecture = props.Architecture
});
}
}
// Beginning of the test suite
var app = new App();
new IntegTest(app, "DifferentArchitectures", new IntegTestProps {
TestCases = new [] {
new StackUnderTest(app, "Stack1", new StackUnderTestProps {
Architecture = Architecture.ARM_64
}),
new StackUnderTest(app, "Stack2", new StackUnderTestProps {
Architecture = Architecture.X86_64
}) }
});
This is all the instruction you need for the integration test runner to know which stacks to synthesize, deploy and destroy. But you may also need to customize the behavior of the runner by changing its parameters. For example:
var app = new App();
var stackUnderTest = new Stack(app, "StackUnderTest");
var stack = new Stack(app, "stack");
var testCase = new IntegTest(app, "CustomizedDeploymentWorkflow", new IntegTestProps {
TestCases = new [] { stackUnderTest },
DiffAssets = true,
StackUpdateWorkflow = true,
CdkCommandOptions = new CdkCommands {
Deploy = new DeployCommand {
Args = new DeployOptions {
RequireApproval = RequireApproval.NEVER,
Json = true
}
},
Destroy = new DestroyCommand {
Args = new DestroyOptions {
Force = true
}
}
}
});
IntegTestCaseStack
In the majority of cases an integration test will contain a single IntegTestCase
.
By default when you create an IntegTest
an IntegTestCase
is created for you
and all of your test cases are registered to this IntegTestCase
. The IntegTestCase
and IntegTestCaseStack
constructs are only needed when it is necessary to
defined different options for individual test cases.
For example, you might want to have one test case where diffAssets
is enabled.
App app;
Stack stackUnderTest;
var testCaseWithAssets = new IntegTestCaseStack(app, "TestCaseAssets", new IntegTestCaseStackProps {
DiffAssets = true
});
new IntegTest(app, "Integ", new IntegTestProps { TestCases = new [] { stackUnderTest, testCaseWithAssets } });
Assertions
This library also provides a utility to make assertions against the infrastructure that the integration test deploys.
There are two main scenarios in which assertions are created.
In this case you would create an integration test using the IntegTest
construct and then make assertions using the assert
property.
You should not utilize the assertion constructs directly, but should instead use the methods
on IntegTest.assertions
.
App app;
Stack stack;
var integ = new IntegTest(app, "Integ", new IntegTestProps { TestCases = new [] { stack } });
integ.Assertions.AwsApiCall("S3", "getObject");
By default an assertions stack is automatically generated for you. You may however provide your own stack to use.
App app;
Stack stack;
Stack assertionStack;
var integ = new IntegTest(app, "Integ", new IntegTestProps { TestCases = new [] { stack }, AssertionStack = assertionStack });
integ.Assertions.AwsApiCall("S3", "getObject");
In this case you may be using assertions as part of a normal CDK deployment in order to make an assertion on the infrastructure before the deployment is considered successful. In this case you can utilize the assertions constructs directly.
Stack myAppStack;
new AwsApiCall(myAppStack, "GetObject", new AwsApiCallProps {
Service = "S3",
Api = "getObject"
});
DeployAssert
Assertions are created by using the DeployAssert
construct. This construct creates it's own Stack
separate from
any stacks that you create as part of your integration tests. This Stack
is treated differently from other stacks
by the integ-runner
tool. For example, this stack will not be diffed by the integ-runner
.
DeployAssert
also provides utilities to register your own assertions.
CustomResource myCustomResource;
Stack stack;
App app;
var integ = new IntegTest(app, "Integ", new IntegTestProps { TestCases = new [] { stack } });
integ.Assertions.Expect("CustomAssertion", ExpectedResult.ObjectLike(new Dictionary<string, object> { { "foo", "bar" } }), ActualResult.FromCustomResource(myCustomResource, "data"));
In the above example an assertion is created that will trigger a user defined CustomResource
and assert that the data
attribute is equal to { foo: 'bar' }
.
API Calls
A common method to retrieve the "actual" results to compare with what is expected is to make an API call to receive some data. This library does this by utilizing CloudFormation custom resources which means that CloudFormation will call out to a Lambda Function which will make the API call.
HttpApiCall
Using the HttpApiCall
will use the
node-fetch JavaScript library to
make the HTTP call.
This can be done by using the class directory (in the case of a normal deployment):
Stack stack;
new HttpApiCall(stack, "MyAsssertion", new HttpCallProps {
Url = "https://example-api.com/abc"
});
Or by using the httpApiCall
method on DeployAssert
(when writing integration tests):
App app;
Stack stack;
var integ = new IntegTest(app, "Integ", new IntegTestProps {
TestCases = new [] { stack }
});
integ.Assertions.HttpApiCall("https://example-api.com/abc");
AwsApiCall
Using the AwsApiCall
construct will use the AWS JavaScript SDK to make the API call.
This can be done by using the class directory (in the case of a normal deployment):
Stack stack;
new AwsApiCall(stack, "MyAssertion", new AwsApiCallProps {
Service = "SQS",
Api = "receiveMessage",
Parameters = new Dictionary<string, string> {
{ "QueueUrl", "url" }
}
});
Or by using the awsApiCall
method on DeployAssert
(when writing integration tests):
App app;
Stack stack;
var integ = new IntegTest(app, "Integ", new IntegTestProps {
TestCases = new [] { stack }
});
integ.Assertions.AwsApiCall("SQS", "receiveMessage", new Dictionary<string, string> {
{ "QueueUrl", "url" }
});
You must specify the service
and the api
when using The AwsApiCall
construct.
The service
is the name of an AWS service, in one of the following forms:
The api
is the name of an AWS API call, in one of the following forms:
By default, the AwsApiCall
construct will automatically add the correct IAM policies
to allow the Lambda function to make the API call. It does this based on the service
and api
that is provided. In the above example the service is SQS
and the api is
receiveMessage
so it will create a policy with Action: 'sqs:ReceiveMessage
.
There are some cases where the permissions do not exactly match the service/api call, for
example the S3 listObjectsV2
api. In these cases it is possible to add the correct policy
by accessing the provider
object.
App app;
Stack stack;
IntegTest integ;
var apiCall = integ.Assertions.AwsApiCall("S3", "listObjectsV2", new Dictionary<string, string> {
{ "Bucket", "mybucket" }
});
apiCall.Provider.AddToRolePolicy(new Dictionary<string, object> {
{ "Effect", "Allow" },
{ "Action", new [] { "s3:GetObject", "s3:ListBucket" } },
{ "Resource", new [] { "*" } }
});
Note that addToRolePolicy() uses direct IAM JSON policy blobs, not a iam.PolicyStatement object like you will see in the rest of the CDK.
EqualsAssertion
This library currently provides the ability to assert that two values are equal
to one another by utilizing the EqualsAssertion
class. This utilizes a Lambda
backed CustomResource
which in tern uses the Match utility from the
@aws-cdk/assertions library.
App app;
Stack stack;
Queue queue;
IFunction fn;
var integ = new IntegTest(app, "Integ", new IntegTestProps {
TestCases = new [] { stack }
});
integ.Assertions.InvokeFunction(new LambdaInvokeFunctionProps {
FunctionName = fn.FunctionName,
InvocationType = InvocationType.EVENT,
Payload = JSON.Stringify(new Dictionary<string, string> { { "status", "OK" } })
});
var message = integ.Assertions.AwsApiCall("SQS", "receiveMessage", new Dictionary<string, object> {
{ "QueueUrl", queue.QueueUrl },
{ "WaitTimeSeconds", 20 }
});
message.AssertAtPath("Messages.0.Body", ExpectedResult.ObjectLike(new Dictionary<string, object> {
{ "requestContext", new Dictionary<string, string> {
{ "condition", "Success" }
} },
{ "requestPayload", new Dictionary<string, string> {
{ "status", "OK" }
} },
{ "responseContext", new Dictionary<string, int> {
{ "statusCode", 200 }
} },
{ "responsePayload", "success" }
}));
Match
integ-tests
also provides a Match
utility similar to the @aws-cdk/assertions
module. Match
can be used to construct the ExpectedResult
. While the utility is similar, only a subset of methods are currently available on the Match
utility of this module: arrayWith
, objectLike
, stringLikeRegexp
and serializedJson
.
AwsApiCall message;
message.Expect(ExpectedResult.ObjectLike(new Dictionary<string, object> {
{ "Messages", Match.ArrayWith(new [] { new Dictionary<string, IDictionary<string, IDictionary<string, object>>> {
{ "Payload", Match.SerializedJson(new Dictionary<string, object> { { "key", "value" } }) }
}, new Dictionary<string, IDictionary<string, IDictionary<string, object[]>>> {
{ "Body", new Struct {
Values = Match.ArrayWith(new [] { new Dictionary<string, int> { { "Asdf", 3 } } }),
Message = Match.StringLikeRegexp("message")
} }
} }) }
}));
Examples
Invoke a Lambda Function
In this example there is a Lambda Function that is invoked and we assert that the payload that is returned is equal to '200'.
IFunction lambdaFunction;
App app;
var stack = new Stack(app, "cdk-integ-lambda-bundling");
var integ = new IntegTest(app, "IntegTest", new IntegTestProps {
TestCases = new [] { stack }
});
var invoke = integ.Assertions.InvokeFunction(new LambdaInvokeFunctionProps {
FunctionName = lambdaFunction.FunctionName
});
invoke.Expect(ExpectedResult.ObjectLike(new Dictionary<string, object> {
{ "Payload", "200" }
}));
The above example will by default create a CloudWatch log group that's never
expired. If you want to configure it with custom log retention days, you need
to specify the logRetention
property.
using Amazon.CDK.AWS.Logs;
IFunction lambdaFunction;
App app;
var stack = new Stack(app, "cdk-integ-lambda-bundling");
var integ = new IntegTest(app, "IntegTest", new IntegTestProps {
TestCases = new [] { stack }
});
var invoke = integ.Assertions.InvokeFunction(new LambdaInvokeFunctionProps {
FunctionName = lambdaFunction.FunctionName,
LogRetention = RetentionDays.ONE_WEEK
});
Make an AWS API Call
In this example there is a StepFunctions state machine that is executed and then we assert that the result of the execution is successful.
App app;
Stack stack;
IStateMachine sm;
var testCase = new IntegTest(app, "IntegTest", new IntegTestProps {
TestCases = new [] { stack }
});
// Start an execution
var start = testCase.Assertions.AwsApiCall("StepFunctions", "startExecution", new Dictionary<string, string> {
{ "stateMachineArn", sm.StateMachineArn }
});
// describe the results of the execution
var describe = testCase.Assertions.AwsApiCall("StepFunctions", "describeExecution", new Dictionary<string, string> {
{ "executionArn", start.GetAttString("executionArn") }
});
// assert the results
describe.Expect(ExpectedResult.ObjectLike(new Dictionary<string, object> {
{ "status", "SUCCEEDED" }
}));
Chain ApiCalls
Sometimes it may be necessary to chain API Calls. Since each API call is its own resource, all you
need to do is add a dependency between the calls. There is an helper method next
that can be used.
IntegTest integ;
integ.Assertions.AwsApiCall("S3", "putObject", new Dictionary<string, string> {
{ "Bucket", "my-bucket" },
{ "Key", "my-key" },
{ "Body", "helloWorld" }
}).Next(integ.Assertions.AwsApiCall("S3", "getObject", new Dictionary<string, string> {
{ "Bucket", "my-bucket" },
{ "Key", "my-key" }
}));
Wait for results
A common use case when performing assertions is to wait for a condition to pass. Sometimes the thing
that you are asserting against is not done provisioning by the time the assertion runs. In these
cases it is possible to run the assertion asynchronously by calling the waitForAssertions()
method.
Taking the example above of executing a StepFunctions state machine, depending on the complexity of the state machine, it might take a while for it to complete.
App app;
Stack stack;
IStateMachine sm;
var testCase = new IntegTest(app, "IntegTest", new IntegTestProps {
TestCases = new [] { stack }
});
// Start an execution
var start = testCase.Assertions.AwsApiCall("StepFunctions", "startExecution", new Dictionary<string, string> {
{ "stateMachineArn", sm.StateMachineArn }
});
// describe the results of the execution
var describe = testCase.Assertions.AwsApiCall("StepFunctions", "describeExecution", new Dictionary<string, string> {
{ "executionArn", start.GetAttString("executionArn") }
}).Expect(ExpectedResult.ObjectLike(new Dictionary<string, object> {
{ "status", "SUCCEEDED" }
})).WaitForAssertions();
When you call waitForAssertions()
the assertion provider will continuously make the awsApiCall
until the
ExpectedResult
is met. You can also control the parameters for waiting, for example:
IntegTest testCase;
IApiCall start;
var describe = testCase.Assertions.AwsApiCall("StepFunctions", "describeExecution", new Dictionary<string, string> {
{ "executionArn", start.GetAttString("executionArn") }
}).Expect(ExpectedResult.ObjectLike(new Dictionary<string, object> {
{ "status", "SUCCEEDED" }
})).WaitForAssertions(new WaiterStateMachineOptions {
TotalTimeout = Duration.Minutes(5),
Interval = Duration.Seconds(15),
BackoffRate = 3
});
Classes
ActualResult | (experimental) Represents the "actual" results to compare. |
ApiCallBase | (experimental) Base class for an ApiCall. |
AssertionRequest | (experimental) A request to make an assertion that the actual value matches the expected. |
AssertionResult | (experimental) The result of an Assertion wrapping the actual result data in another struct. |
AssertionResultData | (experimental) The result of an assertion. |
AssertionsProvider | (experimental) Represents an assertions provider. |
AssertionsProviderProps | (experimental) Properties for defining an AssertionsProvider. |
AssertionType | (experimental) The type of assertion to perform. |
AwsApiCall | (experimental) Construct that creates a custom resource that will perform a query using the AWS SDK. |
AwsApiCallOptions | (experimental) Options to perform an AWS JavaScript V2 API call. |
AwsApiCallProps | (experimental) Construct that creates a custom resource that will perform a query using the AWS SDK. |
AwsApiCallRequest | (experimental) A AWS JavaScript SDK V2 request. |
AwsApiCallResult | (experimental) The result from a SdkQuery. |
EqualsAssertion | (experimental) Construct that creates a CustomResource to assert that two values are equal. |
EqualsAssertionProps | (experimental) Options for an EqualsAssertion. |
ExpectedResult | (experimental) Represents the "expected" results to compare. |
FetchOptions | (experimental) Options to pass to the JavaScript fetch api. |
HttpApiCall | (experimental) Construct that creates a custom resource that will perform an HTTP API Call. |
HttpCallProps | (experimental) Options for creating an HttpApiCall provider. |
HttpRequest | (experimental) Request to the HttpCall resource. |
HttpRequestParameters | |
HttpResponse | (experimental) Response from fetch. |
HttpResponseWrapper | (experimental) Response from the HttpCall resource. |
IntegTest | (experimental) A collection of test cases. |
IntegTestCase | (experimental) An integration test case. Allows the definition of test properties that apply to all stacks under this case. |
IntegTestCaseProps | (experimental) Properties of an integration test case. |
IntegTestCaseStack | (experimental) An integration test case stack. Allows the definition of test properties that should apply to this stack. |
IntegTestCaseStackProps | (experimental) Properties of an integration test case stack. |
IntegTestProps | (experimental) Integration test properties. |
InvocationType | (experimental) The type of invocation. |
LambdaFunctionProviderProps | (experimental) Properties for a lambda function provider. |
LambdaInvokeFunction | (experimental) An AWS Lambda Invoke function API call. |
LambdaInvokeFunctionProps | (experimental) Options to pass to the Lambda invokeFunction API call. |
LogType | (experimental) Set to Tail to include the execution log in the response. |
Match | (experimental) Partial and special matching during assertions. |
Status | (experimental) The status of the assertion. |
WaiterStateMachine | (experimental) A very simple StateMachine construct highly customized to the provider framework. |
WaiterStateMachineOptions | (experimental) Options for creating a WaiterStateMachine. |
WaiterStateMachineProps | (experimental) Props for creating a WaiterStateMachine. |
Interfaces
IApiCall | (experimental) Represents an ApiCall. |
IAssertionRequest | (experimental) A request to make an assertion that the actual value matches the expected. |
IAssertionResult | (experimental) The result of an Assertion wrapping the actual result data in another struct. |
IAssertionResultData | (experimental) The result of an assertion. |
IAssertionsProviderProps | (experimental) Properties for defining an AssertionsProvider. |
IAwsApiCallOptions | (experimental) Options to perform an AWS JavaScript V2 API call. |
IAwsApiCallProps | (experimental) Construct that creates a custom resource that will perform a query using the AWS SDK. |
IAwsApiCallRequest | (experimental) A AWS JavaScript SDK V2 request. |
IAwsApiCallResult | (experimental) The result from a SdkQuery. |
IDeployAssert | (experimental) Interface that allows for registering a list of assertions that should be performed on a construct. |
IEqualsAssertionProps | (experimental) Options for an EqualsAssertion. |
IFetchOptions | (experimental) Options to pass to the JavaScript fetch api. |
IHttpCallProps | (experimental) Options for creating an HttpApiCall provider. |
IHttpRequest | (experimental) Request to the HttpCall resource. |
IHttpRequestParameters | |
IHttpResponse | (experimental) Response from fetch. |
IHttpResponseWrapper | (experimental) Response from the HttpCall resource. |
IIntegTestCaseProps | (experimental) Properties of an integration test case. |
IIntegTestCaseStackProps | (experimental) Properties of an integration test case stack. |
IIntegTestProps | (experimental) Integration test properties. |
ILambdaFunctionProviderProps | (experimental) Properties for a lambda function provider. |
ILambdaInvokeFunctionProps | (experimental) Options to pass to the Lambda invokeFunction API call. |
IWaiterStateMachineOptions | (experimental) Options for creating a WaiterStateMachine. |
IWaiterStateMachineProps | (experimental) Props for creating a WaiterStateMachine. |