Example experiment workflow
At a high level, the process of creating, running, and ending an experiment using Amazon AppConfig experimentation works as follows:
Define the experiment objective, including a name, hypothesis, and launch criteria. For more information, see Documenting your hypothesis. As an Amazon best practice, spend sufficient time on this: the most successful experiments have robust hypotheses and launch criteria.
Complete prerequisites, including configuring IAM permissions and installing Amazon AppConfig Agent. For more information, see Step 1: Configuring prerequisites.
Configure an analytics platform to capture experiment data. This may be CloudWatch, Snowflake, Datadog, or other.
Define the target audience using rules that filter users by attributes such as region, account type, or device. For more information, see Specifying the target audience for the experiment.
Select an existing feature flag to control treatment delivery. For more information, see Selecting the experiment feature flag.
Add treatments that define the control and one or more variations to evaluate. For more information, see About controls and treatments.
Start the experiment at 0% exposure, validate with overrides, then gradually increase exposure while monitoring metrics. For more information, see About running and monitoring an experiment.
In your data warehouse of choice, analyze results, roll out the winning treatment, and clean up the experiment. For more information, see Promoting a winning treatment.
For more detailed information about Amazon AppConfig experimentation processes and tasks, including screenshots with values for a sample experiment called "Implement add-to-cart button for all products displayed", see About experiments in Amazon AppConfig. To get started with an experiment, see Creating and running an experiment.