Step 2: Creating an experiment definition

To create an experiment definition

1. Open the Amazon Systems Manager console at https://console.amazonaws.cn/systems-manager/appconfig/.

2. In the navigation pane, choose Experiments, and then choose Create experiment.

3. For Experiment Name, enter a descriptive name to distinguish this experiment from other experiments.

4. For Experiment hypothesis, describe the goal or hypothesis you want to validate.

5. For Launch criteria, enter information about metrics or data essential to deploying a winning treatment. This information is helpful for long-running experiments.

6. For Experiment application, select the application where this experiment will run.

7. Choose Next.

8. On the Specify target audience page, for Audience description, enter information about the intended audience for the experiment.

9. In the Audience rule section, choose either the Rule builder tab or the Editor tab to create a rule that identifies or filters the audience. Choose Rule blueprints to view and select a pre-constructed rule for your experiment. When building a rule:

   • Add a condition

     Choose an attribute, select an operator, and specify a value.

   • Group conditions

     Use AND to require all conditions in a group to be true, or OR to match any condition.

   • Nest groups

     Combine multiple groups to create more advanced targeting logic.

   • Refine the rule

     Add or remove conditions and groups as needed to match your targeting requirements.

   For more information about building rules, including expression formats, supported operators, and rule examples, see Creating audience rules.

10. Choose Next.

11. On the Select experiment feature flag page, for Environment, select the environment where this experiment will run. For Configuration profile, select the configuration profile associated with the feature flag.

12. For Feature flag, select an existing feature flag to use with the experiment. For more information about experiment feature flags, see Selecting the experiment feature flag.

13. Choose Next.

14. On the Add treatments page, review the Default flag value section. When audience exposure is set between 0% and 99%, users not assigned to a treatment receive the default flag value. This value is always off and cannot be changed.

15. On the Add treatments page, in the Control section, enter a description and verify the flag value is toggled ON.

    Note

    The control treatment represents the baseline experience used for comparison during the experiment. It is typically the current default behavior, but can be any stable, well-understood configuration. The default feature flag value (which is toggled off by default) defines the fallback behavior for users who are not assigned to a treatment. Although the control treatment and default flag value often deliver the same user experience, only users assigned to the control treatment participate in experiment measurement and analysis.

    If one or more keys are listed in the Attribute values section, specify their values in the corresponding Value field.

16. In the Treatment 1 section, enter a description and verify the flag value is toggled ON. If one or more keys are listed in the Attribute values section, specify their values in the corresponding Value field. Repeat this process for any additional treatments.

17. Amazon AppConfig experimentation automatically allocates traffic to all treatments based on the total number of treatments. If you want to allocate traffic to different treatments, you can expand Advanced settings, and choose Use custom allocation weights (not recommended).

    Note

    Treatment traffic allocation determines how traffic is split among treatments. Audience exposure, which controls what percentage of the eligible audience enters the experiment, is configured separately when you start the experiment run.

18. Choose Next.

19. Review the details, and then choose Save experiment definition. Amazon AppConfig displays the completed definition page for the experiment.