Device Advisor detailed console workflow

In the Amazon IoT console, in the navigation pane, expand Test, Device Advisor and then choose Test suites.

Choose Create Test Suite.

Select either Use the Amazon Qualification test suite or Create a new test suite.

For protocol, choose either MQTT 3.1.1 or MQTT 5.

Select Use the Amazon Qualification test suite to qualify and list your device to the Amazon Partner Device Catalog. By choosing this option, test cases required for qualification of your device to the Amazon IoT Core qualification program are pre-selected. Test groups and test cases can't be added or removed. You will still need to configure the test suite properties.

Select Create a new test suite to create and configure a custom test suite. We recommend starting with this option for initial testing and troubleshooting. A custom test suite must have at least one test group, and each test group must have at least one test case. For the purpose of this tutorial, we'll select this option and choose Next.

Choose Test suite properties. You must create the test suite properties when you create your test suite.

Under Test suite properties, fill out the following:

When you've finished, choose Update properties.

To modify the group level configuration, under Test group 1, choose Edit. Then, enter a Name to give the group a custom name.

Optionally, you can also enter a Timeout value in seconds under the selected test group. If you don't specify a timeout value, the default value is used.

Choose Done.

Drag one of the available test cases from Test cases into the test group.

To modify the test case level configuration for the test case that you added to your test group, choose Edit. Then, enter a Name to give the group a custom name.

Optionally, you can also enter a Timeout value in seconds under the selected test group. If you don't specify a timeout value, the default value is used.

Choose Done.

Choose Next.

In Step 3, configure a device role which Device Advisor will use to perform Amazon IoT MQTT actions on behalf of your test device.

If you selected MQTT Connect test case only in Step 2, the Connect action will be checked automatically since that permission is required on device role to run this test suite. If you selected other test cases, the corresponding required actions will be checked. Ensure that the resource values values for each of the actions is provided. For example, for the Connect action, provide the client id that your device will be connecting to the Device Advisor endpoint with. You can provide multiple values by using commas to separate the values, and you can provide prefix values using a wildcard (*) character as well. For example, to provide permission to publish on any topic beginning with MyTopic, you can provide “MyTopic*” as the resource value.

If you already created a device role previously and would like to use that role, select Select an existing role and choose your device role under Select role.

Configure your device role using one of the two provided options and choose Next.

In Step 4, make sure the configuration provided in each of the steps is accurate. To edit configuration provided for a particular step, choose Edit for the corresponding step.

After you verify the configuration, choose Create test suite.

The test suite should be created successfully and you'll be redirected to the Test suites page where you can view all the test suite that have been created.

If the test suite creation failed, make sure the test suite, test groups, test cases, and device role have been configured according to the previous instructions.

In the Amazon IoT console, in the navigation pane, expand Test, Device Advisor, and then choose Test suites.

Choose the test suite for which you'd like to view the test suite details.

The test suite detail page displays all of the information related to the test suite.

Choose Actions, then Run test suite.

Under Run configuration, you'll need to select an Amazon IoT thing or certificate to test using Device Advisor. If you don't have any existing things or certificates, first create Amazon IoT Core resources.

In Test endpoint section, select the endpoint that best fits your case. If you plan to run multiple test suites simultaneously using the same Amazon account in the future, select Device-level endpoint. Otherwise, if you plan to only run one test suite at a time, select Account-level endpoint.

Configure your test device with the selected Device Advisor's test endpoint.

After you select a thing or certificate and choose a Device Advisor endpoint, choose Run test.

Choose Go to results on the top banner for viewing the test run details.

In the Amazon IoT console, in the navigation pane, expand Test, Device Advisor, and then choose Test runs and results.

Choose the test suite in progress that you want to stop.

Choose Actions, then Stop test suite.

The cleanup process will take several minutes to complete. While the cleanup process runs, the test run status will be STOPPING. Wait for the cleanup process to complete and for the test suite status to change to the STOPPED status before starting a new suite run.

In the Amazon IoT console, in the navigation pane, expand Test, Device Advisor and then choose Test runs and results.

This page displays:

Choose the test suite for which you'd like to view the run details and logs.

The run summary page displays the status of the current test suite run. This page automatically refreshes every 10 seconds. We recommend that you have a mechanism built for your device to try connecting to our test endpoint every five seconds for one to two minutes. Then you can run multiple test cases in sequence in an automated manner.

To access the CloudWatch logs for the test suite run, choose Test suite log.

To access CloudWatch logs for any test case, choose Test case log.

Based on your test results, troubleshoot your device until all tests pass.