Preparing to test your microcontroller board for the first time - FreeRTOS
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Preparing to test your microcontroller board for the first time

You can use IDT for FreeRTOS to test your implementation of the FreeRTOS libraries. After you have ported the FreeRTOS libraries for your board’s device drivers, use Amazon IoT Device Tester to run the qualification tests on your microcontroller board.

Add library porting layers and implement a FreeRTOS tests repository

To port FreeRTOS for your device, see the FreeRTOS Porting Guide. When implementing the FreeRTOS tests repository and porting the FreeRTOS layers, you must provide a manifest.yml with paths to each library, including the tests repository. This file will be in the root directory of your source code. See manifest file instructions for details.

Configure your Amazon credentials

You need to configure your Amazon credentials for Amazon IoT Device Tester to communicate with the Amazon Cloud. For more information, see Set up Amazon Credentials and Region for Development. Valid Amazon credentials are specified in the devicetester_extract_location/devicetester_freertos_[win|mac|linux]/configs/config.json configuration file.

"auth": { "method": "environment" } "auth": { "method": "file", "credentials": { "profile": "<your-aws-profile>" } }

The auth attribute of the config.json file has a method field that controls Amazon authentication, and can be declared as either file or environment. Setting the field to environment pulls your Amazon credentials from your host machine’s environment variables. Setting the field to file imports a specified profile from the .aws/credentials config file.

Create a device pool in IDT for FreeRTOS

Devices to be tested are organized in device pools. Each device pool consists of one or more identical devices. You can configure IDT for FreeRTOS to test a single device, or multiple devices in a pool. To accelerate the qualification process, IDT for FreeRTOS can test devices with the same specifications in parallel. It uses a round-robin method to execute a different test group on each device in a device pool.

The device.json file has an array in its top level. Each array attribute is a new device pool. Each device pool has a devices array attribute, which has multiple devices declared. In the template, there is a device pool and only one device in that device pool. You can add one or more devices to a device pool by editing the devices section of the device.json template in the configs folder.

Note

All devices in the same pool must be of the same technical specification and SKU. To enable parallel builds of the source code for different test groups, IDT for FreeRTOS copies the source code to a results folder inside the IDT for FreeRTOS extracted folder. You must reference the source code path in your build or flash command using the testdata.sourcePath variable. IDT for FreeRTOS replaces this variable with a temporary path of the copied source code. For more information, see IDT for FreeRTOS variables.

The following is an example device.json file was used to create a device pool with multiple devices.

[ { "id": "pool-id", "sku": "sku", "features": [ { "name": "Wifi", "value": "Yes | No" }, { "name": "Cellular", "value": "Yes | No" }, { "name": "BLE", "value": "Yes | No" }, { "name": "PKCS11", "value": "RSA | ECC | Both" }, { "name": "OTA", "value": "Yes | No", "configs": [ { "name": "OTADataPlaneProtocol", "value": "MQTT | HTTP | None" } ] }, { "name": "KeyProvisioning", "value": "Onboard | Import | Both | No" } ], "devices": [ { "id": "device-id", "connectivity": { "protocol": "uart", "serialPort": "/dev/tty*" }, "secureElementConfig" : { "publicKeyAsciiHexFilePath": "absolute-path-to/public-key-txt-file: contains-the-hex-bytes-public-key-extracted-from-onboard-private-key", "publiDeviceCertificateArn": "arn:partition:iot:region:account-id:resourcetype:resource:qualifier", "secureElementSerialNumber": "secure-element-serialNo-value", "preProvisioned" : "Yes | No", "pkcs11JITPCodeVerifyRootCertSupport": "Yes | No" }, "identifiers": [ { "name": "serialNo", "value": "serialNo-value" } ] } ] } ]

The following attributes are used in the device.json file:

id

A user-defined alphanumeric ID that uniquely identifies a pool of devices. Devices that belong to a pool must be of the same type. When a suite of tests is running, devices in the pool are used to parallelize the workload.

sku

An alphanumeric value that uniquely identifies the board you are testing. The SKU is used to track qualified boards.

Note

If you want to list your board in Amazon Partner Device Catalog, the SKU you specify here must match the SKU that you use in the listing process.

features

An array that contains the device's supported features. Amazon IoT Device Tester uses this information to select the qualification tests to run.

Supported values are:

Wifi

Indicates if your board has Wi-Fi capabilities.

Cellular

Indicates if your board has cellular capabilities.

PKCS11

Indicates the public key cryptography algorithm that the board supports. PKCS11 is required for qualification. Supported values are ECC, RSA, and Both. Both indicates the board supports both ECC and RSA.

KeyProvisioning

Indicates the method of writing a trusted X.509 client certificate onto your board.

Valid values are Import, Onboard, Both and No. Onboard, Both, or No key provisioning is required for qualification. Import alone is not a valid option for qualification.

  • Use Import only if your board allows the import of private keys. Selecting Import is not a valid configuration for qualification and should be used only for testing purposes, specifically with PKCS11 test cases. Onboard, Both or No is required for qualification.

  • Use Onboard if your board supports on-board private keys (for example, if your device has a secure element, or if you prefer to generate your own device key pair and certificate). Make sure you add a secureElementConfig element in each of the device sections and put the absolute path to the public key file in the publicKeyAsciiHexFilePath field.

  • Use Both if your board supports both importing private keys and on-board key generation for key provisioning.

  • Use No if your board doesn't support key provisioning. Nois only a valid option when your device is also pre-provisioned.

OTA

Indicates if your board supports over-the-air (OTA) update functionality. The OtaDataPlaneProtocol attribute indicates which OTA dataplane protocol the device supports. OTA with either HTTP or MQTT dataplane protocol is required for qualification. To skip running OTA tests while testing, set the OTA feature to No and the OtaDataPlaneProtocol attribute to None. This will not be a qualification run.

BLE

Indicates if your board supports Bluetooth Low Energy (BLE).

devices.id

A user-defined unique identifier for the device being tested.

devices.connectivity.serialPort

The serial port of the host computer used to connect to the devices being tested.

devices.secureElementConfig.PublicKeyAsciiHexFilePath

Required if your board is NOT pre-provisioned or PublicDeviceCertificateArn is not provided. Since Onboardis a required type of Key Provisioning, this field is currently required for the FullTransportInterfaceTLS test group. If your device is pre-provisioned, PublicKeyAsciiHexFilePath is optional and need not be included.

The following block is an absolute path to the file that contains the hex bytes public key extracted from Onboard private key.

3059 3013 0607 2a86 48ce 3d02 0106 082a 8648 ce3d 0301 0703 4200 04cd 6569 ceb8 1bb9 1e72 339f e8cf 60ef 0f9f b473 33ac 6f19 1813 6999 3fa0 c293 5fae 08f1 1ad0 41b7 345c e746 1046 228e 5a5f d787 d571 dcb2 4e8d 75b3 2586 e2cc 0c

If your public key is in .der format, you can hex encode the public key directly to generate the hex file.

To generate the hex file from a .der public key, enter the following xxd command:

xxd -p pubkey.der > outFile

If your public key is in .pem format, you can extract the base64 encoded headers and footers and decode it into binary format. Then, you hex encode the binary string to generate the hex file.

To generate a hex file for a .pem public key, do the following:

  1. Run the following base64 command to remove the base64 header and footer from the public key. The decoded key, named base64key, is then output to the file pubkey.der:

    base64 —decode base64key > pubkey.der
  2. Run the following xxd command to convert pubkey.der to hex format. The resulting key is saved as outFile

    xxd -p pubkey.der > outFile
devices.secureElementConfig.PublicDeviceCertificateArn

The ARN of the certificate from your secure element that is uploaded to Amazon IoT Core. For information about uploading your certificate to Amazon IoT Core, see X.509 client certificates in the Amazon IoT Developer Guide.

devices.secureElementConfig.SecureElementSerialNumber

(Optional) The serial number of the secure element. The serial number is optionally used to create device certificates for JITR key provisioning.

devices.secureElementConfig.preProvisioned

(Optional) Set to "Yes" if the device has a pre-provisioned secure element with locked-down credentials, that cannot import, create, or destroy objects. If this attribute is set to Yes, you must provide the corresponding pkcs11 labels.

devices.secureElementConfig.pkcs11JITPCodeVerifyRootCertSupport

(Optional) Set to Yes if the device’s corePKCS11 implementation supports storage for JITP. This will enable the JITP codeverify test when testing core PKCS 11, and requires code verification key, JITP certificate, and root certificate PKCS 11 labels to be provided.

identifiers

(Optional) An array of arbitrary name-value pairs. You can use these values in the build and flash commands described in the next section.

Configure build, flash, and test settings

IDT for FreeRTOS builds and flashes tests on to your board automatically. To enable this, you must configure IDT to run the build and flash commands for your hardware. The build and flash command settings are configured in the userdata.json template file located in the config folder.

Configure settings for testing devices

Build, flash, and test settings are made in the configs/userdata.json file. The following JSON example shows how you can configure IDT for FreeRTOS to test multiple devices:

{ "sourcePath": "</path/to/freertos>", "retainModifiedSourceDirectories": true | false, "freeRTOSVersion": "<freertos-version>", "freeRTOSTestParamConfigPath": "{{testData.sourcePath}}/path/from/source/path/to/test_param_config.h", "freeRTOSTestExecutionConfigPath": "{{testData.sourcePath}}/path/from/source/path/to/test_execution_config.h", "buildTool": { "name": "your-build-tool-name", "version": "your-build-tool-version", "command": [ "<build command> -any-additional-flags {{testData.sourcePath}}" ] }, "flashTool": { "name": "your-flash-tool-name", "version": "your-flash-tool-version", "command": [ "<flash command> -any-additional-flags {{testData.sourcePath}} -any-additional-flags" ] }, "testStartDelayms": 0, "echoServerConfiguration": { "keyGenerationMethod": "EC | RSA", "serverPort": 9000 }, "otaConfiguration": { "otaE2EFirmwarePath": "{{testData.sourcePath}}/relative-path-to/ota-image-generated-in-build-process", "otaPALCertificatePath": "/path/to/ota/pal/certificate/on/device", "deviceFirmwarePath" : "/path/to/firmware/image/name/on/device", "codeSigningConfiguration": { "signingMethod": "Amazon | Custom", "signerHashingAlgorithm": "SHA1 | SHA256", "signerSigningAlgorithm": "RSA | ECDSA", "signerCertificate": "arn:partition:service:region:account-id:resource:qualifier | /absolute-path-to/signer-certificate-file", "untrustedSignerCertificate": "arn:partition:service:region:account-id:resourcetype:resource:qualifier", "signerCertificateFileName": "signerCertificate-file-name", "compileSignerCertificate": true | false, // ***********Use signerPlatform if you choose Amazon for signingMethod*************** "signerPlatform": "AmazonFreeRTOS-Default | AmazonFreeRTOS-TI-CC3220SF" ] } }, ********** This section is used for PKCS #11 labels of private key, public key, device certificate, code verification key, JITP certificate, and root certificate. When configuring PKCS11, you set up labels and you must provide the labels of the device certificate, public key, and private key for the key generation type (EC or RSA) it was created with. If your device supports PKCS11 storage of JITP certificate, code verification key, and root certificate, set 'pkcs11JITPCodeVerifyRootCertSupport' to 'Yes' in device.json and provide the corresponding labels. ********** "pkcs11LabelConfiguration":{ "pkcs11LabelDevicePrivateKeyForTLS": "<device-private-key-label>", "pkcs11LabelDevicePublicKeyForTLS": "<device-public-key-label>", "pkcs11LabelDeviceCertificateForTLS": "<device-certificate-label>", "pkcs11LabelPreProvisionedECDevicePrivateKeyForTLS": "<preprovisioned-ec-device-private-key-label>", "pkcs11LabelPreProvisionedECDevicePublicKeyForTLS": "<preprovisioned-ec-device-public-key-label>", "pkcs11LabelPreProvisionedECDeviceCertificateForTLS": "<preprovisioned-ec-device-certificate-label>", "pkcs11LabelPreProvisionedRSADevicePrivateKeyForTLS": "<preprovisioned-rsa-device-private-key-label>", "pkcs11LabelPreProvisionedRSADevicePublicKeyForTLS": "<preprovisioned-rsa-device-public-key-label>", "pkcs11LabelPreProvisionedRSADeviceCertificateForTLS": "<preprovisioned-rsa-device-certificate-label>", "pkcs11LabelCodeVerifyKey": "<code-verification-key-label>", "pkcs11LabelJITPCertificate": "<JITP-certificate-label>", "pkcs11LabelRootCertificate": "<root-certificate-label>" } }

The following lists the attributes used in userdata.json:

sourcePath

The path to the root of the ported FreeRTOS source code.

retainModifiedSourceDirectories

(Optional) Checks if to retain the modified source directories used during building and flashing for debugging purposes. If set to true, the modified source directories are named retainedSrc and found within the results log folders in each test group run. If not included, the field defaults to false.

freeRTOSTestParamConfigPath

The path to test_param_config.h file for FreeRTOS-Libraries-Integration-Tests integration. This file must use the {{testData.sourcePath}} placeholder variable to make it relative to the source code root. Amazon IoT Device Tester uses the parameters in this file to configure the tests.

freeRTOSTestExecutionConfigPath

The path to test_execution_config.h file for FreeRTOS-Libraries-Integration-Tests integration. This file must use the {{testData.sourcePath}} placeholder variable to make it relative to the repository root. Amazon IoT Device Tester uses this file to control which tests must run.

freeRTOSVersion

The version of FreeRTOS including the patch version used in your implementation. See Supported versions of Amazon IoT Device Tester for FreeRTOS for the FreeRTOS versions compatible with Amazon IoT Device Tester for FreeRTOS.

buildTool

The command to build your source code. All references to the source code path in the build command must be replaced by the Amazon IoT Device Tester variable {{testData.sourcePath}}. Use the {{config.idtRootPath}} placeholder to reference a build script relative to the Amazon IoT Device Tester root path.

flashTool

The command to flash an image to your device. All references to the source code path in the flash command must be replaced by the Amazon IoT Device Tester variable {{testData.sourcePath}}. Use the {{config.idtRootPath}} placeholder to reference a flash script relative to the Amazon IoT Device Tester root path.

Note

The new integration tests structure with FRQ 2.0 doesn't require path variables such as {{enableTests}} and {{buildImageName}}. The OTA End to End tests are run with the config templates provided in the FreeRTOS-Libraries-Integration-Tests GitHub repository. If the files in the GitHub repository are present in your parent source project, the source code isn't changed between tests. If a different build image for OTA End to End is needed, you must build this image in the build script and specify it in the userdata.json file specified under otaConfiguration.

testStartDelayms

Specifies how many milliseconds the FreeRTOS test runner will wait before starting to run tests. This can be useful if the device under test begins to output important test information before IDT has a chance to connect and start logging due to network or other latency issues. This value is applicable to FreeRTOS test groups only, and not to other test groups that do not utilize the FreeRTOS test runner, such as the OTA tests. If you receive an error related to expected 10 but received 5, this field should be set to 5000.

echoServerConfiguration

The configuration to setup the echo server for the TLS test. This field is required.

keyGenerationMethod

The echo server is configured with this option. The options are EC, or RSA.

serverPort

The port number on which the echo server runs.

otaConfiguration

The configuration for OTA PAL and OTA E2E tests. This field is required.

otaE2EFirmwarePath

Path to the OTA bin image that IDT uses for the OTA End to End tests.

otaPALCertificatePath

The path to the certificate for OTA PAL test on device. This is used to verify the signature. For example, ecdsa-sha256-signer.crt.pem.

deviceFirmwarePath

The path to the hard coded name for the firmware image to boot. If your device does NOT use the file system for firmware boot, specify this field as 'NA'. If your device uses the file system for firmware boot, specify the path or name to the firmware boot image.

codeSigningConfiguration
signingMethod

The code signing method. Possible values are Amazon or Custom.

Note

For the Beijing and Ningxia Regions, use Custom. Amazon code signing is not supported in that region.

signerHashingAlgorithm

The hashing algorithm supported on the device. Possible values are SHA1 or SHA256.

signerSigningAlgorithm

The signing algorithm supported on the device. Possible values are RSA or ECDSA.

signerCertificate

The trusted certificate used for OTA. For the Amazon code signing method, use the Amazon Resource Name (ARN) for the trusted certificate uploaded to the Amazon Certificate Manager. For the Custom code signing method, use the absolute path to the signer certificate file. For information about creating a trusted certificate, see Create a code-signing certificate.

untrustedSignerCertificate

The ARN or filepath for a second certificate used in some OTA tests as an untrusted certificate. For information about creating a certificate, see Create a code-signing certificate.

signerCertificateFileName

The file name of the code signing certificate on the device. This value must match the file name that you provided when you ran the aws acm import-certificate command.

compileSignerCertificate

Boolean value that determines the status of the signature verification certificate. Valid values are true and false.

Set this value to true if the code signer signature verification certificate is not provisioned or flashed. It must be compiled into the project. Amazon IoT Device Tester fetches the trusted certificate and compiles it into aws_codesigner_certificate.h.

signerPlatform

The signing and hashing algorithm that Amazon Code Signer uses while creating the OTA update job. Currently, the possible values for this field are AmazonFreeRTOS-TI-CC3220SF and AmazonFreeRTOS-Default.

  • Choose AmazonFreeRTOS-TI-CC3220SF if SHA1 and RSA.

  • Choose AmazonFreeRTOS-Default if SHA256 and ECDSA.

  • If you need SHA256 | RSA or SHA1 | ECDSA for your configuration, contact us for further support.

  • Configure signCommand if you chose Custom for signingMethod.

signCommand

Two placeholders {{inputImageFilePath}} and {{outputSignatureFilePath}} are required in the command. {{inputImageFilePath}} is the file path of the image built by IDT to be signed. {{outputSignatureFilePath}} is the file path of the signature which will be generated by the script.

pkcs11LabelConfiguration

PKCS11 label configuration requires at least one set of labels of device certificate label, public key label, and private key label to run the PKCS11 test groups. The required PKCS11 labels are based on your device configuration in the device.json file. If pre-provisioned is set to Yes in device.json, then the required labels must be one of the below depending on what's chosen for the PKCS11 feature.

  • PreProvisionedEC

  • PreProvisionedRSA

If pre-provisioned is set to No in device.json, then the required labels are:

  • pkcs11LabelDevicePrivateKeyForTLS

  • pkcs11LabelDevicePublicKeyForTLS

  • pkcs11LabelDeviceCertificateForTLS

The following three labels are required only if you select Yes for pkcs11JITPCodeVerifyRootCertSupport in your device.json file.

  • pkcs11LabelCodeVerifyKey

  • pkcs11LabelRootCertificate

  • pkcs11LabelJITPCertificate

The values for these fields should match the values defined in the FreeRTOS Porting Guide.

pkcs11LabelDevicePrivateKeyForTLS

(Optional) This label is used for the PKCS #11 label of the private key. For devices with onboard and import support of key provisioning, this label is used for testing. This label may be different than the one defined for the pre-provisioned case. If you have key provisioning set to No and pre-provisioned set to Yes, in device.json, this will be undefined.

pkcs11LabelDevicePublicKeyForTLS

(Optional) This label is used for the PKCS #11 label of the public key. For devices with onboard and import support of key provisioning, this label is used for testing. This label may be different than the one defined for pre-provisioned case. If you have key provisioning set to No and pre-provisioned set to Yes, in device.json, this will be undefined.

pkcs11LabelDeviceCertificateForTLS

(Optional) This label is used for the PKCS #11 label of the device certificate. For devices with onboard and import support of key provisioning, this label will be used for testing. This label may be different than the one defined for pre-provisioned case. If you have key provisioning set to No and pre-provisioned set to Yes, in device.json, this will be undefined.

pkcs11LabelPreProvisionedECDevicePrivateKeyForTLS

(Optional) This label is used for the PKCS #11 label of the private key. For devices with secure elements or hardware limitations, this will have a different label to preserve Amazon IoT credentials. If your device supports pre-provisioning with an EC key, provide this label. When preProvisioned is set to Yes in device.json, this label, pkcs11LabelPreProvisionedRSADevicePrivateKeyForTLS, or both must be provided. This label may be different than the one defined for onboard and import cases.

pkcs11LabelPreProvisionedECDevicePublicKeyForTLS

(Optional) This label is used for the PKCS #11 label of the public key. For devices with secure elements or hardware limitations, this will have a different label to preserve Amazon IoT credentials. If your device supports pre-provisioning with an EC key, provide this label. When preProvisioned is set to Yes in device.json, this label, pkcs11LabelPreProvisionedRSADevicePublicKeyForTLS, or both must be provided. This label may be different than the one defined for onboard and import cases.

pkcs11LabelPreProvisionedECDeviceCertificateForTLS

(Optional) This label is used for the PKCS #11 label of the device certificate. For devices with secure elements or hardware limitations, this will have a different label to preserve Amazon IoT credentials. If your device supports pre-provisioning with an EC key, provide this label. When preProvisioned is set to Yes in device.json, this label, pkcs11LabelPreProvisionedRSADeviceCertificateForTLS, or both must be provided. This label may be different than the one defined for onboard and import cases.

pkcs11LabelPreProvisionedRSADevicePrivateKeyForTLS

(Optional) This label is used for the PKCS #11 label of the private key. For devices with secure elements or hardware limitations, this will have a different label to preserve Amazon IoT credentials. If your device supports pre-provisioning with an RSA key, provide this label. When preProvisioned is set to Yes in device.json, this label, pkcs11LabelPreProvisionedECDevicePrivateKeyForTLS, or both must be provided.

pkcs11LabelPreProvisionedRSADevicePublicKeyForTLS

(Optional) This label is used for the PKCS #11 label of the public key. For devices with secure elements or hardware limitations, this will have a different label to preserve Amazon IoT credentials. If your device supports pre-provisioning with an RSA key, provide this label. When preProvisioned is set to Yes in device.json, this label, pkcs11LabelPreProvisionedECDevicePublicKeyForTLS, or both must be provided.

pkcs11LabelPreProvisionedRSADeviceCertificateForTLS

(Optional) This label is used for the PKCS #11 label of the device certificate. For devices with secure elements or hardware limitations, this will have a different label to preserve Amazon IoT credentials. If your device supports pre-provisioning with an RSA key, provide this label. When preProvisioned is set to Yes in device.json, this label, pkcs11LabelPreProvisionedECDeviceCertificateForTLS, or both must be provided.

pkcs11LabelCodeVerifyKey

(Optional) This label is used for the PKCS #11 label of the code verification key. If your device has PKCS #11 storage support of the JITP certificate, code verification key, and root certificate, provide this label. When pkcs11JITPCodeVerifyRootCertSupport in device.json is set to Yes, this label must be provided.

pkcs11LabelJITPCertificate

(Optional) This label is used for the PKCS #11 label of the JITP certificate. If your device has PKCS #11 storage support of the JITP certificate, code verification key, and root certificate, provide this label. When pkcs11JITPCodeVerifyRootCertSupport in device.json is set to Yes, this label must be provided.

IDT for FreeRTOS variables

The commands to build your code and flash the device might require connectivity or other information about your devices to run successfully. Amazon IoT Device Tester allows you to reference device information in flash and build commands using JsonPath. By using simple JsonPath expressions, you can fetch the required information specified in your device.json file.

Path variables

IDT for FreeRTOS defines the following path variables that can be used in command lines and configuration files:

{{testData.sourcePath}}

Expands to the source code path. If you use this variable, it must be used in both the flash and build commands.

{{device.connectivity.serialPort}}

Expands to the serial port.

{{device.identifiers[?(@.name == 'serialNo')].value[0]}}

Expands to the serial number of your device.

{{config.idtRootPath}}

Expands to the Amazon IoT Device Tester root path.