Stream manager
The stream manager component (aws.greengrass.StreamManager
) enables you to process data
streams to transfer to the Amazon Web Services Cloud from Greengrass core devices.
For more information about how to configure and use stream manager in custom components, see Manage data streams on Greengrass core devices.
Versions
This component has the following versions:
-
2.1.x
-
2.0.x
If you use stream manager to export data to the cloud, you can't upgrade version 2.0.7 of the stream manager component to a version between v2.0.8 and v2.0.11. If you are deploying stream manager for the first time, we strongly recommend that you deploy the latest version of the stream manager component.
Type
This component is a generic component (aws.greengrass.generic
). The Greengrass nucleus runs the component's lifecycle
scripts.
For more information, see Component types.
Operating system
This component can be installed on core devices that run the following operating systems:
Linux
Windows
Requirements
This component has the following requirements:
-
The token exchange role must allow access to the Amazon Web Services Cloud destinations that you use with stream manager. For more information, see:
Endpoints and ports
This component must be able to perform outbound requests to the following endpoints and ports, in addition to endpoints and ports required for basic operation. For more information, see Allow device traffic through a proxy or firewall.
Endpoint | Port | Required | Description |
---|---|---|---|
|
443 | No |
Required if you publish data to Amazon IoT Analytics. |
|
443 | No |
Required if you publish data to Kinesis Data Firehose. |
|
443 | No |
Required if you publish data to Amazon IoT SiteWise. |
|
443 | No |
Required if you publish data to S3 buckets. You can replace |
Dependencies
When you deploy a component, Amazon IoT Greengrass also deploys compatible versions of its dependencies. This means that you must meet the requirements for the component and all of its dependencies to successfully deploy the component. This section lists the dependencies for the released versions of this component and the semantic version constraints that define the component versions for each dependency. You can also view the dependencies for each version of the component in the Amazon IoT Greengrass console
For more information about component dependencies, see the component recipe reference.
Configuration
This component provides the following configuration parameters that you can customize when you deploy the component.
STREAM_MANAGER_STORE_ROOT_DIR
-
(Optional) The absolute path of the local directory used to store streams. This value must start with a forward slash (for example,
/data
).You must specify an existing folder, and the system user who runs the stream manager component must have permissions to read and write to this folder. For example, you can run the following commands to create and configure a folder,
/var/greengrass/streams
, which you specify as the stream manager root folder. These commands allow the default system user,ggc_user
, to read and write to this folder.sudo mkdir /var/greengrass/streams sudo chown ggc_user /var/greengrass/streams sudo chmod 700 /var/greengrass/streams
Default:
/greengrass/v2
/work/aws.greengrass.StreamManager STREAM_MANAGER_SERVER_PORT
-
(Optional) The local port number to use to communicate with stream manager.
You can specify
0
to use a random available port.Default:
8088
STREAM_MANAGER_AUTHENTICATE_CLIENT
-
(Optional) You can make it mandatory for clients to authenticate before they can interact with stream manager. The Stream Manager SDK controls interaction between clients and stream manager. This parameter determines which clients can call the Stream Manager SDK to work with streams. For more information, see stream manager client authentication.
If you specify
true
, the Stream Manager SDK allows only Greengrass components as clients.If you specify
false
, the Stream Manager SDK allows all processes on the core device to be clients.Default:
true
STREAM_MANAGER_EXPORTER_MAX_BANDWIDTH
-
(Optional) The average maximum bandwidth (in kilobits per second) that stream manager can use to export data.
Default: No limit
STREAM_MANAGER_THREAD_POOL_SIZE
-
(Optional) The maximum number of active threads that stream manager can use to export data.
The optimal size depends on your hardware, stream volume, and planned number of export streams. If your export speed is slow, you can adjust this setting to find the optimal size for your hardware and business case. The CPU and memory of your core device hardware are limiting factors. To start, you might try setting this value equal to the number of processor cores on the device.
Be careful not to set a size that's higher than your hardware can support. Each stream consumes hardware resources, so try to limit the number of export streams on constrained devices.
Default: 5 threads
STREAM_MANAGER_EXPORTER_S3_DESTINATION_MULTIPART_UPLOAD_MIN_PART_SIZE_BYTES
-
(Optional) The minimum size (in bytes) of a part in a multipart upload to Amazon S3. Stream manager uses this setting and the size of the input file to determine how to batch data in a multipart PUT request.
Note Stream manager uses the streams
sizeThresholdForMultipartUploadBytes
property to determine whether to export to Amazon S3 as a single or multipart upload. Amazon IoT Greengrass components can set this threshold when they create a stream that exports to Amazon S3.Default:
5242880
(5 MB). This is also the minimum value. LOG_LEVEL
-
(Optional) The logging level for the component. Choose from the following log levels, listed here in level order:
-
TRACE
-
DEBUG
-
INFO
-
WARN
-
ERROR
Default:
INFO
-
JVM_ARGS
-
(Optional) The custom Java Virtual Machine arguments to pass to stream manager at startup. Separate multiple arguments by spaces.
Use this parameter only when you must override the default settings used by the JVM. For example, you might need to increase the default heap size if you plan to export a large number of streams.
Example: Configuration merge update
The following example configuration specifies to use a non-default port.
{ "STREAM_MANAGER_SERVER_PORT": "18088" }
Local log file
This component uses the following log file.
To view this component's logs
-
Run the following command on the core device to view this component's log file in real time. Replace
/greengrass/v2
orC:\greengrass\v2
with the path to the Amazon IoT Greengrass root folder.
Changelog
The following table describes the changes in each version of the component.
Version |
Changes |
---|---|
2.1.3 |
|
2.1.2 |
|
2.1.1 |
Version updated for Greengrass nucleus version 2.8.0 release. |
2.1.0 |
|
2.0.15 |
Version updated for Greengrass nucleus version 2.6.0 release. |
2.0.14 |
This version contains bug fixes and improvements. |
2.0.13 |
Version updated for Greengrass nucleus version 2.5.0 release. |
2.0.12 |
|
2.0.11 |
Version updated for Greengrass nucleus version 2.4.0 release. |
2.0.10 |
Version updated for Greengrass nucleus version 2.3.0 release. |
2.0.9 |
Version updated for Greengrass nucleus version 2.2.0 release. |
2.0.8 |
Version updated for Greengrass nucleus version 2.1.0 release. |
2.0.7 |
Initial version. |