# Create and manage log transformers
A log transformer includes one or more *processors* that are in a logical pipeline together. Each processor is applied to a log event, one after the other in the order that they are listed in the transformer configuration.
Some processors are of the *parser* type. Each transformer must have at least one parser, and the first processor in a transformer must be a parser.
Some of the parsers are built-in parsers that are configured for a certain type of Amazon vended log.
Other processor types are string mutators, JSON mutators, and data processors.
You can create transformers for individual log groups, and you can also create account-level transformers that apply to all or many log groups in your account. If a log group has a log group-level transformer, that transformer overrides any account-level transformer that would otherwise apply to that log group. You can have as many as 20 account-level transformers in a Region in your account.
You must follow these guidelines when you create a transformer:
+ If you include a pre-configured parser for a type of Amazon vended logs, it must be the first processor listed in the transformer. You can include only one such processor in a transformer.
+ You can include only one `grok` processor in a transformer.
+ You must have at least one parser-type processor in a transformer. You can include as many as five parser-type processors. This limit of five includes both built-in parsers and configurable parsers.
+ You can have as many as 20 processors in a transformer.
+ You can include only one **addKeys** processor in a transformer.
+ You can include only one **copyValue** processor in a transformer.
+ Each transformer can extract up to 200 fields from a log event.
+ Each log event **MUST** be below 512KB. Total size of log events can still go over 512KB.
**Topics**
+ [Create an account-level transformer policy](CloudWatchLogs-Transformer-CreateAccountLevel.md)
+ [Edit or delete an account-level transformer policy](CloudWatchLogs-Transformer-EditAccountLevel.md)
+ [Create a log-group-level log transformer from scratch](CloudWatch-Logs-Transformation-CreateNew.md)
+ [Create a log-group-level transformer by copying an existing one](CloudWatch-Logs-Transformation-Copy.md)
+ [Edit a log-group-level transformer](CloudWatch-Logs-Transformation-Edit.md)
+ [Delete a log-group-level transformer](CloudWatch-Logs-Transformation-Delete.md)
# Create an account-level transformer policy
Use the steps in this section to create a transformer policy that applies to all log groups in the account, or to multiple log groups that have log group names that start with the same string (prefix). You can have as many as 20 account-level transformer policies in a Region.
You can't create two transformer policies in the same Region that use the same prefix or have one prefix contained within another. For example, if you create one transformer policy for the string prefix `/aws/lambda`, you can't create another with the prefix `/aws`. But you could have one transformer for `/aws/lambda` and another for `/aws/waf`
**To create an account-level transformer policy**
1. Open the CloudWatch console at [https://console.amazonaws.cn/cloudwatch/](https://console.amazonaws.cn/cloudwatch/).
1. In the left navigation pane, choose **Settings** and then choose the **Logs** tab.
1. In the **Transformer policy for account** section, choose **Create transformer policy**.
1. For **Transformer policy name**, enter a name for your new poiicy.
1. For **Select log groups**, do one of the following:
+ Choose **All standard log groups** to have the transformer policy apply to all Standard Class log groups in the account.
+ choose **Log groups by prefix match** to apply the policy to a subset of log groups that all have names that start with the same string. Then, enter the prefix for these log groups in **Selection criteria**.
1. In the **Select parsers** area, use **Parsers** to select a parser to include in your transformer.
If it is a pre-configured parser for a type of Amazon vended log, you don't have to specify any configuration for it.
If it is a different parser, you need to specify its configuration. For more information, see the information for that processor in [Configurable parser-type processors](CloudWatch-Logs-Transformation-Configurable.md).
1. To add another processor, choose **Select processor**. Then select the processor that you want in the **Processor** box, and fill in the configuration parameters.
Remember that processors operate on the log events in the order that you add them to the transformer.
1. (Optional) To add additional processors, choose **\$1 Processor** and repeat the previous step.
1. (Optional) At any time, you can test the transformer that you have built so far on a sample log event. To do so, do one of the following in the **Transformer preview** section:
+ Select as many as five log groups in **Select log groups** and then choose **Load latest log events**. Then choose **Test transformer**.
+ Copy log events directly into **Sample log events** and then choose **Test transformer**.
The transformed version of the log then appears.
1. When you are finished adding processors and satisfied with the tests on sample logs, choose **Save**.
1. When you have finished, choose **Create**.
# Edit or delete an account-level transformer policy
Use the steps in this section to edit or delete an account-level transformer policy.
**To edit or delete an account-level transformer policy**
1. Open the CloudWatch console at [https://console.amazonaws.cn/cloudwatch/](https://console.amazonaws.cn/cloudwatch/).
1. In the left navigation pane, choose **Settings** and then choose the **Logs** tab.
1. In the **Transformer account policy** section, choose **Manage**.
1. Select the button by the transformer policy that you want to manage, and then choose **Edit** or **Delete**.
If you're editing the policy, see steps 5-11 in [Configurable parser-type processors](CloudWatch-Logs-Transformation-Configurable.md) to see your options.
# Create a log-group-level log transformer from scratch
Use these steps to create a log-group-level transformer from scratch.
**To use the console to create a log transformer for a log group**
1. Open the CloudWatch console at [https://console.amazonaws.cn/cloudwatch/](https://console.amazonaws.cn/cloudwatch/).
1. In the navigation pane, choose **Logs**, **Log groups**.
1. Choose the log group that you want to create the transformer for.
1. Choose the **Transformer** tab. You might have to scroll the tab list to the right to see it.
1. Choose **Create transformer**.
1. In the **Choose a parser** box, select a parser to include in your transformer.
If it is a pre-configured parser for a type of Amazon vended log, you don't have to specify any configuration for it.
If it is a different parser, you need to specify its configuration. For more information, see the information for that processor in [Configurable parser-type processors](CloudWatch-Logs-Transformation-Configurable.md).
1. To add another processor, choose **\$1 Add processor**. Then select the processor that you want in the **Choose processors** box, and fill in the configuration parameters.
Remember that processors operate on the log events in the order that you add them to the transformer.
1. (Optional) At any time, you can test the transformer that you have built so far on a sample log event. To do so, do the following:
1. In the **Transformation preview** section, either choose **Load sample log** to load a sample log event from the log group that this transformer is for, or paste a log event into the text box.
Choose **Test transformer**. The transformed version of the log appears
1. When you are finished adding processors and satisfied with the tests on sample logs, choose **Save**.
**To use the Amazon CLI to create a log transformer from scratch**
+ Use the `aws logs put-transformer` command. When using `parseJSON` as the first processor, you must parse the entire log event using `@message` as the source field. After the initial JSON parsing, you can then manipulate specific fields in subsequent processors. The following is an example that creates a transformer that includes the `parseJSON` and `addKeys` processors:
```
aws logs put-transformer \
--transformer-config '[{"parseJSON":{"source":"@message"}},{"addKeys":{"entries":[{"key":"metadata.transformed_in","value":"CloudWatchLogs"},{"key":"feature","value":"Transformation"}]}},{"trimString":{"withKeys":["status"]}}]' \
--log-group-identifier my-log-group-name
```
# Create a log-group-level transformer by copying an existing one
You can use the console to copy the JSON configuration of an existing transformer. You can then use that code to create an identical transformer by using the Amazon CLI, or you can modify the configuration first.
**To create a log transformer by copying an existing one**
1. Open the CloudWatch console at [https://console.amazonaws.cn/cloudwatch/](https://console.amazonaws.cn/cloudwatch/).
1. In the navigation pane, choose **Logs**, **Log groups**.
1. Choose the log group that has the transformer that you want to copy.
1. Choose the **Transformations** tab. You might have to scroll the tab list to the right to see it.
1. Choose **Manage transformer**.
1. Choose **Copy transformer**. This copies the transformer JSON to your clipboard.
1. Create a file and paste in the transformer configuration. In this example we'll call the file `CopiedTransformer.json`
1. Use the Amazon CLI to create a new transformer with that configuration.
```
aws logs put-transformer --log-group-identifier my-log-group-name \
--transformer-config file://CopiedTransformer.json
```
# Edit a log-group-level transformer
Use these steps to edit an existing log transformer.
**To edit a log transformer**
1. Open the CloudWatch console at [https://console.amazonaws.cn/cloudwatch/](https://console.amazonaws.cn/cloudwatch/).
1. In the navigation pane, choose **Logs**, **Log groups**.
1. Choose the log group that has the transformer that you want to edit.
1. Choose the **Transformations** tab. You might have to scroll the tab list to the right to see it.
1. Choose **Manage transformer**.
1. In the **Parsers** and **Processors** sections, make your changes.
1. To add another processor, choose **\$1 Add Processor**. Then select the processor that you want in the **Processor** box, and fill in the configuration parameters.
Remember that processors operate on the log events in the order that you add them to the transformer.
1. (Optional) At any time, you can test the transformer that you have built so far on a sample log event. To do so, do the following:
1. In the **Transformation Preview** section, either choose **Load Sample Log** to load a sample log event from the log group that this transformer is for, or paste a log event into the text box.
Choose **Test Transformation**. The transformed version of the log appears
1. When you are finished adding processors and satisfied with the tests on sample logs, choose **Save**.
# Delete a log-group-level transformer
Use these steps to delete a log transformer.
**To delete a log transformer**
1. Open the CloudWatch console at [https://console.amazonaws.cn/cloudwatch/](https://console.amazonaws.cn/cloudwatch/).
1. In the navigation pane, choose **Logs**, **Log groups**.
1. Choose the log group that has the transformer that you want to edit.
1. Choose the **Transformations** tab. You might have to scroll the tab list to the right to see it.
1. Choose **Delete**.
1. In the confirmation box, choose **Delete Policy**.