dedup

Use dedup to remove duplicate results based on specific values in fields that you specify. You can use dedup with one or more fields. If you specify one field with dedup, only one log event is returned for each unique value of that field. If you specify multiple fields, then one log event is returned for each unique combination of values for those fields.

Duplicates are discarded based on the sort order, with only the first result in the sort order being kept. We recommend that you sort your results before putting them through the dedup command. If the results are not sorted before being run through dedup, then the default descending sort order using @timestamp is used.

Null values are not considered duplicates for evaluation. Log events with null values for any of the specified fields are retained. To eliminate fields with null values, use filter using the isPresent(field) function.

The only query command that you can use in a query after the dedup command is limit.

Example: See only the most recent log event for each unique value of the field named server

The following example displays the timestamp, server, severity, and message fields for only the most recent event for each unique value of server.

fields @timestamp, server, severity, message 
| sort @timestamp desc 
| dedup server

For more samples of CloudWatch Logs Insights queries, see General queries.