CloudWatch Logs Insights query syntax

With CloudWatch Logs Insights, you use a query language to query your log groups. The query syntax supports different functions and operations that include but aren't limited to general functions, arithmetic and comparison operations, and regular expressions. Create queries that contain multiple commands. Separate commands with the pipe character (|). Create queries that contain comments. Set off comments with the hash character (#).

CloudWatch Logs Insights query commands

This section includes a list of the supported CloudWatch Logs Insights query commands.

Guidelines for working with query commands

You must surround log fields named in queries that include characters other than the @ symbol, period (.), and non-alphanumeric characters in backtick keys (`). For example, the log field foo-bar must be enclosed in backticks (`foo-bar`) because it contains a non-alphanumeric character, the hyphen (-).

Use the display command to show the field or fields that you want to see in your query results. The display command only shows the fields you specify. If your query contains multiple display commands, the query results show only the field or fields that you specified in the final display command.

You can use fields command with the keyword as to create ephemeral fields that use fields and functions in your log events. For example, fields ispresent as isRes creates an ephemeral field named isRes, and the ephemeral field can be used in the rest of your query.

The value of isRes equals 0 or 1, depending on whether resolverArn is a discovered field . If your query contains multiple fields commands and doesn't include a display command, you'll display all of the fields that are specified in the fields commands.

Matches and regular expressions in the filter command

The filter command supports the use of regular expressions. You can use the following comparison operators (=, !=, <, <=, >, >=) and Boolean operators (and, or, and not).

You can use the keyword in to test for set membership and check for elements in an array. To check for elements in an array, put the array after in. You can use the Boolean operator not with in. You can create queries that use in to return log events where fields are string matches. The fields must be complete strings. For example, the following code snippet shows a query that uses in to return log events where the field logGroup is the complete string example_group.

fields @timestamp, @message
| filter logGroup in ["example_group"]

You can use the keyword phrases like and not like to match substrings. You can use the regular expression operator =~ to match substrings. To match a substring with like and not like, enclose the substring that you want to match in single or double quotation marks. You can use regular expression patterns with like and not like. To match a substring with the regular expression operator, enclose the substring that you want to match in forward slashes. The following examples contain code snippets that show how you can match substrings using the filter command.

Examples: Match substrings

The following examples return log events where f1 contains the word Exception. All three examples are case sensitive.

The first example matches a substring with like.

fields f1, f2, f3 
| filter f1 like "Exception"

The second example matches a substring with like and a regular expression pattern.

fields f1, f2, f3 
| filter f1 like /Exception/

The third example matches a substring with a regular expression.

fields f1, f2, f3 
| filter f1 =~ /Exception/

Example: Match substrings with wildcards

You can use the period symbol (.) as a wildcard in regular expressions to match substrings. In the following example, the query returns matches where the value for f1 begins with the string ServiceLog.

fields f1, f2, f3
| filter f1 like /ServiceLog./

You can place the asterisk symbol after the period symbol (.*) to create a greedy quantifier that returns as many matches as possible. For example, the following query returns matches where the value for f1 not only begins with the string ServiceLog, but also includes the string ServiceLog.

fields f1, f2, f3
| filter f1 like /ServiceLog.*/

Possible matches can be formatted like the following:

Example: Exclude substrings from matches

The following example shows a query that returns log events where f1 doesn't contain the word Exception. The example is case senstive.

fields f1, f2, f3 
| filter f1 not like "Exception"

Example: Match substrings with case-insensitive patterns

You can match substrings that are case insensitive with like and regular expressions. Place the following parameter (?i) before the substring you want to match. The following example shows a query that returns log events where f1 contains the word Exception or exception.

fields f1, f2, f3 
| filter f1 like /(?i)Exception/ 

Using aliases in queries

Create queries that contain aliases. Use aliases to rename log fields or when extracting values into ephemeral fields. Use the keyword as to give a log field or result an alias. You can use more than one alias in a query. You can use aliases in the following commands:

The following examples show how to create queries that contain aliases.

Example

The query contains an alias in the fields command.

fields @timestamp, @message, accountId as ID
| sort @timestamp desc
| limit 20

The query returns the values for the fields @timestamp, @message, and accountId. The results are sorted in descending order and limited to 20. The values for accountId are listed under the alias ID.

Example

The query contains aliases in the sort and stats commands.

stats count(*) by duration as time 
| sort time desc

The query counts the number of times the field duration occurs in the log group and sorts the results in descending order. The values for duration are listed under the alias time.

Using comments in queries

CloudWatch Logs Insights supports comments in queries. Use the hash character (#) to set off comments. You can use comments to ignore lines in queries or document queries.

Example: Query

When the following query is run, the second line is ignored.

fields @timestamp, @message, accountId
# | filter accountId not like "7983124201998"
| sort @timestamp desc
| limit 20

Supported operations and functions

CloudWatch Logs Insights supports the following operations and functions.

Arithmetic operators

Arithmetic operators accept numeric data types as arguments and return numeric results. Use arithmetic operators in the filter and fields commands and as arguments for other functions.

Boolean operators

Use the Boolean operators and, or, and not.

Comparison operators

Comparison operators accept all data types as arguments and return a Boolean result. Use comparison operations in the filter command and as arguments for other functions.

Numeric operations

Numeric operations accept numeric data types as arguments and return numeric results. Use numeric operations in the filter and fields commands and as arguments for other functions.

Datetime functions

Use datetime functions in the fields and filtercommands and as arguments for other functions. Use these functions to create time buckets for queries with aggregate functions. Use time periods that consist of a number and m for minutes or h for hours. For example, 10m is 10 minutes, and 1h is 1 hour. The following table contains a list of the different datetime functions that you can use in query commands. The table lists each function's result type and contains a description of each function.

filter @message like /Exception/ 
    | stats count(*) as exceptionCount by bin(1h)
    | sort exceptionCount desc

General functions

Use general functions in the fields and filter commands and as arguments for other functions.

IP address string functions

Use IP address string functions in the filter and fields commands and as arguments for other functions.

Stats aggregation functions

Use aggregation functions in the stats command and as arguments for other functions.

Stats non-aggregation functions

Use non-aggregation functions in the stats command and as arguments for other functions.

String functions

Use string functions in the fields and filter commands and as arguments for other functions.