Filtering Fluentd and Fluent Bit logs

There might be times when you want to filter what logs get sent, for example send logs that contain a specific error.

Fluentd and Fluent Bit both support filtering of logs based on their content. FireLens provides a simple method for enabling this filtering.

You can configure these in the logConfiguration options in a container definition by specifying the regular expressions to match.

The exclude-pattern key causes all logs that match its regular expression to be dropped. The include-pattern only sends logs that match the specified regular expression. You can use these keys separately or together

The following example demonstrates how to use this filter.

{
   "containerDefinitions":[
      {
         "logConfiguration":{
            "logDriver":"awsfirelens",
            "options":{
               "@type":"cloudwatch_logs",
               "log_group_name":"firelens-testing",
               "auto_create_stream":"true",
               "use_tag_as_stream":"true",
               "region":"us-west-2",
               "exclude-pattern":"^[a-z][aeiou].*$",
               "include-pattern":"^.*[aeiou]$"
            }
         }
      }
   ]
}