From e239a95c8ab643afcbc53305113c091777a75032 Mon Sep 17 00:00:00 2001 From: Vincent Boutour Date: Thu, 6 Nov 2025 15:05:24 +0100 Subject: [PATCH 1/2] docs(aws): Update documentation related to include at match Signed-off-by: Vincent Boutour --- aws/logs_monitoring/README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/aws/logs_monitoring/README.md b/aws/logs_monitoring/README.md index ac0fd3332..653ed37ec 100644 --- a/aws/logs_monitoring/README.md +++ b/aws/logs_monitoring/README.md @@ -401,15 +401,14 @@ Datadog recommends using at least 10 reserved concurrency, but this defaults to `IncludeAtMatch` : Only send logs matching the supplied regular expression, and not excluded by `ExcludeAtMatch`. -Filtering rules are applied to the full JSON-formatted log, including any metadata that is automatically added by the Forwarder. However, transformations applied by [log pipelines][21], which occur after logs are sent to Datadog, cannot be used to filter logs in the Forwarder. Using an inefficient regular expression, such as `.*`, may slow down the Forwarder. +Filtering rules are applied to the message log. However, transformations applied by [log pipelines][21], which occur after logs are sent to Datadog, cannot be used to filter logs in the Forwarder. Using an inefficient regular expression, such as `.*`, may slow down the Forwarder. Some examples of regular expressions that can be used for log filtering: -- Include (or exclude) Lambda platform logs: `"(START|END) RequestId:\s`. The preceding `"` is needed to match the start of the log message, which is in a JSON blob (`{"message": "START RequestId...."}`). Datadog recommends keeping the `REPORT` logs, as they are used to populate the invocations list in the serverless function views. +- Include (or exclude) Lambda platform logs: `(START|END) RequestId:\s`. Datadog recommends keeping the `REPORT` logs, as they are used to populate the invocations list in the serverless function views. - Include CloudTrail error messages only: `errorMessage`. - Include only logs containing an HTTP 4XX or 5XX error code: `\b[4|5][0-9][0-9]\b`. -- Include only CloudWatch logs where the `message` field contains a specific JSON key/value pair: `\"awsRegion\":\"us-east-1\"`. - - The message field of a CloudWatch log event is encoded as a string. For example,`{"awsRegion": "us-east-1"}` is encoded as `{\"awsRegion\":\"us-east-1\"}`. Therefore, the pattern you provide must include `\` escape characters, like this: `\"awsRegion\":\"us-east-1\"`. +- Include only CloudWatch logs where the `message` field contains a specific JSON key/value pair: `"awsRegion":"us-east-1"`. To test different patterns against your logs, turn on [debug logs](#troubleshooting). From 2ec714570dd908ef034e373bffb30847b01aeb26 Mon Sep 17 00:00:00 2001 From: Vincent Boutour Date: Thu, 6 Nov 2025 16:23:09 +0100 Subject: [PATCH 2/2] Update aws/logs_monitoring/README.md Co-authored-by: Georgi --- aws/logs_monitoring/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aws/logs_monitoring/README.md b/aws/logs_monitoring/README.md index 653ed37ec..f1667afcd 100644 --- a/aws/logs_monitoring/README.md +++ b/aws/logs_monitoring/README.md @@ -401,7 +401,7 @@ Datadog recommends using at least 10 reserved concurrency, but this defaults to `IncludeAtMatch` : Only send logs matching the supplied regular expression, and not excluded by `ExcludeAtMatch`. -Filtering rules are applied to the message log. However, transformations applied by [log pipelines][21], which occur after logs are sent to Datadog, cannot be used to filter logs in the Forwarder. Using an inefficient regular expression, such as `.*`, may slow down the Forwarder. +Filtering rules are applied to the log message. However, transformations applied by [log pipelines][21], which occur after logs are sent to Datadog, cannot be used to filter logs in the Forwarder. Using an inefficient regular expression, such as `.*`, may slow down the Forwarder. Some examples of regular expressions that can be used for log filtering: