Add substring expression function documentation

_data-prepper/pipelines/functions.md

@@ -17,6 +17,9 @@ OpenSearch Data Prepper offers a range of built-in functions that can be used wi
 - [`hasTags()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/has-tags/)
 - [`join()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/join/)
 - [`length()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/length/)
-- [`subList()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/sublist/)
 - [`startsWith()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/startswith/)
-
+- [`subList()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/sublist/)
+- [`substringAfter()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/substring-after/)
+- [`substringAfterLast()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/substring-after-last/)
+- [`substringBefore()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/substring-before/)
+- [`substringBeforeLast()`]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/substring-before-last/)

_data-prepper/pipelines/sublist.md

@@ -3,7 +3,7 @@ layout: default
 title: subList()
 parent: Functions
 grand_parent: Pipelines
-nav_order: 35
+nav_order: 50
 ---
 
 # subList(<key>, <start_index, inclusive>, <end_index, exclusive>)

_data-prepper/pipelines/substring-after-last.md

@@ -0,0 +1,112 @@
+---
+layout: default
+title: substringAfterLast()
+parent: Functions
+grand_parent: Pipelines
+nav_order: 70
+---
+
+# substringAfterLast()
+
+The `substringAfterLast()` function is used to extract the portion of a string that follows the last occurrence of a specified delimiter. It takes two arguments:
+
+1. The first argument is either a literal string or a JSON pointer that represents the source string.
+
+1. The second argument is the delimiter string to search for within the first argument.
+
+If the delimiter is found, the function returns the portion of the string after the last occurrence of the delimiter. If the delimiter is not found, the original string is returned. If the source resolves to `null`, the function returns `null`. If the delimiter is `null` or empty, the original string is returned.
+
+For example, to extract the file extension from the `/filepath` field containing a file path, use the `substringAfterLast()` function as follows:
+
+```
+'substringAfterLast(/filepath, ".")'
+```
+{% include copy.html %}
+
+If the `/filepath` field contains `archive.tar.gz`, the function returns `gz`.
+
+Alternatively, you can use a literal string as the first argument:
+
+```
+'substringAfterLast("one-two-three", "-")'
+```
+{% include copy.html %}
+
+The function returns `three` because it extracts the portion of the string after the last `-` character.
+
+The `substringAfterLast()` function performs a case-sensitive search.
+{: .note}
+
+## Example
+
+The following pipeline uses the `substringAfterLast()` function to extract the file name from a full file path. It adds the extracted file name as a new field called `filename`:
+
+```yaml
+substring-after-last-demo:
+  source:
+    http:
+      ssl: false
+
+  processor:
+    - add_entries:
+        entries:
+          - key: filename
+            value_expression: 'substringAfterLast(/filepath, "/")'
+
+  sink:
+    - opensearch:
+        hosts: ["https://opensearch:9200"]
+        insecure: true
+        username: admin
+        password: admin_password
+        index_type: custom
+        index: demo-index-%{yyyy.MM.dd}
+```
+{% include copy.html %}
+
+You can test the pipeline using the following command:
+
+```bash
+curl -sS -X POST "http://localhost:2021/log/ingest" \
+  -H "Content-Type: application/json" \
+  -d '[
+        {"filepath":"/var/log/syslog"},
+        {"filepath":"/home/user/docs/report.pdf"}
+      ]'
+```
+{% include copy.html %}
+
+The documents stored in OpenSearch contain the following information:
+
+```json
+{
+  ...
+  "hits": {
+    "total": {
+      "value": 2,
+      "relation": "eq"
+    },
+    "max_score": 1,
+    "hits": [
+      {
+        "_index": "demo-index-2026.03.13",
+        "_id": "abc123",
+        "_score": 1,
+        "_source": {
+          "filepath": "/var/log/syslog",
+          "filename": "syslog"
+        }
+      },
+      {
+        "_index": "demo-index-2026.03.13",
+        "_id": "def456",
+        "_score": 1,
+        "_source": {
+          "filepath": "/home/user/docs/report.pdf",
+          "filename": "report.pdf"
+        }
+      }
+    ]
+  }
+}
+```

_data-prepper/pipelines/substring-after.md

@@ -0,0 +1,112 @@
+---
+layout: default
+title: substringAfter()
+parent: Functions
+grand_parent: Pipelines
+nav_order: 60
+---
+
+# substringAfter()
+
+The `substringAfter()` function is used to extract the portion of a string that follows the first occurrence of a specified delimiter. It takes two arguments:
+
+1. The first argument is either a literal string or a JSON pointer that represents the source string.
+
+1. The second argument is the delimiter string to search for within the first argument.
+
+If the delimiter is found, the function returns the portion of the string after the first occurrence of the delimiter. If the delimiter is not found, the original string is returned. If the source resolves to `null`, the function returns `null`. If the delimiter is `null` or empty, the original string is returned.
+
+For example, to extract the value after the first occurrence of the `=` character in a field named `header`, use the `substringAfter()` function as follows:
+
+```
+'substringAfter(/header, "=")'
+```
+{% include copy.html %}
+
+If `/header` contains `Content-Type=application/json`, the function returns `application/json`.
+
+Alternatively, you can use a literal string as the first argument:
+
+```
+'substringAfter("hello-world-foo", "-")'
+```
+{% include copy.html %}
+
+The function returns `world-foo` because it extracts the portion of the string after the first `-` character.
+
+The `substringAfter()` function performs a case-sensitive search.
+{: .note}
+
+## Example
+
+The following pipeline uses the `substringAfter()` function to extract the domain name from an email address field. It adds the extracted domain name as a new field called `domain`:
+
+```yaml
+substring-after-demo:
+  source:
+    http:
+      ssl: false
+
+  processor:
+    - add_entries:
+        entries:
+          - key: domain
+            value_expression: 'substringAfter(/email, "@")'
+
+  sink:
+    - opensearch:
+        hosts: ["https://opensearch:9200"]
+        insecure: true
+        username: admin
+        password: admin_password
+        index_type: custom
+        index: demo-index-%{yyyy.MM.dd}
+```
+{% include copy.html %}
+
+You can test the pipeline using the following command:
+
+```bash
+curl -sS -X POST "http://localhost:2021/log/ingest" \
+  -H "Content-Type: application/json" \
+  -d '[
+        {"email":"user@example.com"},
+        {"email":"admin@opensearch.org"}
+      ]'
+```
+{% include copy.html %}
+
+The documents stored in OpenSearch contain the following information:
+
+```json
+{
+  ...
+  "hits": {
+    "total": {
+      "value": 2,
+      "relation": "eq"
+    },
+    "max_score": 1,
+    "hits": [
+      {
+        "_index": "demo-index-2026.03.13",
+        "_id": "abc123",
+        "_score": 1,
+        "_source": {
+          "email": "user@example.com",
+          "domain": "example.com"
+        }
+      },
+      {
+        "_index": "demo-index-2026.03.13",
+        "_id": "def456",
+        "_score": 1,
+        "_source": {
+          "email": "admin@opensearch.org",
+          "domain": "opensearch.org"
+        }
+      }
+    ]
+  }
+}
+```

_data-prepper/pipelines/substring-before-last.md

@@ -0,0 +1,112 @@
+---
+layout: default
+title: substringBeforeLast()
+parent: Functions
+grand_parent: Pipelines
+nav_order: 90
+---
+
+# substringBeforeLast()
+
+The `substringBeforeLast()` function is used to extract the portion of a string that precedes the last occurrence of a specified delimiter. It takes two arguments:
+
+1. The first argument is either a literal string or a JSON pointer that represents the source string.
+
+1. The second argument is the delimiter string to search for within the first argument.
+
+If the delimiter is found, the function returns the portion of the string before the last occurrence of the delimiter. If the delimiter is not found, the original string is returned. If the source resolves to `null`, the function returns `null`. If the delimiter is `null` or empty, the original string is returned.
+
+For example, to remove the file extension from a filename field, use the `substringBeforeLast()` function as follows:
+
+```
+'substringBeforeLast(/filename, ".")'
+```
+{% include copy.html %}
+
+If the `/filename` field contains `archive.tar.gz`, the function returns `archive.tar`.
+
+Alternatively, you can use a literal string as the first argument:
+
+```
+'substringBeforeLast("one-two-three", "-")'
+```
+{% include copy.html %}
+
+The function returns `one-two` because it extracts the portion of the string before the last `-` character.
+
+The `substringBeforeLast()` function performs a case-sensitive search.
+{: .note}
+
+## Example
+
+The following pipeline uses the `substringBeforeLast()` function to extract the directory path from a full file path. It adds the extracted directory path as a new field called `directory`:
+
+```yaml
+substring-before-last-demo:
+  source:
+    http:
+      ssl: false
+
+  processor:
+    - add_entries:
+        entries:
+          - key: directory
+            value_expression: 'substringBeforeLast(/filepath, "/")'
+
+  sink:
+    - opensearch:
+        hosts: ["https://opensearch:9200"]
+        insecure: true
+        username: admin
+        password: admin_password
+        index_type: custom
+        index: demo-index-%{yyyy.MM.dd}
+```
+{% include copy.html %}
+
+You can test the pipeline using the following command:
+
+```bash
+curl -sS -X POST "http://localhost:2021/log/ingest" \
+  -H "Content-Type: application/json" \
+  -d '[
+        {"filepath":"/var/log/syslog"},
+        {"filepath":"/home/user/docs/report.pdf"}
+      ]'
+```
+{% include copy.html %}
+
+The documents stored in OpenSearch contain the following information:
+
+```json
+{
+  ...
+  "hits": {
+    "total": {
+      "value": 2,
+      "relation": "eq"
+    },
+    "max_score": 1,
+    "hits": [
+      {
+        "_index": "demo-index-2026.03.13",
+        "_id": "abc123",
+        "_score": 1,
+        "_source": {
+          "filepath": "/var/log/syslog",
+          "directory": "/var/log"
+        }
+      },
+      {
+        "_index": "demo-index-2026.03.13",
+        "_id": "def456",
+        "_score": 1,
+        "_source": {
+          "filepath": "/home/user/docs/report.pdf",
+          "directory": "/home/user/docs"
+        }
+      }
+    ]
+  }
+}
+```