Add substring expression functions#6621
Merged
oeyh merged 1 commit intoMar 17, 2026
Merged
Conversation
bagmarnikhil
requested review from
KarstenSchnitter,
dinujoh,
dlvenable,
engechas,
graytaylor0,
kkondaka,
oeyh,
san81,
sb2k16 and
srikanthjg
as code owners
Member
|
@bagmarnikhil , Please rebase this from the latest |
dlvenable
reviewed
Mar 10, 2026
| throw new RuntimeException("substringAfter() takes exactly two arguments"); | ||
| } | ||
|
|
||
| final String[] strArgs = new String[NUMBER_OF_ARGS]; |
Member
There was a problem hiding this comment.
We should have an abstract class for substring functions as they all share the same basic behavior.
abstract AbstractSubstringExpressionFunction implements ExpressionFunction
You can use getFunctionName in the user messages.
Contributor
Author
There was a problem hiding this comment.
Updated the PR based on the suggested refactor
bagmarnikhil
force-pushed
the
feature/substring-expression-functions
branch
from
48e3bfb to
0d6c6b3
Compare
The expression language has no way to extract a portion of a string by delimiter. Existing string processors mutate fields in-place but cannot produce a value for assignment via value_expression. Add four new expression functions: - substringAfter(s, d): text after the first occurrence of d - substringBefore(s, d): text before the first occurrence of d - substringAfterLast(s, d): text after the last occurrence of d - substringBeforeLast(s, d): text before the last occurrence of d Both arguments accept JSON Pointers or string literals. If the delimiter is not found, the original string is returned. If the source resolves to null, null is returned. Resolve opensearch-project#6612 Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
bagmarnikhil
force-pushed
the
feature/substring-expression-functions
branch
from
0d6c6b3 to
e545d87
Compare
graytaylor0
reviewed
Mar 13, 2026
| - If the IP address is in the range of any given CIDR blocks, the function evaluates to true; otherwise, the function evaluates to false. | ||
| - The function supports both IPv4 and IPv6 addresses. | ||
| For example, `cidrContains(/sourceIp,"192.0.2.0/24","10.0.1.0/16")` evaluates to true if the event has `sourceIp` field with value "192.0.2.5". | ||
| * `substringAfter()` |
Member
There was a problem hiding this comment.
Please create a follow up PR to the documentation website here (https://github.com/opensearch-project/documentation-website).
This documentation is what users will use (https://docs.opensearch.org/latest/data-prepper/pipelines/functions)
Contributor
Author
There was a problem hiding this comment.
Created a pull request for documentation: opensearch-project/documentation-website#12094
graytaylor0
approved these changes
Mar 13, 2026
bagmarnikhil
added a commit
to bagmarnikhil/documentation-website
that referenced
this pull request
Mar 13, 2026
Add documentation for four new Data Prepper expression functions: substringAfter, substringBefore, substringAfterLast, and substringBeforeLast. These functions extract portions of a string by delimiter and were added in opensearch-project/data-prepper#6621. Update the functions index page to include the new functions. Resolves: opensearch-project/data-prepper#6612 Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
Merged
1 task
kolchfa-aws
added a commit
to opensearch-project/documentation-website
that referenced
this pull request
Mar 17, 2026
* Add substring expression function documentation Add documentation for four new Data Prepper expression functions: substringAfter, substringBefore, substringAfterLast, and substringBeforeLast. These functions extract portions of a string by delimiter and were added in opensearch-project/data-prepper#6621. Update the functions index page to include the new functions. Resolves: opensearch-project/data-prepper#6612 Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
opensearch-trigger-bot Bot
pushed a commit
to opensearch-project/documentation-website
that referenced
this pull request
Mar 17, 2026
* Add substring expression function documentation Add documentation for four new Data Prepper expression functions: substringAfter, substringBefore, substringAfterLast, and substringBeforeLast. These functions extract portions of a string by delimiter and were added in opensearch-project/data-prepper#6621. Update the functions index page to include the new functions. Resolves: opensearch-project/data-prepper#6612 Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> (cherry picked from commit 08ff56d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
oeyh
approved these changes
Mar 17, 2026
aryasoni98
pushed a commit
to aryasoni98/documentation-website
that referenced
this pull request
Jun 29, 2026
…2094) * Add substring expression function documentation Add documentation for four new Data Prepper expression functions: substringAfter, substringBefore, substringAfterLast, and substringBeforeLast. These functions extract portions of a string by delimiter and were added in opensearch-project/data-prepper#6621. Update the functions index page to include the new functions. Resolves: opensearch-project/data-prepper#6612 Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: Arya Soni <aryasoni98@gmail.com>
kkondaka
pushed a commit
to kkondaka/kk-data-prepper-f2
that referenced
this pull request
Jul 1, 2026
The expression language has no way to extract a portion of a string by delimiter. Existing string processors mutate fields in-place but cannot produce a value for assignment via value_expression. Add four new expression functions: - substringAfter(s, d): text after the first occurrence of d - substringBefore(s, d): text before the first occurrence of d - substringAfterLast(s, d): text after the last occurrence of d - substringBeforeLast(s, d): text before the last occurrence of d Both arguments accept JSON Pointers or string literals. If the delimiter is not found, the original string is returned. If the source resolves to null, null is returned. Resolve opensearch-project#6612 Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The expression language has no way to extract a portion of a string by delimiter. Existing string processors mutate fields in-place but cannot produce a value for assignment via value_expression.
Add four new expression functions:
Both arguments accept JSON Pointers or string literals. If the delimiter is not found, the original string is returned. If the source resolves to null, null is returned.
Issues Resolved
Resolves #6612
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.