Skip to content

Iterate on support for add_entries and delete_entries processors#5773

Merged
graytaylor0 merged 5 commits into
opensearch-project:mainfrom
graytaylor0:IterateOnSupport
Jun 13, 2025
Merged

Iterate on support for add_entries and delete_entries processors#5773
graytaylor0 merged 5 commits into
opensearch-project:mainfrom
graytaylor0:IterateOnSupport

Conversation

@graytaylor0

Copy link
Copy Markdown
Member

Description

Completes this unfinished PR (#5620) by rebasing and fixing merge conflicts

Also manually tested

- add_entries:
      entries:
        - iterate_on: "list-key"
          key: "new-key"
          value_expression: '/add_value'
  - delete_entries:
      with_keys: [ "delete-key" ]
      iterate_on: "list-key"

Issues Resolved

Resolves #2853

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

chenqi0805 and others added 4 commits June 11, 2025 16:42
Signed-off-by: George Chen <qchea@amazon.com>
Signed-off-by: George Chen <qchea@amazon.com>
Signed-off-by: George Chen <qchea@amazon.com>
Signed-off-by: Taylor Gray <tylgry@amazon.com>
san81
san81 previously approved these changes Jun 11, 2025

@san81 san81 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice enhancement. We should see if this can be more generalizable to be able too apply to other processors as well.

…add options for delete_from_element_when and add_to_element_when

Signed-off-by: Taylor Gray <tylgry@amazon.com>
@graytaylor0 graytaylor0 changed the title Iterate on support Iterate on support for add_entries and delete_entries processors Jun 12, 2025

@san81 san81 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice separation of logic to make it clear and maintainable

@graytaylor0 graytaylor0 merged commit b26cb82 into opensearch-project:main Jun 13, 2025
46 of 47 checks passed
jeffreyAaron pushed a commit to jeffreyAaron/data-prepper that referenced this pull request Jun 13, 2025
…nsearch-project#5773)

Signed-off-by: George Chen <qchea@amazon.com>
Signed-off-by: Taylor Gray <tylgry@amazon.com>
Co-authored-by: George Chen <qchea@amazon.com>
Signed-off-by: Jeffrey Aaron Jeyasingh <jeffreyaaron06@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for iterating over a list and manipulating each element, such as renaming a certain key in the element

4 participants