Extended UT Framework for E2E Query conversion

[nssuresh2007]

Changes to test the end to end DSL query conversion without OpenSearch cluster. Currently adds support for match_all and terms with agg for the initial commit.

Previous PR: #21060 (in feature/datafusion branch)

Description

This feature introduces a golden-file-based unit test framework for the OpenSearch DSL query executor plugin (sandbox/plugins/dsl-query-executor/). The framework validates the correctness of both the forward conversion path (DSL → RelNode) and the reverse conversion path (RelNode execution results → SearchResponse DSL output) by comparing actual outputs against pre-approved golden files. Each golden file encodes a complete test case: the input DSL, the expected RelNode logical plan, simulated execution rows, and the expected DSL output.

Requirements

• As a developer, I want each golden file to contain all inputs and expected outputs for a single test case, so that test cases are self-contained and easy to review in code reviews.
• As a developer, I want to validate that a given DSL input produces the expected Calcite logical plan, so that regressions in the conversion pipeline are caught automatically.
• As a developer, I want to validate that simulated execution results are correctly converted back into a SearchResponse, so that regressions in the response building logic are caught automatically.
• As a developer, I want to verify that the forward and reverse paths are consistent within a golden file, so that the entire DSL → RelNode → execution → DSL output pipeline is validated end-to-end.
• As a developer, I want golden file test cases covering the hits pipeline scenarios, so that query translation correctness is validated for common query types.
• As a developer, I want golden file test cases covering the aggregation pipeline scenarios, so that aggregation translation correctness is validated for supported metric and bucket types.
• As a developer, I want the golden file tests to run as fast unit tests without requiring a running OpenSearch cluster, so that they can be part of the standard build cycle.

Related Issues

Resolves #21429

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 1f07e44.

Path	Line	Severity	Description
sandbox/plugins/dsl-query-executor/build.gradle	42	high	New dependency added: com.fasterxml.jackson.core:jackson-databind. Dependency additions must be flagged regardless of apparent legitimacy — artifact authenticity cannot be verified without inspecting the resolved artifact against a trusted source.
sandbox/plugins/dsl-query-executor/build.gradle	43	high	New dependency added: com.fasterxml.jackson.core:jackson-annotations. Dependency additions must be flagged regardless of apparent legitimacy — artifact authenticity cannot be verified without inspecting the resolved artifact against a trusted source.

The table above displays the top 10 most important findings.

Total: 2 | Critical: 0 | High: 2 | Medium: 0 | Low: 0

---

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

---

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

[expani]

Thanks for the PR @nssuresh2007

I like the approach of representing everything in JSON files so it's clear how the conversion happens completely end to end from DSL -> RelNode -> SearchResponse

Considering @vinaykpud comment #21060 (comment) in the previous PR i think we should also consider reusing this JSON within SearchSourceConverterTests so that a new test case can be added with little duplicate effort.

And SearchResponseConversionTest ( will be new ) can reuse this to also assert the response conversion but get the query plan from the same place.

[expani]

Thanks for the iterations @nssuresh2007 LGTM

[expani]

Thanks @nssuresh2007 LGTM

[github-actions]

❌ Gradle check result for 1f07e44: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 3e78cf8: SUCCESS

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.40%. Comparing base (878afa4) to head (3e78cf8).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #21227      +/-   ##
============================================
- Coverage     73.42%   73.40%   -0.02%     
+ Complexity    74538    74518      -20     
============================================
  Files          5978     5978              
  Lines        338734   338734              
  Branches      48842    48842              
============================================
- Hits         248709   248652      -57     
- Misses        70180    70269      +89     
+ Partials      19845    19813      -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.