Fix QTF stitcher precision mismatch on date_nanos fields

[vinaykpud]

Description

ArrowCalciteTypes.toArrow() hardcoded TimeUnit.MILLISECOND for every Calcite TIMESTAMP. Shards emit Timestamp(NANOSECOND) for date_nanos, but the late-materialization Stitcher pre-allocated a TimestampMilliVector for the output. Arrow's copyFromSafe requires identical fixed-width subclasses, so it threw an empty IllegalArgumentException — surfacing to clients as HTTP 400 "Invalid Query".

Fix: branch on Calcite precision — precision == 9 → NANOSECOND, else MILLISECOND.

Query shapes this fixes

Triggered on a multi-shard parquet-backed index with a date_nanos field when the plan includes:

• projection + sort on the date_nanos column
• head N (or other top-K)
• at least one fetch-only field above the anchor

Example PPL:

source = idx | where match(body, 'failed') and service = 'checkout'
             | sort - ts | fields ts, severity, body | head 4

Drop any of those and the LM rewriter declines QTF or the bug doesn't fire.

Tests

• ArrowCalciteTypesTests — 3 new cases covering precision 3, 9, and default.
• LateMaterializationDateNanosIT — new 2-shard parquet+lucene IT running the q01-style plan. Pre-fix: HTTP 400. Post-fix: 4 rows in DESC order with sub-millisecond precision preserved.

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

📝 TODO sections sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/planner/ArrowCalciteTypes.java [49]: TIMESTAMP_WITH_LOCAL_TIME_ZONE, DATE, TIME, DECIMAL still missing.

🔀 No multiple PR themes

⚡ No major issues detected

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Validate TIMESTAMP precision range The precision check should validate the range of supported precisions. Calcite TIMESTAMP precision can be 0-9, but only precision 9 maps to NANOSECOND. Consider handling other precision values explicitly or documenting the assumption that non-9 precisions default to MILLISECOND. sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/planner/ArrowCalciteTypes.java [51] -case TIMESTAMP -> new ArrowType.Timestamp(t.getPrecision() == 9 ? TimeUnit.NANOSECOND : TimeUnit.MILLISECOND, null); +case TIMESTAMP -> { + int precision = t.getPrecision(); + if (precision == 9) { + yield new ArrowType.Timestamp(TimeUnit.NANOSECOND, null); + } else if (precision >= 0 && precision <= 3) { + yield new ArrowType.Timestamp(TimeUnit.MILLISECOND, null); + } else { + throw new IllegalArgumentException("Unsupported TIMESTAMP precision: " + precision + " (expected 0-3 or 9)"); + } +} Suggestion importance[1-10]: 4 __ Why: The suggestion raises a valid concern about handling intermediate precision values (4-8), but the proposed solution may be overly restrictive. The current implementation uses a simple binary check (precision 9 vs. others), which aligns with the test cases showing precision 0, 3, and 9 are handled. However, throwing an exception for precisions 4-8 could break existing functionality if those values are used elsewhere in the codebase.	Low

[github-actions]

✅ Gradle check result for db76edf: SUCCESS

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.48%. Comparing base (a8692f7) to head (db76edf).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #22095      +/-   ##
============================================
- Coverage     73.52%   73.48%   -0.05%     
+ Complexity    75682    75673       -9     
============================================
  Files          6038     6038              
  Lines        343009   343011       +2     
  Branches      49348    49348              
============================================
- Hits         252189   252045     -144     
- Misses        70746    70973     +227     
+ Partials      20074    19993      -81     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vinaykpud]

opensearch-project/sql#5498