DATE/TIME label and rendering for date-only / time-only fields (#5521)

* fix(core): cluster D — TIME-typed list elements format as HH:mm:ss

Companion to OpenSearch fix/ai/datetime-clusters @ 9009736c2dc.
list(<TIME-typed field>) now returns "HH:mm:ss[.fraction]" without the
1970-01-01 epoch-date prefix.

The analytics-engine path rewrites PPL list() to DataFusion's list_merge,
so the legacy ListAggFunction never fires. Instead, AnalyticsExecutionEngine
now post-processes List-typed cells in the result conversion: when an
element string matches "1970-01-01[ T]HH:mm:ss[.fraction]", only the time
portion is kept. Scalar cells are untouched, preserving the wider
timestamp-stringification regression baseline.

Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>

* fix(core): cluster A — span() output column type for date/time UDT

Recognize the new sandbox DateOnlyType / TimeOnlyType UDT markers in:
- OpenSearchTypeFactory.convertAnalyticsEngineRelDataTypeToExprType:
  DateOnlyType → ExprCoreType.DATE, TimeOnlyType → ExprCoreType.TIME so the
  user-visible response schema labels span() bucket columns as `date` /
  `time` instead of `timestamp`.
- AnalyticsExecutionEngine.toExprValue: when the column carries a
  DateOnlyType marker, strip the trailing ` HH:MM:SS` from the
  Timestamp(ms)-formatted wire value so dates render as `YYYY-MM-DD`.
  Symmetric handling for TimeOnlyType strips the `1970-01-01 ` prefix.

Pairs with the sandbox schema-builder change in
opensearch-project/OpenSearch@b69c5ff8888.

Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>

* Preserve DATE/TIME return type for ADDDATE/SUBDATE/ADDTIME/SUBTIME on date/time columns

On the analytics-engine route a 'date'/'time' column is a TIMESTAMP-backed
DateOnlyType/TimeOnlyType marker, so operand-conditional return-type inference
mis-read it as TIMESTAMP — ADDDATE(date_col, N) reported (and rendered) TIMESTAMP
instead of DATE. Add isDateExprType/isTimeExprType helpers recognizing those
markers (off the general conversion path, no substrait round-trip risk) and use
them in AddSubDateFunction and PPLReturnTypes.TIME_APPLY_RETURN_TYPE. Fixes the
6 *Null column-operand cases end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>

* Tighten ADDDATE/SUBDATE return-type helper comments

Comment trims on top of d12407b (Marc Handalian's cherry-pick) — collapse
the multi-line Javadoc on isDateExprType/isTimeExprType to one-line case
names, and drop the inline narration at the two call sites
(PPLReturnTypes.TIME_APPLY_RETURN_TYPE,
AddSubDateFunction#getReturnTypeInference) that restated the helper
contract. Behavior unchanged.

Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>

* Fix tests

Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>

---------

Signed-off-by: Vinay Krishna Pudyodu <vinkrish.neo@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: vinaykpud

core/src/main/java/org/opensearch/sql/calcite/utils/OpenSearchTypeFactory.java

@@ -47,7 +47,9 @@
 import org.apache.calcite.sql.type.SqlTypeUtil;
 import org.checkerframework.checker.nullness.qual.Nullable;
 import org.opensearch.analytics.schema.BinaryType;
+import org.opensearch.analytics.schema.DateOnlyType;
 import org.opensearch.analytics.schema.IpType;
+import org.opensearch.analytics.schema.TimeOnlyType;
 import org.opensearch.sql.calcite.type.AbstractExprRelDataType;
 import org.opensearch.sql.calcite.type.ExprBinaryType;
 import org.opensearch.sql.calcite.type.ExprDateType;
@@ -277,6 +279,16 @@ public static ExprType convertRelDataTypeToExprType(RelDataType type) {
     return exprType;
   }
 
+  /** DATE check for return-type inference; recognizes the analytics-route {@link DateOnlyType}. */
+  public static boolean isDateExprType(RelDataType type) {
+    return type instanceof DateOnlyType || convertRelDataTypeToExprType(type) == ExprCoreType.DATE;
+  }
+
+  /** TIME counterpart of {@link #isDateExprType}; recognizes {@link TimeOnlyType}. */
+  public static boolean isTimeExprType(RelDataType type) {
+    return type instanceof TimeOnlyType || convertRelDataTypeToExprType(type) == ExprCoreType.TIME;
+  }
+
   /**
    * Result-schema-only variant of {@link #convertRelDataTypeToExprType} that recognizes the
    * analytics-engine {@link IpType} / {@link BinaryType} markers as {@link ExprCoreType#IP} /
@@ -293,6 +305,13 @@ public static ExprType convertAnalyticsEngineRelDataTypeToExprType(RelDataType t
     if (type instanceof BinaryType) {
       return BINARY;
     }
+    // span() over date / time UDT — schema label is DATE / TIME, not TIMESTAMP.
+    if (type instanceof DateOnlyType) {
+      return DATE;
+    }
+    if (type instanceof TimeOnlyType) {
+      return TIME;
+    }
     return convertRelDataTypeToExprType(type);
   }
 

core/src/main/java/org/opensearch/sql/calcite/utils/PPLReturnTypes.java

@@ -12,7 +12,6 @@
 import org.apache.calcite.sql.type.SqlReturnTypeInference;
 import org.apache.calcite.sql.type.SqlTypeTransforms;
 import org.apache.calcite.sql.type.SqlTypeUtil;
-import org.opensearch.sql.data.type.ExprCoreType;
 
 /**
  * Return types used in PPL. This class complements the {@link
@@ -36,8 +35,7 @@ private PPLReturnTypes() {}
   public static SqlReturnTypeInference TIME_APPLY_RETURN_TYPE =
       opBinding -> {
         RelDataType temporalType = opBinding.getOperandType(0);
-        if (ExprCoreType.TIME.equals(
-            OpenSearchTypeFactory.convertRelDataTypeToExprType(temporalType))) {
+        if (OpenSearchTypeFactory.isTimeExprType(temporalType)) {
           return UserDefinedFunctionUtils.NULLABLE_TIME_UDT;
         }
         return UserDefinedFunctionUtils.NULLABLE_TIMESTAMP_UDT;

core/src/main/java/org/opensearch/sql/executor/analytics/AnalyticsExecutionEngine.java

@@ -12,14 +12,17 @@
 import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
+import java.util.regex.Pattern;
 import org.apache.calcite.plan.RelOptUtil;
 import org.apache.calcite.rel.RelNode;
 import org.apache.calcite.rel.type.RelDataType;
 import org.apache.calcite.rel.type.RelDataTypeField;
 import org.opensearch.analytics.exec.QueryPlanExecutor;
 import org.opensearch.analytics.exec.profile.ProfiledResult;
 import org.opensearch.analytics.schema.BinaryType;
+import org.opensearch.analytics.schema.DateOnlyType;
 import org.opensearch.analytics.schema.IpType;
+import org.opensearch.analytics.schema.TimeOnlyType;
 import org.opensearch.common.network.InetAddresses;
 import org.opensearch.core.action.ActionListener;
 import org.opensearch.sql.ast.statement.ExplainMode;
@@ -47,6 +50,15 @@
  */
 public class AnalyticsExecutionEngine implements ExecutionEngine {
 
+  // TIME-typed list elements round-trip via Timestamp and bypass ArrowValues' scalar
+  // post-processing (see DataFusion list_merge), arriving as "1970-01-01[ T]HH:mm:ss[.frac]".
+  private static final Pattern EPOCH_DATE_TIME_PREFIX =
+      Pattern.compile("^1970-01-01[ T](\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?)$");
+
+  // DateOnlyType wire is Timestamp(ms) at midnight — keep the date, drop the time.
+  private static final Pattern DATE_WITH_MIDNIGHT_TIME =
+      Pattern.compile("^(\\d{4}-\\d{2}-\\d{2})[ T]\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?$");
+
   private final QueryPlanExecutor<RelNode, Iterable<Object[]>> planExecutor;
 
   public AnalyticsExecutionEngine(QueryPlanExecutor<RelNode, Iterable<Object[]>> planExecutor) {
@@ -206,21 +218,7 @@ private List<ExprValue> convertRows(Iterable<Object[]> rows, List<RelDataTypeFie
     return results;
   }
 
-  /**
-   * Converts a single result cell to an {@link ExprValue}, dispatching on the column's UDT when
-   * present so {@code byte[]} payloads are rendered correctly:
-   *
-   * <ul>
-   *   <li>{@link IpType} + {@code byte[]} &rarr; canonical address string (matches {@code
-   *       IpFieldMapper}'s {@code valueFetcher} output).
-   *   <li>{@link BinaryType} + {@code byte[]} &rarr; base64-encoded string (matches the OpenSearch
-   *       {@code binary} field wire format).
-   *   <li>Anything else &rarr; existing {@link ExprValueUtils#fromObjectValue} path.
-   * </ul>
-   *
-   * <p>Without this dispatch, {@code fromObjectValue} throws {@code unsupported object class [B} on
-   * byte[] cells, and IP buffers leak through as raw 16-byte ipv4-mapped-ipv6 garbage.
-   */
+  /** Renders UDT cells (IP/binary byte[]; date / time string) and strips TIME prefixes in lists. */
   private static ExprValue toExprValue(Object value, RelDataType type) {
     if (value instanceof byte[] bytes) {
       if (type instanceof IpType) {
@@ -234,9 +232,39 @@ private static ExprValue toExprValue(Object value, RelDataType type) {
         return ExprValueUtils.stringValue(Base64.getEncoder().encodeToString(bytes));
       }
     }
+    // DateOnlyType scalar — strip midnight time suffix off the Timestamp(ms) wire.
+    if (type instanceof DateOnlyType && value instanceof String s) {
+      var m = DATE_WITH_MIDNIGHT_TIME.matcher(s);
+      if (m.matches()) {
+        return ExprValueUtils.stringValue(m.group(1));
+      }
+    }
+    // TimeOnlyType scalar — strip 1970-01-01 prefix off the Timestamp(ms) wire.
+    if (type instanceof TimeOnlyType && value instanceof String s) {
+      var m = EPOCH_DATE_TIME_PREFIX.matcher(s);
+      if (m.matches()) {
+        return ExprValueUtils.stringValue(m.group(1));
+      }
+    }
+    if (value instanceof List<?> list) {
+      return ExprValueUtils.collectionValue(stripEpochDatePrefixInList(list));
+    }
     return ExprValueUtils.fromObjectValue(value);
   }
 
+  private static List<Object> stripEpochDatePrefixInList(List<?> list) {
+    List<Object> out = new ArrayList<>(list.size());
+    for (Object element : list) {
+      if (element instanceof String s) {
+        var m = EPOCH_DATE_TIME_PREFIX.matcher(s);
+        out.add(m.matches() ? m.group(1) : s);
+      } else {
+        out.add(element);
+      }
+    }
+    return out;
+  }
+
   private Schema buildSchema(List<RelDataTypeField> fields) {
     List<Schema.Column> columns = new ArrayList<>();
     for (RelDataTypeField field : fields) {

core/src/main/java/org/opensearch/sql/expression/function/udf/datetime/AddSubDateFunction.java

@@ -63,7 +63,7 @@ public SqlReturnTypeInference getReturnTypeInference() {
     return opBinding -> {
       RelDataType temporalType = opBinding.getOperandType(0);
       RelDataType temporalDeltaType = opBinding.getOperandType(1);
-      if (OpenSearchTypeFactory.convertRelDataTypeToExprType(temporalType) == ExprCoreType.DATE
+      if (OpenSearchTypeFactory.isDateExprType(temporalType)
           && SqlTypeFamily.NUMERIC.contains(temporalDeltaType)) {
         return NULLABLE_DATE_UDT;
       } else {

core/src/test/java/org/opensearch/sql/executor/analytics/AnalyticsExecutionEngineTest.java

@@ -30,7 +30,9 @@
 import org.junit.jupiter.api.Test;
 import org.opensearch.analytics.exec.QueryPlanExecutor;
 import org.opensearch.analytics.schema.BinaryType;
+import org.opensearch.analytics.schema.DateOnlyType;
 import org.opensearch.analytics.schema.IpType;
+import org.opensearch.analytics.schema.TimeOnlyType;
 import org.opensearch.core.action.ActionListener;
 import org.opensearch.sql.calcite.CalcitePlanContext;
 import org.opensearch.sql.calcite.SysLimit;
@@ -237,6 +239,67 @@ void executeRelNode_binaryColumnRendersAsBase64() {
         "byte[] should base64-encode to match OpenSearch binary wire format. " + dump);
   }
 
+  /** DateOnlyType — schema reports DATE, value strips midnight suffix. */
+  @Test
+  void executeRelNode_dateOnlyTypeStripsTimeSuffix() {
+    RelNode relNode =
+        mockRelNodeWithType("d", new DateOnlyType(RelDataTypeSystem.DEFAULT, true, 3));
+    Iterable<Object[]> rows = Collections.singletonList(new Object[] {"1984-04-12 00:00:00"});
+    stubExecutorWith(relNode, rows);
+
+    QueryResponse response = executeAndCapture(relNode);
+    String dump = dumpResponse(response);
+
+    assertEquals(ExprCoreType.DATE, response.getSchema().getColumns().get(0).getExprType(), dump);
+    assertEquals(
+        "1984-04-12", response.getResults().get(0).tupleValue().get("d").stringValue(), dump);
+  }
+
+  /** TimeOnlyType — schema reports TIME, value strips 1970-01-01 prefix. */
+  @Test
+  void executeRelNode_timeOnlyTypeStripsEpochDatePrefix() {
+    RelNode relNode =
+        mockRelNodeWithType("t", new TimeOnlyType(RelDataTypeSystem.DEFAULT, true, 3));
+    Iterable<Object[]> rows = Collections.singletonList(new Object[] {"1970-01-01 09:00:00"});
+    stubExecutorWith(relNode, rows);
+
+    QueryResponse response = executeAndCapture(relNode);
+    String dump = dumpResponse(response);
+
+    assertEquals(ExprCoreType.TIME, response.getSchema().getColumns().get(0).getExprType(), dump);
+    assertEquals(
+        "09:00:00", response.getResults().get(0).tupleValue().get("t").stringValue(), dump);
+  }
+
+  /** TIME-typed list elements arrive as "1970-01-01[ T]HH:mm:ss[.frac]" — strip the prefix. */
+  @Test
+  void executeRelNode_listOfStringStripsEpochDatePrefix() {
+    SqlTypeFactoryImpl typeFactory = new SqlTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
+    RelDataType arrayOfVarchar =
+        typeFactory.createArrayType(typeFactory.createSqlType(SqlTypeName.VARCHAR), -1);
+    RelNode relNode = mockRelNodeWithType("time_list", arrayOfVarchar);
+    java.util.List<String> input =
+        Arrays.asList(
+            "1970-01-01 19:36:22",
+            "1970-01-01T02:05:25",
+            "1970-01-01 12:34:56.123456789",
+            "2020-10-13 13:00:00",
+            "hello");
+    stubExecutorWith(relNode, Collections.singletonList(new Object[] {input}));
+
+    QueryResponse response = executeAndCapture(relNode);
+    String dump = dumpResponse(response);
+
+    java.util.List<String> result =
+        response.getResults().get(0).tupleValue().get("time_list").collectionValue().stream()
+            .map(org.opensearch.sql.data.model.ExprValue::stringValue)
+            .toList();
+    assertEquals(
+        Arrays.asList("19:36:22", "02:05:25", "12:34:56.123456789", "2020-10-13 13:00:00", "hello"),
+        result,
+        dump);
+  }
+
   @Test
   void executeRelNode_emptyResults() {
     RelNode relNode = mockRelNode("name", SqlTypeName.VARCHAR);