Skip to content

Commit 2f1f0ec

Browse files
bowenlan-amznahkcs
authored andcommitted
Route analytics queries by index setting, not table-name prefix (opensearch-project#5432)
Today `RestUnifiedQueryAction.isAnalyticsIndex` dispatches to the analytics engine when the source index name starts with `parquet_`. That's brittle — it conflates naming convention with storage type. An index created without the prefix but with pluggable dataformat enabled is silently sent to the Lucene path; an index named `parquet_foo` without the setting is mis-dispatched to analytics. Use the authoritative signal instead: the `index.pluggable.dataformat.enabled` setting on cluster-state metadata. This is the same setting integration tests (`CoordinatorReduceIT`, `CompositeCommitDeletionIT`, etc.) already use to create analytics-backed indices, and it's what `FieldStorageResolver` reads to decide field-level storage. Behavior: - `index.pluggable.dataformat.enabled=true` → analytics engine (DataFusion) - flag absent / false / index missing → Calcite→OpenSearch DSL path Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
1 parent b0b6c12 commit 2f1f0ec

2 files changed

Lines changed: 4 additions & 4 deletions

File tree

plugin/src/main/java/org/opensearch/sql/plugin/rest/RestUnifiedQueryAction.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ public RestUnifiedQueryAction(
7676
/**
7777
* Returns true iff the target index has {@link
7878
* IndexSettings#PLUGGABLE_DATAFORMAT_ENABLED_SETTING} set and {@link
79-
* IndexSettings#PLUGGABLE_DATAFORMAT_VALUE_SETTING} is {@code "parquet"}, routing it to
79+
* IndexSettings#PLUGGABLE_DATAFORMAT_VALUE_SETTING} is {@code "composite"}, routing it to
8080
* DataFusion instead of the Calcite→DSL path.
8181
*
8282
* <p>Note: This creates a separate UnifiedQueryContext for parsing. The context cannot be shared
@@ -110,7 +110,7 @@ private boolean isPluggableDataformatIndex(String indexName) {
110110
}
111111
var settings = indexMetadata.getSettings();
112112
return IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.get(settings)
113-
&& "parquet".equals(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.get(settings));
113+
&& "composite".equals(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.get(settings));
114114
}
115115

116116
/** Execute a query through the unified query pipeline on the sql-worker thread pool. */

plugin/src/test/java/org/opensearch/sql/plugin/rest/RestUnifiedQueryActionTest.java

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525

2626
/**
2727
* Tests for analytics index routing in RestUnifiedQueryAction. Routing requires both {@code
28-
* index.pluggable.dataformat.enabled=true} and {@code index.pluggable.dataformat=parquet}.
28+
* index.pluggable.dataformat.enabled=true} and {@code index.pluggable.dataformat=composite}.
2929
*/
3030
public class RestUnifiedQueryActionTest {
3131

@@ -57,7 +57,7 @@ public void pluggableDataformatIndexRoutesToAnalytics() {
5757
"parquet_logs",
5858
Settings.builder()
5959
.put(IndexSettings.PLUGGABLE_DATAFORMAT_ENABLED_SETTING.getKey(), true)
60-
.put(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.getKey(), "parquet")
60+
.put(IndexSettings.PLUGGABLE_DATAFORMAT_VALUE_SETTING.getKey(), "composite")
6161
.build());
6262

6363
assertTrue(action.isAnalyticsIndex("source = parquet_logs | fields ts", QueryType.PPL));

0 commit comments

Comments
 (0)