feat(query): add KQL (Kibana Query Language) support

Adds optional KQL parsing as a thin translation layer at the REST entry
point. KQL inputs are parsed and lowered to QueryAst using only existing
variants (BoolQuery, FullTextQuery, RangeQuery, FieldPresenceQuery,
WildcardQuery, MatchAll, UserInputQuery) — the core enum, visitor traits,
tag pruning, and root-search remain unchanged.

Wire surface:
  * Native REST: `?kql=<expr>` on /api/v1/{index}/search, mutually
    exclusive with the existing `query=` parameter.
  * Elastic-compat JSON: `{"query": {"kql": {"query": "...", ...}}}` on
    /api/v1/_elastic/{index}/_search. Documented as a Quickwit-only
    extension since real Elasticsearch returns parsing_exception.

Supported grammar (matches the public Kibana KQL reference):
  field-value, quoted phrases, bare default-field terms, `*` (match-all
  fast path), `?`/`*` wildcards, `field:*` exists, `field:>=N` /
  `>` / `<=` / `<` ranges (numeric literals coerced to JsonLiteral::Number,
  non-numeric falls back to String), boolean and/or/not (case-insensitive),
  juxtaposition as implicit AND, parens, `field:(a or b)` value groups
  with proper field distribution, escape semantics (`\and`, `\:`, `\+`).

Safety rails (all return HTTP 400 with specific error messages):
  * Max KQL input length: 16 KiB (REST layer)
  * Max parser nesting depth: 64
  * Max bare-token length: 1 KiB
  * Max quoted-phrase length: 4 KiB
  * `{...}` nested-field syntax rejected (Quickwit has no nested type)
  * Nested field qualifier inside value group rejected
  * `query` and `kql` mutually exclusive

Observability:
  * quickwit_kql_parse_total counter
  * quickwit_kql_parse_failures_total counter
  * quickwit_kql_parse_duration_seconds histogram
  * Structured `kql=<bool>`, `tantivy_grammar=<bool>` fields on search
    log lines so SRE can split KQL vs Tantivy-grammar traffic without
    parsing raw query strings.

OpenAPI:
  * `query` is now `#[serde(default)]` (semantically optional at the
    wire layer); utoipa override exposes both `query` and `kql` as
    `Option<String>` so generated SDK clients no longer encode the
    obsolete `required: ["query"]` contract.

Tests:
  * 246 unit tests in quickwit-query covering lexer, parser
    (recursive-descent with depth guard), lowering (with
    Tantivy-grammar escape handling for default-field deferral),
    metrics wiring, JSON DSL deserialization, and proptest fuzz
    (~6k cases) confirming the parser never panics on arbitrary input.
  * Kibana conformance corpus pinning expected ASTs for each
    documented KQL idiom + explicit notes on intentional divergences.
  * REST handler unit tests for `kql`/`query` mutual exclusion,
    whitespace handling, size caps, and search_fields propagation.
  * Integration scenarios under rest-api-tests/scenarii/kql_search/
    asserting exact hit counts against a known dataset.
  * Concurrent load harness (load_test.py) mixing happy-path and
    adversarial shapes; multi-node docker-compose template for
    distributed root→leaf testing.

Line coverage on KQL production code: 95-99% per file; the remaining
gaps are defensive code, test panic-guards in let-else patterns, and
lazy_counter!/lazy_histogram! macro internals the coverage tool cannot
introspect.

Files modified outside the new kql/ module: 5 (Cargo.lock,
quickwit-cli/src/tool.rs, quickwit-query/Cargo.toml,
quickwit-query/src/elastic_query_dsl/mod.rs,
quickwit-query/src/lib.rs, quickwit-serve/src/search_api/rest_handler.rs).
The core QueryAst enum, QueryAstVisitor, QueryAstTransformer,
tag_pruning, and root-search are untouched.

Author: siva-abstract-security

quickwit/Cargo.lock

@@ -542,6 +542,7 @@ pub async fn local_search_cli(args: LocalSearchArgs) -> anyhow::Result<()> {
     let sort_by: SortBy = args.sort_by_field.map(SortBy::from).unwrap_or_default();
     let search_request_query_string = SearchRequestQueryString {
         query: args.query,
+        kql: None,
         start_offset: args.start_offset as u64,
         max_hits: args.max_hits as u64,
         search_fields: args.search_fields,

quickwit/quickwit-cli/src/tool.rs

@@ -29,6 +29,7 @@ rustc-hash = { workspace = true }
 
 quickwit-common = { workspace = true }
 quickwit-datetime = { workspace = true }
+quickwit-metrics = { workspace = true }
 quickwit-proto = { workspace = true }
 
 [dev-dependencies]

quickwit/quickwit-query/Cargo.toml

@@ -0,0 +1,53 @@
+// Copyright 2021-Present Datadog, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+/// Parsed KQL expression. This is an internal representation that lowers to
+/// `QueryAst` via `lower_kql_ast`.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub(crate) enum KqlAst {
+    /// Conjunction of subqueries. Empty vector is not produced by the parser.
+    And(Vec<KqlAst>),
+    /// Disjunction of subqueries.
+    Or(Vec<KqlAst>),
+    /// Negation of a subquery.
+    Not(Box<KqlAst>),
+    /// `field:value` clause.
+    FieldValue { field: String, value: KqlValue },
+    /// `field:<op><value>` numeric/datetime range bound.
+    FieldRange {
+        field: String,
+        op: RangeOp,
+        value: String,
+    },
+    /// `field:*` — checks whether the field is present on the document.
+    FieldExists { field: String },
+    /// Bare value with no field qualifier — matches against default fields.
+    DefaultValue(KqlValue),
+}
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub(crate) enum KqlValue {
+    /// Unquoted token. May contain `*` and `?` wildcards.
+    Literal(String),
+    /// Double-quoted phrase. Wildcards inside are treated literally.
+    Phrase(String),
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub(crate) enum RangeOp {
+    Gt,
+    Gte,
+    Lt,
+    Lte,
+}