feat: add exists() support via FT.AGGREGATE APPLY/FILTER (#14)

* feat: add IS NULL / IS NOT NULL support via RediSearch ismissing()

Translate SQL IS NULL and IS NOT NULL to RediSearch ismissing(@field)
and -ismissing(@field) respectively. Requires Redis 7.4+ with the
INDEXMISSING attribute declared on the target field.

Changes:
- parser: handle sqlglot exp.Is nodes, emit IS_NULL / IS_NOT_NULL operators
- query_builder: add build_missing_condition() for ismissing() syntax
- translator: short-circuit IS_NULL/IS_NOT_NULL before field-type dispatch
- translator: make DIALECT 2 the default for all FT.SEARCH and FT.AGGREGATE
- executor: catch and re-raise ResponseError with clear version guidance
  when ismissing() fails (both sync and async paths)
- translator: emit UserWarning on IS NULL/IS NOT NULL noting Redis 7.4+
  and INDEXMISSING requirements

Tests:
- 18 integration tests against Redis 8 (TAG, TEXT, NUMERIC field types,
  combined conditions, edge cases, raw command verification)
- Unit tests for parser, query_builder, and translator
- Warning and error message verification tests

* chore: linter

* feat: add exists() support via FT.AGGREGATE APPLY/FILTER

Translate SQL exists(field) to RediSearch exists(@field) in
FT.AGGREGATE APPLY (projection) and FILTER (HAVING) clauses.

- Parser: handle exp.Exists in SELECT → computed_fields and
  HAVING → filters, distinguishing from SQL EXISTS (SELECT ...)
- Analyzer: extract field references from exists() for schema
  validation
- Translator: force FT.AGGREGATE when exists() is used, generate
  APPLY/FILTER clauses, include exists() fields in LOAD
- Reject exists() in WHERE with clear error (aggregate-only function)
- No INDEXMISSING attribute required — works on any indexed field

Tests: 8 integration, 5 parser, 7 translator (350 total pass)

* fix: handle sqlglot uppercasing exists() to EXISTS() in arithmetic expressions like exists(a) + exists(b), ensuring the referenced fields are included in LOAD args and don't cause a Property not loaded runtime error.

* fix: handle Paren in HAVING clause and move exists() FILTER after GROUPBY/REDUCE

- parser: unwrap exp.Paren in _process_having_clause so queries like
  HAVING (exists(email)) don't raise 'Unsupported HAVING expression'
- translator: move exists() FILTER emission to after GROUPBY/REDUCE block
  so it acts as post-aggregation filtering (correct HAVING semantics)
  instead of pre-group document filtering

* refactor: move all local imports to module scope

Move import re, import asyncio, import warnings, import time, and
several from-imports out of function/method bodies to module-level
imports. Local imports inside TYPE_CHECKING guards and try/except
compatibility blocks are intentionally kept.

* fix: remove benchmark file accidentally committed to feat/exists

* fix: emit LOAD * in FT.AGGREGATE when SELECT * is used

When FT.AGGREGATE is forced (e.g., by HAVING exists()), SELECT * was
skipping the wildcard in the LOAD field collection loop, resulting in
no fields being loaded and empty/partial rows.

Now detects SELECT * in aggregate mode and emits LOAD * so RediSearch
returns all document attributes.

Adds test for SELECT * with HAVING exists() verifying LOAD * is emitted.

* fix: emit LOAD * in FT.AGGREGATE when SELECT * is used

When FT.AGGREGATE is forced (e.g., by HAVING exists()), SELECT * was
skipping the wildcard in the LOAD field collection loop, resulting in
no fields being loaded and empty/partial rows.

Now detects SELECT * in aggregate mode and emits LOAD * so RediSearch
returns all document attributes.

Author: nkanu17

sql_redis/analyzer.py

@@ -111,6 +111,12 @@ def analyze(self, parsed: ParsedQuery) -> AnalyzedQuery:
                 if field_name in computed.expression:
                     referenced_fields.add(field_name)
 
+        # Fields from filters (HAVING exists(field))
+        for filter_expr in parsed.filters:
+            for field_name in schema.keys():
+                if field_name in filter_expr:
+                    referenced_fields.add(field_name)
+
         # Fields from vector search
         if parsed.vector_search:
             referenced_fields.add(parsed.vector_search.field)

sql_redis/parser.py

@@ -218,6 +218,7 @@ class ParsedQuery:
     )  # (field, ASC|DESC)
     limit: int | None = None
     offset: int | None = None
+    filters: list[str] = dataclasses.field(default_factory=list)
 
 
 class SQLParser:
@@ -260,6 +261,11 @@ def parse(self, sql: str) -> ParsedQuery:
                 if isinstance(expr, exp.Column):
                     result.groupby_fields.append(expr.name)
 
+        # Extract HAVING clause — exists() in HAVING → FILTER
+        having = ast.find(exp.Having)
+        if having:
+            self._process_having_clause(having.this, result)
+
         # Extract ORDER BY clause
         order = ast.find(exp.Order)
         if order:
@@ -392,6 +398,24 @@ def _process_select_expression_inner(
                     extra_args=extra_args,
                 )
             )
+        elif isinstance(expression, exp.Exists):
+            # exists(field) — RediSearch aggregation function
+            # sqlglot parses exists(col) as exp.Exists(this=Column),
+            # distinct from EXISTS (SELECT ...) which has this=Select.
+            inner = expression.this
+            if isinstance(inner, exp.Column):
+                field_name = inner.name
+                expr_str = f"exists({field_name})"
+                field_alias = alias if alias else f"exists_{field_name}"
+                result.computed_fields.append(
+                    ComputedField(expression=expr_str, alias=field_alias)
+                )
+            else:
+                raise ValueError(
+                    "exists() in SELECT expects a column reference, "
+                    f"got {type(inner).__name__}. "
+                    "Use exists(field_name) for RediSearch field existence checks."
+                )
         elif isinstance(expression, exp.Anonymous):
             # Custom function call (e.g., vector_distance) - check before exp.Func
             # since Anonymous is a subclass of Func
@@ -664,10 +688,43 @@ def _process_where_clause(
                     "Unsupported IS expression in WHERE clause; only "
                     "`column IS NULL` and `column IS NOT NULL` are supported."
                 )
+        elif isinstance(expression, exp.Exists):
+            # Distinguish exists(column) from EXISTS (SELECT ...)
+            inner = expression.this
+            if isinstance(inner, exp.Column):
+                # exists(field) — RediSearch aggregate function, not valid in WHERE
+                raise ValueError(
+                    "exists() is a RediSearch aggregate function and cannot be "
+                    "used in WHERE clauses. Use HAVING exists(field) instead "
+                    "for post-aggregate filtering."
+                )
+            # EXISTS (SELECT ...) — SQL subquery, silently ignored (not supported)
         elif isinstance(expression, exp.Anonymous):
             # Custom function like MATCH(field, value)
             self._add_function_condition(expression, result, negated)
 
+    def _process_having_clause(self, expression, result: ParsedQuery) -> None:
+        """Process HAVING clause — routes exists() to filters."""
+        if isinstance(expression, exp.Exists):
+            inner = expression.this
+            if isinstance(inner, exp.Column):
+                result.filters.append(f"exists({inner.name})")
+            else:
+                raise ValueError(
+                    "exists() in HAVING expects a column reference, "
+                    f"got {type(inner).__name__}."
+                )
+        elif isinstance(expression, exp.Paren):
+            self._process_having_clause(expression.this, result)
+        elif isinstance(expression, exp.And):
+            self._process_having_clause(expression.this, result)
+            self._process_having_clause(expression.expression, result)
+        else:
+            raise ValueError(
+                f"Unsupported HAVING expression: {type(expression).__name__}. "
+                "Only exists(field) is supported in HAVING."
+            )
+
     def _add_condition(
         self, expression, operator: str, result: ParsedQuery, negated: bool
     ) -> None:

sql_redis/schema.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import asyncio
 from typing import TYPE_CHECKING, Callable
 
 import redis
@@ -172,8 +173,6 @@ async def load_all(self) -> None:
 
         Uses asyncio.gather() to load all index schemas concurrently.
         """
-        import asyncio
-
         self._schemas.clear()
         indexes = await self._client.execute_command("FT._LIST")
         # Decode bytes to strings

sql_redis/translator.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import re
 import warnings
 from dataclasses import dataclass, field
 
@@ -120,6 +121,7 @@ def _build_command(self, analyzed: AnalyzedQuery) -> TranslatedQuery:
             or geo_requires_aggregate  # geo_distance with >, >=, BETWEEN
             or len(analyzed.date_functions) > 0
             or has_date_func_conditions
+            or len(parsed.filters) > 0  # exists() in HAVING → FILTER
         )
 
         # Build query string from conditions
@@ -333,33 +335,44 @@ def _build_aggregate(
         geo_filter_conditions = list(parsed.geo_conditions)
 
         # LOAD fields if needed
-        load_fields = set()
-        for agg in analyzed.aggregations:
-            if agg.field:
-                load_fields.add(agg.field)
-        for field_name in analyzed.groupby_fields:
-            load_fields.add(field_name)
-        # Load geo fields used in geo_distance() SELECT expressions
-        for geo_select in parsed.geo_distance_selects:
-            load_fields.add(geo_select.field)
-        # Load geo fields used in geo_distance() WHERE with >, >=, BETWEEN
-        for geo_cond in geo_filter_conditions:
-            load_fields.add(geo_cond.field)
-        # Load source fields for date functions in SELECT
-        for date_func in analyzed.date_functions:
-            load_fields.add(date_func.field)
-        # Load source fields for date function conditions in WHERE
-        for condition in parsed.conditions:
-            if self._is_date_function_condition(condition):
-                load_fields.add(condition.field)
-        # Load explicit SELECT fields for FT.AGGREGATE
-        for field_name in parsed.fields:
-            if field_name != "*":
+        # SELECT * in aggregate mode → LOAD * (all document attributes)
+        load_all = "*" in (parsed.fields or [])
+
+        load_fields: set[str] = set()
+        if not load_all:
+            for agg in analyzed.aggregations:
+                if agg.field:
+                    load_fields.add(agg.field)
+            for field_name in analyzed.groupby_fields:
+                load_fields.add(field_name)
+            # Load geo fields used in geo_distance() SELECT expressions
+            for geo_select in parsed.geo_distance_selects:
+                load_fields.add(geo_select.field)
+            # Load geo fields used in geo_distance() WHERE with >, >=, BETWEEN
+            for geo_cond in geo_filter_conditions:
+                load_fields.add(geo_cond.field)
+            # Load source fields for date functions in SELECT
+            for date_func in analyzed.date_functions:
+                load_fields.add(date_func.field)
+            # Load source fields for date function conditions in WHERE
+            for condition in parsed.conditions:
+                if self._is_date_function_condition(condition):
+                    load_fields.add(condition.field)
+            # Load explicit SELECT fields for FT.AGGREGATE
+            for field_name in parsed.fields:
                 # Skip computed fields (they have aliases from geo_distance)
                 if field_name not in [gs.alias for gs in parsed.geo_distance_selects]:
                     load_fields.add(field_name)
-
-        if load_fields:
+            # Load fields referenced in exists() filters (HAVING)
+            for filter_expr in parsed.filters:
+                self._extract_exists_fields(filter_expr, load_fields)
+            # Load fields referenced in exists() computed fields (SELECT)
+            for computed in analyzed.computed_fields:
+                self._extract_exists_fields(computed.expression, load_fields)
+
+        if load_all:
+            args.extend(["LOAD", "*"])
+        elif load_fields:
             args.append("LOAD")
             args.append(str(len(load_fields)))
             # Redis expects property names prefixed with '@' in LOAD
@@ -498,6 +511,13 @@ def _build_aggregate(
                 alias = agg.alias or agg.function.lower()
                 args.extend(["AS", alias])
 
+        # FILTER for exists() from HAVING clause (post-aggregation)
+        for filter_expr in parsed.filters:
+            prefixed = self._prefix_fields_in_expression(
+                filter_expr, analyzed.field_types
+            )
+            args.extend(["FILTER", prefixed])
+
         # SORTBY
         if parsed.orderby_fields:
             args.append("SORTBY")
@@ -593,12 +613,16 @@ def _convert_to_meters(self, value: float, unit: str) -> float:
             )
         return value * conversions[normalized_unit]
 
+    @staticmethod
+    def _extract_exists_fields(expression: str, load_fields: set[str]) -> None:
+        """Extract field names from exists() calls and add to load_fields."""
+        for match in re.finditer(r"exists\((\w+)\)", expression, re.IGNORECASE):
+            load_fields.add(match.group(1))
+
     def _prefix_fields_in_expression(
         self, expression: str, schema: dict[str, str]
     ) -> str:
         """Prefix field names with @ in an expression for Redis APPLY."""
-        import re
-
         result = expression
         for field_name in schema:
             # Match field name as a whole word, not already prefixed with @

tests/test_date_fields.py

@@ -1,8 +1,10 @@
 """Tests for DATE/DATETIME literal parsing and conversion."""
 
 import pytest
+import redis as redis_lib
 
 from sql_redis.parser import SQLParser
+from sql_redis.schema import SchemaRegistry
 from sql_redis.translator import Translator
 
 
@@ -126,8 +128,6 @@ class TestDateTranslation:
     @pytest.fixture
     def date_index(self, redis_client):
         """Create an index with NUMERIC field for dates."""
-        import redis as redis_lib
-
         index_name = "test_dates"
         try:
             redis_client.execute_command("FT.DROPINDEX", index_name, "DD")
@@ -153,8 +153,6 @@ def date_index(self, redis_client):
     @pytest.fixture
     def date_translator(self, redis_client, date_index):
         """Create a translator with the date index loaded."""
-        from sql_redis.schema import SchemaRegistry
-
         registry = SchemaRegistry(redis_client)
         registry.load_all()
         return Translator(registry)

tests/test_date_functions.py

@@ -1,8 +1,11 @@
 """Tests for DATE function parsing and translation (Phase 2 & 3)."""
 
 import pytest
+import redis as redis_lib
 
 from sql_redis.parser import SQLParser
+from sql_redis.schema import SchemaRegistry
+from sql_redis.translator import Translator
 
 
 class TestDateFunctionParsing:
@@ -108,8 +111,6 @@ class TestDateFunctionTranslation:
     @pytest.fixture
     def date_index(self, redis_client):
         """Create an index with NUMERIC field for dates."""
-        import redis as redis_lib
-
         index_name = "test_date_funcs"
         try:
             redis_client.execute_command("FT.DROPINDEX", index_name, "DD")
@@ -135,9 +136,6 @@ def date_index(self, redis_client):
     @pytest.fixture
     def date_translator(self, redis_client, date_index):
         """Create a translator with the date index loaded."""
-        from sql_redis.schema import SchemaRegistry
-        from sql_redis.translator import Translator
-
         registry = SchemaRegistry(redis_client)
         registry.load_all()
         return Translator(registry)