fix(service-analytics): $or / $not filters reach the query (#4128 follow-up) (#4175)

The last of the silently-dropped filter family, and the one dropped for a
structural reason rather than an oversight: `normalizeAnalyticsFilters`
produced a flat ARRAY, which cannot carry a disjunction. Both strategies
therefore skipped `$or` and `$not`, so a widget or dataset filtering with
either compiled a WHERE clause that simply did not contain it and drew every
row — #3650's symptom, and unlike a rejected query it looks like a working
chart.

The normalizer now produces a TREE (`normalizeAnalyticsFilterTree`), the
single owner of the vocabulary, and each strategy compiles it the way its own
backend expresses a disjunction:

- NativeSQLStrategy builds the WHERE recursively, routing every leaf through
  its existing clause emitter — so the storage-form coercion and the
  calendar-day upper-bound rule (#3777) apply at every depth, including inside
  an $or, where a second combinator-aware emitter would have been free to
  drift from the first. Parentheses are explicit rather than resting on SQL
  precedence: `a AND b OR c` happens to be right, and being right by
  construction is what stops a later edit making it wrong.
- ObjectQLStrategy hands $or/$not to the engine, which speaks them natively.
  AND-ed leaves still merge per field exactly as before, so a query without
  combinators produces byte-identical engine input.
- /analytics/sql renders the same tree. That block's own comment warns against
  echoing something other than what executes; a dropped disjunction was that
  lie in the other direction.
- The cross-object envelope check now sees members nested in an $or. It
  rejects cross-object filters, so a member it could not see was a filter it
  could not reject.

Empty $and/$or arrays throw rather than being ignored, and the tree walker's
combinator handling deliberately mirrors `read-scope-sql.ts` — the compiler in
this same package that has always handled the full tree. That the package held
one correct implementation and one lossy one for the same filter shape, with
nothing holding them to each other, is the actual defect behind this one.

`native-sql-filter-logic-conformance.test.ts` runs the SHARED combinator table
(FILTER_LOGIC_CASES, #3774) against a real SQLite engine and asserts row ids,
so the analytics raw-SQL path now stands beside driver-sql, driver-memory,
formula and read-scope-sql under one standard. 14 of its 17 cases fail without
this change.

Refs #4128, #3774, #3650.

Author: os-zhuang

.changeset/analytics-or-not-combinators.md

@@ -0,0 +1,40 @@
+---
+"@objectstack/service-analytics": patch
+---
+
+fix(service-analytics): a `$or` / `$not` filter no longer vanishes from an analytics query (#4128 follow-up)
+
+The last of the silently-dropped filter family. `normalizeAnalyticsFilters`
+produced a flat **array**, which cannot carry a disjunction, so both strategies
+skipped `$or` and `$not` outright — a widget or dataset whose filter used
+either compiled a WHERE clause that simply did not contain it, and drew every
+row. That is #3650's symptom, and unlike a rejected query it looks like a
+working chart.
+
+The normalizer now produces a **tree** (`normalizeAnalyticsFilterTree`), and
+each strategy compiles it the way its own backend expresses a disjunction:
+
+- **`NativeSQLStrategy`** builds the WHERE recursively, routing every leaf
+  through its existing clause emitter — so the storage-form coercion and the
+  calendar-day upper-bound rule (#3777) apply at every depth, including inside
+  an `$or`. Parentheses are explicit rather than relying on SQL precedence.
+- **`ObjectQLStrategy`** hands `$or` / `$not` to the engine, which speaks them
+  natively. AND-ed leaves still merge per field exactly as before, so a query
+  without combinators produces byte-identical engine input.
+- **`/analytics/sql`** renders the same tree, so the echoed statement keeps
+  reproducing what executes rather than showing a conjunction where the engine
+  runs a disjunction.
+- The **cross-object envelope check** now sees members nested inside an `$or`.
+  It rejects cross-object filters, so a member it could not see was a filter it
+  could not reject.
+
+Empty `$and` / `$or` arrays now throw instead of being ignored, matching the
+fail-closed stance of `read-scope-sql.ts` — the compiler in this same package
+that has always handled the full tree, and whose semantics the tree walker now
+mirrors deliberately.
+
+Cover is `native-sql-filter-logic-conformance.test.ts`, which runs the shared
+combinator table (`FILTER_LOGIC_CASES`, #3774) against a real SQLite engine and
+asserts row ids. The analytics raw-SQL path now stands beside `driver-sql`,
+`driver-memory`, `formula` and `read-scope-sql` under that one standard; 14 of
+its 17 cases fail without this change.

packages/services/service-analytics/src/__tests__/filter-operator-coverage.test.ts

@@ -28,7 +28,7 @@ import type { AnalyticsQuery, FilterCondition } from '@objectstack/spec/data';
 import type { StrategyContext } from '@objectstack/spec/contracts';
 
 import { NativeSQLStrategy } from '../strategies/native-sql-strategy.js';
-import { normalizeAnalyticsFilters } from '../strategies/filter-normalizer.js';
+import { normalizeAnalyticsFilterTree } from '../strategies/filter-normalizer.js';
 
 interface Row {
   id: string;
@@ -181,12 +181,12 @@ describe('analytics filters — every authorable operator reaches the query (#41
     // rows the filter excludes. A typo'd or non-spec operator is a caller
     // error, and a loud one — the same call driver-memory made in #3948.
     expect(() =>
-      normalizeAnalyticsFilters({ where: { name: { $sortOf: 'alpha' } } }),
+      normalizeAnalyticsFilterTree({ where: { name: { $sortOf: 'alpha' } } }),
     ).toThrow(/Unsupported filter operator "\$sortOf"/);
   });
 
   it('a malformed $between throws rather than binding a half-open guess', () => {
-    expect(() => normalizeAnalyticsFilters({ where: { score: { $between: [10] } } })).toThrow(
+    expect(() => normalizeAnalyticsFilterTree({ where: { score: { $between: [10] } } })).toThrow(
       /two-element/,
     );
   });

packages/services/service-analytics/src/__tests__/native-sql-filter-logic-conformance.test.ts

@@ -0,0 +1,141 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * Filter logical-combinator conformance for the analytics raw-SQL strategy,
+ * executed against a real SQLite engine (`sql.js`, pure WASM).
+ *
+ * The cases come from `@objectstack/spec/data`, so this backend now stands
+ * beside `driver-sql`, `driver-memory`, `formula`'s `matchesFilterCondition`
+ * and `read-scope-sql` under one standard — see `filter-logic-conformance.ts`
+ * for why that standard exists (#3774).
+ *
+ * ## Why this consumer arrives late, and what it proves
+ *
+ * The analytics strategies could not have passed this table before: their
+ * normalizer produced a flat ARRAY, which cannot carry a disjunction, so an
+ * author's `{$or: […]}` was skipped outright and the compiled WHERE simply
+ * did not contain it. That is not "unsupported" — a missing predicate WIDENS
+ * the query, returning rows the author excluded, and it is invisible to a
+ * test that asserts the emitted SQL string (the SQL stays valid, just
+ * broader). Every case below whose filter carries a combinator fails against
+ * the pre-tree normalizer, most of them by returning the entire fixture.
+ *
+ * The read-scope compiler in this same package (`read-scope-sql.ts`) has
+ * always compiled the full tree, and is already a consumer of this table —
+ * so the package contained one correct implementation and one lossy one, for
+ * the same filter shape, with nothing holding them to each other. It does
+ * now.
+ *
+ * ## Why `sql.js` and not `better-sqlite3`
+ *
+ * Same reason as `read-scope-sql-conformance.test.ts`: the native binding is
+ * loadable only by the exact Node ABI it was built for and aborts the vitest
+ * worker on CI's Node, taking the file's cases silently with it. `sql.js` is
+ * the pure-WASM engine `driver-sql` itself falls back to.
+ */
+
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import { FILTER_LOGIC_CASES, FILTER_LOGIC_ROWS } from '@objectstack/spec/data';
+import type { Cube } from '@objectstack/spec/data';
+import type { AnalyticsQuery, StrategyContext } from '@objectstack/spec/contracts';
+
+import { NativeSQLStrategy } from '../strategies/native-sql-strategy.js';
+
+/**
+ * Dimension ids match the fixture's column names so the shared cases apply
+ * unchanged. `id` is selected and grouped by, which makes the result rows the
+ * matched row ids.
+ */
+const CUBE: Cube = {
+  name: 'logic',
+  title: 'Logic',
+  sql: 't',
+  measures: { total: { name: 'total', label: 'Total', type: 'count', sql: '*' } },
+  dimensions: Object.fromEntries(
+    ['id', 'a', 'b', 'c', 'owner', 'status', 'parent_object', 'parent_id'].map((n) => [
+      n,
+      { name: n, label: n, type: 'string', sql: n },
+    ]),
+  ),
+  public: false,
+} as unknown as Cube;
+
+/** Point sql.js at the `.wasm` shipped inside its own package (Node-safe). */
+async function locateWasm(): Promise<((file: string) => string) | undefined> {
+  try {
+    const { createRequire } = await import('node:module');
+    const require = createRequire(import.meta.url);
+    const pkgJsonPath = require.resolve('sql.js/package.json');
+    const { dirname, join } = await import('node:path');
+    const dir = dirname(pkgJsonPath);
+    return (file: string) => join(dir, 'dist', file);
+  } catch {
+    return undefined;
+  }
+}
+
+describe('NativeSQLStrategy — filter logic conformance', () => {
+  let db: any;
+  let ctx: StrategyContext;
+
+  beforeAll(async () => {
+    const mod: any = await import('sql.js');
+    const initSqlJs = mod.default ?? mod;
+    const locateFile = await locateWasm();
+    const SQL = await initSqlJs(locateFile ? { locateFile } : undefined);
+
+    db = new SQL.Database();
+    db.run(`
+      CREATE TABLE "t" (
+        "id" TEXT PRIMARY KEY,
+        "a" TEXT, "b" TEXT, "c" TEXT,
+        "owner" TEXT, "status" TEXT,
+        "parent_object" TEXT, "parent_id" TEXT
+      );
+    `);
+    const insert = db.prepare(
+      `INSERT INTO "t" ("id","a","b","c","owner","status","parent_object","parent_id")
+       VALUES (?,?,?,?,?,?,?,?)`,
+    );
+    for (const r of FILTER_LOGIC_ROWS) {
+      insert.run([r.id, r.a, r.b, r.c, r.owner, r.status, r.parent_object, r.parent_id]);
+    }
+    insert.free();
+
+    ctx = {
+      getCube: (name: string) => (name === 'logic' ? CUBE : undefined),
+      queryCapabilities: () => ({ nativeSql: true, objectqlAggregate: false, inMemory: false }),
+      // The strategy binds `$1`-style placeholders in ascending order, each
+      // pushed immediately before it is referenced, so a positional rewrite to
+      // SQLite's `?` preserves the pairing.
+      executeRawSql: async (_object: string, sql: string, params: unknown[]) => {
+        const stmt = db.prepare(sql.replace(/\$\d+/g, '?'));
+        stmt.bind(params as any[]);
+        const out: Record<string, unknown>[] = [];
+        while (stmt.step()) out.push(stmt.getAsObject());
+        stmt.free();
+        return out;
+      },
+    } as StrategyContext;
+  });
+
+  afterAll(() => {
+    db?.close();
+  });
+
+  for (const c of FILTER_LOGIC_CASES) {
+    it(c.name, async () => {
+      const result = await new NativeSQLStrategy().execute(
+        {
+          cube: 'logic',
+          measures: ['total'],
+          dimensions: ['id'],
+          where: c.filter,
+        } as AnalyticsQuery,
+        ctx,
+      );
+      const got = result.rows.map((r) => String(r.id)).sort((x, y) => x.localeCompare(y));
+      expect(got, c.note).toEqual(c.expected);
+    });
+  }
+});