Add Yardstick upstream parity CI (#187)

Author: nicosuave

.github/workflows/yardstick-upstream.yml

@@ -0,0 +1,53 @@
+name: Yardstick Upstream Parity
+
+on:
+  workflow_dispatch:
+    inputs:
+      yardstick_ref:
+        description: "Yardstick ref to test against"
+        required: true
+        default: "main"
+  schedule:
+    - cron: "17 10 * * *"
+  pull_request:
+    paths:
+      - ".github/workflows/yardstick-upstream.yml"
+      - "docs/compatibility/yardstick.md"
+      - "sidemantic/adapters/yardstick.py"
+      - "sidemantic/sql/aggregation_detection.py"
+      - "sidemantic/sql/query_rewriter.py"
+      - "tests/queries/test_yardstick_measures_replay.py"
+      - "tests/queries/test_yardstick_query_rewriter.py"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: yardstick-upstream-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  upstream-parity:
+    name: Live upstream SQL replay
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    env:
+      SIDEMANTIC_YARDSTICK_UPSTREAM_TESTS: "1"
+      YARDSTICK_UPSTREAM_REF: ${{ github.event.inputs.yardstick_ref || 'main' }}
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+
+      - name: Set up Python
+        run: uv python install 3.12
+
+      - name: Install dependencies
+        run: uv sync --extra dev
+
+      - name: Replay upstream Yardstick tests
+        run: uv run pytest -q tests/queries/test_yardstick_measures_replay.py -m yardstick_upstream

docs/compatibility/yardstick.md

@@ -1,6 +1,6 @@
 # Yardstick Compatibility
 
-Sidemantic's Yardstick adapter parses SQL files containing `CREATE VIEW` statements that use the `AS MEASURE` syntax from Julian Hyde's ["Measures in SQL" proposal](https://arxiv.org/abs/2307.14009). It maps Yardstick concepts to Sidemantic's semantic model (Model, Dimension, Metric) and supports the `SEMANTIC SELECT`, `AGGREGATE()`, and `AT` query modifiers for measure-aware SQL queries.
+Sidemantic's Yardstick adapter parses SQL files containing `CREATE VIEW` statements that use the `AS MEASURE` syntax from Julian Hyde's ["Measures in SQL" proposal](https://arxiv.org/abs/2307.14009). It maps Yardstick concepts to Sidemantic's semantic model (Model, Dimension, Metric) and supports `SEMANTIC SELECT`, optional-prefix `AGGREGATE()`, and `AT` query modifiers for measure-aware SQL queries.
 
 Features are marked **supported**, **partial support**, or **unsupported**. Partial support entries include notes explaining the limitation.
 
@@ -110,23 +110,27 @@ Derived measure detection works by scanning the expression's column references a
 | `MODE(expr) AS MEASURE name` | Supported (stored as raw SQL expression metric with `agg=None`) |
 | `PERCENTILE_CONT(n) WITHIN GROUP (ORDER BY expr) AS MEASURE name` | Supported (stored as raw SQL expression metric) |
 | `CASE WHEN AGG(...) THEN ... END AS MEASURE name` | Supported (detected as having aggregate semantics; stored as raw SQL expression metric) |
-| Other aggregate functions not in the standard list | Supported (full expression preserved as `Metric.sql`) |
+| `PRODUCT(expr)`, `ENTROPY(expr)`, `KURTOSIS(expr)`, `SKEWNESS(expr)`, `LIST(expr)`, and related DuckDB aggregate functions | Supported (stored as raw SQL expression metrics with aggregate semantics) |
+| Other aggregate functions not in the standard list | Supported when sqlglot identifies them as aggregates; otherwise preserved as raw SQL only when aggregate semantics can be detected |
 
-When a measure expression contains aggregate functions (detected by walking the AST for `AggFunc` nodes or known anonymous aggregations like `mode`) but doesn't match a simple aggregation pattern, the full expression is preserved as-is for query-time evaluation.
+When a measure expression contains aggregate functions (detected by walking the AST for `AggFunc` nodes or known anonymous aggregations like `mode`, `product`, and `entropy`) but doesn't match a simple aggregation pattern, the full expression is preserved as-is for query-time evaluation.
 
 ---
 
 ## Query Semantics
 
-The Yardstick adapter works in tandem with Sidemantic's query rewriter to support the `SEMANTIC SELECT`, `AGGREGATE()`, and `AT` modifiers described in the Measures in SQL proposal.
+The Yardstick adapter works in tandem with Sidemantic's query rewriter to support `SEMANTIC SELECT`, optional-prefix `AGGREGATE()`, and `AT` modifiers described in the Measures in SQL proposal.
 
 ### SEMANTIC Prefix
 
 | Feature | Status |
 |---------|--------|
 | `SEMANTIC SELECT ...` | Supported (enables measure-aware query rewriting) |
 | `SEMANTIC WITH ... SELECT ...` | Supported (CTEs within semantic queries) |
-| Implicit measure detection without `SEMANTIC` prefix | Supported (queries containing `AT` modifiers or curly-brace measure references are auto-detected) |
+| `SELECT ... AGGREGATE(measure)` without `SEMANTIC` prefix | Supported |
+| `CREATE TABLE ... AS SELECT ... AGGREGATE(...)` | Supported |
+| `INSERT INTO ... SELECT ... AGGREGATE(...)` | Supported |
+| Implicit measure detection without `SEMANTIC` prefix | Supported (queries containing `AGGREGATE()`, `AT` modifiers, or curly-brace measure references are auto-detected) |
 
 ### AGGREGATE() Function
 
@@ -139,7 +143,36 @@ The Yardstick adapter works in tandem with Sidemantic's query rewriter to suppor
 | `AGGREGATE()` in arithmetic expressions (`2 * AGGREGATE(revenue)`) | Supported |
 | `AGGREGATE(measure) / AGGREGATE(measure) AT (...)` | Supported (each AGGREGATE evaluated independently) |
 | Scalar `AGGREGATE()` without GROUP BY | Supported (produces a single grand-total row) |
-| `AGGREGATE()` without `SEMANTIC` prefix and without `AT` | Error: raises `ValueError` requiring the `SEMANTIC` prefix |
+| `AGGREGATE()` without `SEMANTIC` prefix and without `AT` | Supported |
+| Native DuckDB `aggregate(list, 'function')` | Supported (falls through to DuckDB; not treated as Yardstick syntax) |
+
+### Upstream Parity Tests
+
+The default test suite replays a vendored Yardstick `measures.test` fixture for stable CI coverage. To check against the live upstream Yardstick repository without copying fixtures into Sidemantic, run:
+
+```bash
+SIDEMANTIC_YARDSTICK_UPSTREAM_TESTS=1 uv run pytest -q tests/queries/test_yardstick_measures_replay.py -m yardstick_upstream
+```
+
+The same command runs in the `Yardstick Upstream Parity` GitHub Actions workflow. That workflow runs nightly, can be triggered manually with a Yardstick ref override, and runs on pull requests that touch Yardstick-specific code or tests.
+
+The live replay fetches `https://github.com/sidequery/yardstick.git` at `main` by default, checks all upstream `test/sql/*.test` files, and validates both facets:
+
+| Facet | Coverage |
+|-------|----------|
+| Model/metric definitions | Parses every upstream `CREATE VIEW ... AS MEASURE` statement and asserts model name, source table/base SQL, primary key, Yardstick metadata, dimension SQL/type/granularity, and metric `agg`/`sql`/`filters`/`type` |
+| Query execution | Replays every upstream query block against Sidemantic's Yardstick rewriter and compares result rows |
+
+The live definition check covers the `CREATE VIEW ... AS MEASURE` definitions used by Yardstick's SQL tests. Sidemantic's native SQL definition parser owns `MODEL(...)`, `METRIC(...)`, and `DIMENSION(...)` files separately from the Yardstick adapter; the live upstream replay does not treat Yardstick's top-level `yardstick_definitions.sql` helper file as part of the SQL-test corpus.
+
+Optional environment variables:
+
+| Variable | Purpose |
+|----------|---------|
+| `YARDSTICK_UPSTREAM_PATH` | Use an existing local Yardstick checkout instead of fetching |
+| `YARDSTICK_UPSTREAM_REPO` | Override the upstream Git URL |
+| `YARDSTICK_UPSTREAM_REF` | Override the ref fetched from upstream |
+| `YARDSTICK_UPSTREAM_CACHE_DIR` | Override the temporary checkout path |
 
 ### AT Modifiers
 

pyproject.toml

@@ -169,6 +169,7 @@ testpaths = ["tests"]
 pythonpath = ["."]
 markers = [
     "integration: marks tests as integration tests requiring external services (deselect with '-m \"not integration\"')",
+    "yardstick_upstream: opt-in tests that fetch and replay live upstream Yardstick SQL tests",
 ]
 addopts = "-m 'not integration' --cov=sidemantic --cov-report=term-missing"  # Skip integration tests by default and show coverage
 

sidemantic/adapters/yardstick.py

@@ -102,7 +102,15 @@ def __init__(self, dialect: str = "duckdb"):
         exp.Variance: "variance",
         exp.VariancePop: "variance_pop",
     }
-    _ANONYMOUS_AGGREGATIONS: set[str] = {"mode"}
+    _ANONYMOUS_AGGREGATIONS: set[str] = {
+        "entropy",
+        "geometric_mean",
+        "kurtosis",
+        "mode",
+        "product",
+        "skewness",
+        "weighted_avg",
+    }
 
     def parse(self, source: str | Path) -> SemanticGraph:
         """Parse Yardstick SQL files into a semantic graph."""
@@ -344,6 +352,8 @@ def _has_aggregate_semantics(self, expression: exp.Expression) -> bool:
             return True
 
         for node in expression.walk():
+            if isinstance(node, exp.List):
+                return True
             if isinstance(node, exp.Anonymous) and (node.name or "").lower() in self._ANONYMOUS_AGGREGATIONS:
                 return True
         return False

sidemantic/sql/aggregation_detection.py

@@ -10,11 +10,20 @@
 # SQLGlot treats some engine-specific aggregate functions as Anonymous.
 # Keep this focused on known aggregate forms we need to support.
 _ANONYMOUS_AGGREGATE_FUNCTIONS = {
+    "entropy",
+    "geometric_mean",
+    "kurtosis",
     "mode",
+    "product",
+    "skewness",
+    "weighted_avg",
 }
 
 _AGGREGATE_REGEX = re.compile(
-    r"\b(sum|count|avg|min|max|median|stddev|stddev_pop|variance|variance_pop|mode|quantile|percentile)\s*\(",
+    r"\b("
+    r"sum|count|avg|min|max|median|stddev|stddev_pop|variance|variance_pop|mode|"
+    r"quantile|percentile|product|entropy|kurtosis|skewness|geometric_mean|weighted_avg|list"
+    r")\s*\(",
     re.IGNORECASE,
 )
 
@@ -25,6 +34,8 @@ def expression_has_aggregate(expression: exp.Expression) -> bool:
         return True
 
     for node in expression.walk():
+        if isinstance(node, exp.List):
+            return True
         if isinstance(node, exp.Anonymous) and (node.name or "").lower() in _ANONYMOUS_AGGREGATE_FUNCTIONS:
             return True