Merge pull request #28 from nadeem4/feat/deterministic-aggregation

Feat/deterministic aggregation

Author: nadeem4

.gitignore

@@ -26,4 +26,6 @@ site
 
 data
 
-last_reasoning.json
+last_reasoning.json
+
+artifacts/

audit/remediation_plan.md

@@ -6,3 +6,9 @@ default:
   model: gpt-5.2
   temperature: 0.0
   api_key: ${env:OPENAI_API_KEY}
+agents:
+  indexing_enrichment:
+    provider: openai
+    model: gpt-5.2
+    temperature: 0.0
+    api_key: ${env:OPENAI_API_KEY}

audit/remediation_plan_observability.md

@@ -3,8 +3,8 @@ default:
   provider: openai
   model: gpt-5.2
 agents:
-  intent_validator:
+  indexing_enrichment:
     provider: openai
-    model: gpt-4o-mini
+    model: gpt-5.2
     temperature: 0.0
 

configs/llm.demo.yaml

@@ -9,7 +9,7 @@ There are two primary ways to build an adapter. Choose the one that fits your ta
 | If you are checking... | Use... | Reference |
 | :--- | :--- | :--- |
 | A standard SQL Database (Postgres, Oracle, Snowflake) | `nl2sql-adapter-sqlalchemy` | **[SQLAlchemy Adapter Reference](sqlalchemy.md)** |
-| A NoSQL DB, REST API, or custom driver | `nl2sql-adapter-sdk` | **[Adapter SDK Reference](sdk.md)** |
+| A NoSQL DB, REST API, or custom driver | Core adapter protocol | **[Adapter Interface Reference](sdk.md)** |
 
 ## Option 1: The "Fast Lane" (SQLAlchemy)
 
@@ -35,11 +35,11 @@ class PostgresAdapter(BaseSQLAlchemyAdapter):
 
 > See the **[SQLAlchemy Adapter Reference](sqlalchemy.md)** for full API details.
 
-## Option 2: The "Custom" Path (SDK)
+## Option 2: The "Custom" Path (Protocol)
 
-If you need to connect to something else (e.g., ElasticSearch, a CRM API, or a raw SQL driver), you must implement the raw interface.
+If you need to connect to something else (e.g., ElasticSearch, a CRM API, or a raw SQL driver), implement the core adapter protocol.
 
-**Implement `DatasourceAdapter`**. You must manually handle:
+**Implement `DatasourceAdapterProtocol`**. You must manually handle:
 
 * Fetching and normalizing schema metadata.
 * Executing queries and formatting results.
@@ -48,25 +48,29 @@ If you need to connect to something else (e.g., ElasticSearch, a CRM API, or a r
 ### Example
 
 ```python
-from nl2sql_adapter_sdk import DatasourceAdapter
+from nl2sql.datasources.protocols import DatasourceAdapterProtocol
+from nl2sql_adapter_sdk.contracts import AdapterRequest, ResultFrame
+from nl2sql_adapter_sdk.capabilities import DatasourceCapability
 
-class MyRestAdapter(DatasourceAdapter):
-    def fetch_schema(self) -> SchemaMetadata:
+class MyRestAdapter(DatasourceAdapterProtocol):
+    def capabilities(self):
+        return {DatasourceCapability.SUPPORTS_REST}
+
+    def fetch_schema_snapshot(self):
         # call API, return schema
         pass
 
-    def execute(self, query: str) -> QueryResult:
-        # run query, return rows
+    def execute(self, request: AdapterRequest) -> ResultFrame:
+        # run request, return rows
         pass
 ```
 
-> See the **[Adapter SDK Reference](sdk.md)** for the mandatory method signatures and compliance testing guide.
+> See the **[Adapter Interface Reference](sdk.md)** for the method signatures and compliance guide.
 
 ## Compliance Testing
 
-Regardless of which path you choose, your adapter **MUST** pass the compliance test suite to ensuring it handles types and errors correctly.
+Regardless of which path you choose, your adapter **MUST** pass the compliance test suite to ensure it handles types and errors correctly.
 
 ```python
-from nl2sql_adapter_sdk.testing import BaseAdapterTest
-# ... see SDK Reference for test setup
+# See Adapter Interface Reference for test setup
 ```

configs/llm.yaml

@@ -1,82 +1,64 @@
-# Adapter SDK Reference
+# Adapter Interface Reference
 
-The **Adapter SDK** (`nl2sql-adapter-sdk`) defines the core contract that all datasources must implement.
+The adapter contract lives in the adapter SDK:
 
-## Interface: `DatasourceAdapter`
+- `nl2sql.datasources.protocols.DatasourceAdapterProtocol`
+- `nl2sql_adapter_sdk.contracts.AdapterRequest`
+- `nl2sql_adapter_sdk.contracts.ResultFrame`
 
-All adapters must inherit from `nl2sql_adapter_sdk.interfaces.DatasourceAdapter`.
+## Interface: `DatasourceAdapterProtocol`
+
+Adapters expose a capability-driven interface:
 
 ```python
-from nl2sql_adapter_sdk import DatasourceAdapter
+from nl2sql.datasources.protocols import DatasourceAdapterProtocol
+from nl2sql_adapter_sdk.contracts import AdapterRequest, ResultFrame
+from nl2sql_adapter_sdk.capabilities import DatasourceCapability
 ```
 
 ### Mandatory Properties
 
 | Property | Type | Description |
 | :--- | :--- | :--- |
 | `datasource_id` | `str` | Unique identifier (e.g., "production_db"). |
-| `row_limit` | `int` | **Safety Breaker**. Must return a safe limit (e.g., 1000) to prevent OOM errors. |
-| `max_bytes` | `int` | **Safety Breaker**. Recommended limit for network payloads. |
+| `datasource_engine_type` | `str` | Engine type string (e.g., `postgres`, `rest`). |
+| `row_limit` | `int` | Safety breaker (limit returned rows). |
+| `max_bytes` | `int` | Safety breaker (limit payload size). |
 
 ### Mandatory Methods
 
-#### `fetch_schema()`
+#### `capabilities() -> set[DatasourceCapability]`
 
-Returns `SchemaMetadata`.
+Declares supported capabilities (e.g., `supports_sql`, `supports_rest`).
 
-* **Returns**: `SchemaMetadata` containing tables, columns, PKs, FKs.
-* **Requirement**: Must populate `col.statistics` (samples, min/max) for the validation logic to work effectively.
+#### `execute(request: AdapterRequest) -> ResultFrame`
 
-#### `execute(sql: str)`
+Executes a plan-specific request and returns a normalized `ResultFrame`.
 
-Executes a query and returns results.
+* **Args**: `AdapterRequest` with `plan_type` and `payload`
+* **Returns**: `ResultFrame` with `columns`, `rows`, `row_count`, and error metadata
 
-* **Args**: `sql` (str) - The SQL query to run.
-* **Returns**: `QueryResult` with `rows` (list of dicts) and `columns` (list of names).
+#### `fetch_schema_snapshot()`
 
-#### `dry_run(sql: str)`
+Required only if `supports_schema_introspection` is advertised.
 
-Validates SQL without executing it (or safely rolling back).
+### Optional Methods (SQL adapters)
 
-* **Args**: `sql` (str)
-* **Returns**: `DryRunResult(is_valid=bool, error_message=str)`
+#### `dry_run(sql: str)`
 
-### Optional Methods
+Validates SQL without executing it (or safely rolling back).
 
 #### `explain(sql: str)`
 
 Returns the execution plan.
 
-* **Returns**: `QueryPlan(plan_text=str)`
-
 #### `cost_estimate(sql: str)`
 
 Returns cost/row estimates for the Physical Validator.
 
-* **Returns**: `CostEstimate(estimated_cost=float, estimated_rows=int)`
-
 ---
 
 ## Compliance Testing
 
-The SDK provides a compliance test suite. **All Adapters MUST pass this suite.**
-
-It verifies:
-
-* Schema Introspection (PKs/FKs detected?)
-* Type Mapping (Date -> Python Date, Numeric -> Python Float)
-* Error Handling (Bad SQL -> AdapterError)
-
-### Running Tests
-
-```python
-# tests/test_my_adapter.py
-from nl2sql_adapter_sdk.testing import BaseAdapterTest
-from my_adapter import MyAdapter
-import pytest
-
-class TestMyAdapter(BaseAdapterTest):
-    @pytest.fixture
-    def adapter(self):
-        return MyAdapter(...)
-```
+All adapters should pass the compliance test suite (schema introspection, type mapping,
+error handling, and result contract validation).

docs/adapters/development.md

@@ -4,7 +4,7 @@
 
 ### The Core Issue: "Blast Radius" & Reliability
 
-Currently, the NL2SQL Agent executes SQL queries via `ExecutorNode`, performs dry-runs via `PhysicalValidatorNode`, and indexes schemas via `OrchestratorVectorStore` **in-process**.
+Currently, the NL2SQL Agent executes SQL queries via `ExecutorNode`, performs dry-runs via `PhysicalValidatorNode`, and indexes schemas via `VectorStore` **in-process**.
 
 This couples the stability of the Agent to the stability of the underlying SQL Drivers and the Database itself.
 
@@ -34,7 +34,7 @@ Spawn worker processes on the same machine (Process Boundary Isolation).
 
 * **Mechanism**:
   * **Execution Pool**: A low-latency pool for `ExecutorNode` and `PhysicalValidatorNode` (User-facing queries).
-  * **Indexing Pool**: A separate, lower-priority pool for `OrchestratorVectorStore` (Background tasks).
+  * **Indexing Pool**: A separate, lower-priority pool for `VectorStore` (Background tasks).
 * **Controls**:
   * `max_workers`: Hard cap on concurrency to prevent OOM.
   * `initializer`: Setup persistent global `SQLAlchemy Engine` for connection pooling.