feat: add sql_snippets, join_specs, and benchmark accuracy improvements

- Add genie_sql_filters, genie_sql_measures, genie_sql_expressions, and
  genie_join_specs to serialized_space for better Genie SQL generation
- Update ABAC_PROMPT.md with unambiguous benchmark rules, business default
  instructions, and domain-adaptive generation guidance
- Use two-step create-then-patch for Genie Space (CREATE endpoint doesn't
  support sql_snippets/join_specs)
- Restructure TUNING.md to prioritize Genie accuracy review checklist
- Increase Databricks FMAPI timeout to 600s for larger prompt responses
- Remove abac.auto.tfvars from make setup (generated by make generate)

Made-with: Cursor

Author: louiscsq

uc-quickstart/utils/genie/aws/ABAC_PROMPT.md

@@ -228,40 +228,111 @@ Violating any of these causes validation failures. Double-check consistency acro
 6. Select masking functions from the library above (or create new ones)
 7. Generate both output files. For entity names in tag_assignments, always use **fully qualified** names (`catalog.schema.table` or `catalog.schema.table.column`). For function_name in fgac_policies, use relative names only (e.g. `mask_pii`). Every fgac_policy MUST include `catalog`, `function_catalog`, and `function_schema`. **CRITICAL**: set `function_schema` to the schema where the tagged columns actually live — do NOT default all policies to the first schema. In `masking_functions.sql`, group the `CREATE FUNCTION` statements by schema with separate `USE SCHEMA` blocks. Only create each function in the schema where it is needed
 8. Every `match_condition` and `when_condition` MUST only use `hasTagValue()` and/or `hasTag()` — no other functions or operators
-9. Generate Genie Space config — all five fields below. Tailor everything to the user's actual tables, domain, and business context:
-   - `genie_space_title` — a concise, descriptive title (e.g., "Financial Compliance Analytics", "Clinical Data Explorer")
+9. Generate Genie Space config — all nine fields below. **Derive everything from the user's actual tables, columns, and domain** — do NOT copy the finance/healthcare examples below if the user's data is from a different industry. Adapt terminology, metrics, filters, and joins to whatever vertical the tables belong to (retail, manufacturing, telecom, education, logistics, etc.):
+   - `genie_space_title` — a concise, descriptive title reflecting the user's domain (e.g., finance → "Financial Compliance Analytics", retail → "Retail Sales & Inventory Explorer", telecom → "Network Performance Dashboard")
    - `genie_space_description` — 1–2 sentence summary of what the space covers and who it's for
-   - `genie_sample_questions` — 5–10 natural-language questions a business user would ask (shown as conversation starters in the UI)
-   - `genie_instructions` — domain-specific guidance for the Genie LLM (e.g., how to calculate metrics, date conventions, terminology, masking behaviour awareness)
-   - `genie_benchmarks` — 3–5 benchmark questions with ground-truth SQL for evaluating accuracy
+   - `genie_sample_questions` — 5–10 natural-language questions a business user in that domain would ask (shown as conversation starters in the UI). Must reference the user's actual table/column names.
+   - `genie_instructions` — domain-specific guidance for the Genie LLM. **Must include business defaults** — look at status/state columns in the user's tables and define which values are the default filter (e.g., if a table has `OrderStatus` with values like 'Fulfilled'/'Cancelled'/'Pending', instruct: "default to fulfilled orders"). Also cover date conventions, metric calculations, terminology, and masking awareness relevant to the user's domain.
+   - `genie_benchmarks` — 3–5 benchmark questions with ground-truth SQL. **Each question must be unambiguous and self-contained** — include explicit qualifiers so the question and SQL agree on scope (e.g., "What is the average risk score for active customers?" not "What is the average customer risk score?"). Avoid questions that could reasonably be interpreted with different WHERE clauses.
+   - `genie_sql_filters` — default WHERE clauses derived from the user's status/state columns (e.g., active records, completed transactions, open orders). Each filter has `sql`, `display_name`, `comment`, and `instruction`.
+   - `genie_sql_measures` — standard aggregate metrics derived from the user's numeric columns (e.g., sums, averages, counts that are meaningful in the domain). Each measure has `alias`, `sql`, `display_name`, `comment`, and `instruction`.
+   - `genie_sql_expressions` — computed dimensions derived from the user's date/category columns (e.g., year extraction, bucketing, status grouping). Each expression has `alias`, `sql`, `display_name`, `comment`, and `instruction`.
+   - `genie_join_specs` — relationships between the user's tables based on foreign key columns (look for matching ID columns like `CustomerID`, `OrderID`, `ProductID`). Each join has `left_table`, `left_alias`, `right_table`, `right_alias`, `sql`, `comment`, and `instruction`.
 
 ### Output Format — Genie Space Config (in `abac.auto.tfvars`)
 
-Include these variables alongside groups, tag_policies, etc.:
+Include these variables alongside groups, tag_policies, etc. The example below shows a **finance/healthcare** scenario — adapt all values to match the user's actual tables and industry:
 
 ```hcl
 genie_space_title       = "Financial & Clinical Analytics"
 genie_space_description = "Explore transaction data, patient encounters, and compliance metrics. Designed for analysts, compliance officers, and clinical staff."
 
 genie_sample_questions = [
   "What is the total revenue by region for last quarter?",
-  "Show the top 10 customers by transaction volume",
+  "Show the top 10 active customers by transaction volume",
   "Which accounts have been flagged for AML review?",
   "How many patient encounters occurred last month?",
-  "What is the average transaction amount by account type?",
+  "What is the average completed transaction amount by account type?",
 ]
 
-genie_instructions = "When calculating revenue, sum the Amount column. 'Last month' means the previous calendar month (not last 30 days). Round monetary values to 2 decimal places. Patient names are masked for non-clinical roles — queries about patient counts or encounter dates are always allowed."
+genie_instructions = "When asked about 'customers' without a status qualifier, default to active customers (CustomerStatus = 'Active'). When asked about 'transactions' without specifying status, default to completed transactions (TransactionStatus = 'Completed'). 'Last month' means the previous calendar month (not last 30 days). Round monetary values to 2 decimal places. Patient names are masked for non-clinical roles — queries about patient counts or encounter dates are always allowed."
 
 genie_benchmarks = [
   {
-    question = "What is the total transaction amount?"
-    sql      = "SELECT SUM(Amount) as total_amount FROM catalog.schema.transactions"
+    question = "What is the total amount of completed transactions?"
+    sql      = "SELECT SUM(Amount) as total_amount FROM catalog.schema.transactions WHERE TransactionStatus = 'Completed'"
   },
   {
-    question = "How many patients were seen last month?"
+    question = "How many patient encounters occurred last month?"
     sql      = "SELECT COUNT(*) FROM catalog.schema.encounters WHERE EncounterDate >= DATE_TRUNC('month', CURRENT_DATE - INTERVAL 1 MONTH) AND EncounterDate < DATE_TRUNC('month', CURRENT_DATE)"
   },
+  {
+    question = "What is the average risk score for active customers?"
+    sql      = "SELECT AVG(RiskScore) as avg_risk_score FROM catalog.schema.customers WHERE CustomerStatus = 'Active'"
+  },
+]
+
+genie_sql_filters = [
+  {
+    sql          = "customers.CustomerStatus = 'Active'"
+    display_name = "active customers"
+    comment      = "Only include customers with Active status"
+    instruction  = "Apply when the user asks about customers without specifying a status"
+  },
+  {
+    sql          = "transactions.TransactionStatus = 'Completed'"
+    display_name = "completed transactions"
+    comment      = "Only include completed transactions"
+    instruction  = "Apply when the user asks about transactions or amounts without specifying a status"
+  },
+]
+
+genie_sql_measures = [
+  {
+    alias        = "total_revenue"
+    sql          = "SUM(transactions.Amount)"
+    display_name = "total revenue"
+    comment      = "Sum of all transaction amounts"
+    instruction  = "Use for revenue, total amount, or sales calculations"
+  },
+  {
+    alias        = "avg_risk_score"
+    sql          = "AVG(customers.RiskScore)"
+    display_name = "average risk score"
+    comment      = "Average AML risk score across customers"
+    instruction  = "Use when asked about risk scores or risk averages"
+  },
+]
+
+genie_sql_expressions = [
+  {
+    alias        = "transaction_year"
+    sql          = "YEAR(transactions.TransactionDate)"
+    display_name = "transaction year"
+    comment      = "Extracts year from transaction date"
+    instruction  = "Use for year-over-year analysis of transactions"
+  },
+]
+
+genie_join_specs = [
+  {
+    left_table  = "catalog.schema.accounts"
+    left_alias  = "accounts"
+    right_table = "catalog.schema.customers"
+    right_alias = "customers"
+    sql         = "accounts.CustomerID = customers.CustomerID"
+    comment     = "Join accounts to customers on CustomerID"
+    instruction = "Use when you need customer details for account queries"
+  },
+  {
+    left_table  = "catalog.schema.transactions"
+    left_alias  = "transactions"
+    right_table = "catalog.schema.accounts"
+    right_alias = "accounts"
+    sql         = "transactions.AccountID = accounts.AccountID"
+    comment     = "Join transactions to accounts on AccountID"
+    instruction = "Use when you need account or customer context for transactions"
+  },
 ]
 ```
 

uc-quickstart/utils/genie/aws/Makefile

@@ -19,12 +19,6 @@ setup: ## Copy example files and prompt for credentials
 	else \
 		echo "env.auto.tfvars already exists — skipping."; \
 	fi
-	@if [ ! -f abac.auto.tfvars ]; then \
-		cp abac.auto.tfvars.example abac.auto.tfvars; \
-		echo "Created abac.auto.tfvars — edit it with your ABAC config."; \
-	else \
-		echo "abac.auto.tfvars already exists — skipping."; \
-	fi
 	@mkdir -p ddl generated
 	@echo "Created ddl/ and generated/ directories."
 	@echo ""

uc-quickstart/utils/genie/aws/README.md

@@ -1,6 +1,6 @@
 # OneReady — Genie Onboarding Quickstart
 
-Get your workspace **OneReady** for Genie in Databricks One. An AI-powered Terraform quickstart that automates business-user onboarding — from ABAC governance and masking functions to a fully configured Genie Space with AI-generated sample questions, instructions, and benchmarks — all from three config files, no `.tf` editing required.
+Get your workspace **OneReady** for Genie in Databricks One. An AI-powered Terraform quickstart that automates business-user onboarding — from ABAC governance and masking functions to a fully configured Genie Space with AI-generated sample questions, instructions, benchmarks, SQL filters, measures, and join specs — all from three config files, no `.tf` editing required.
 
 ## What This Quickstart Automates
 
@@ -12,8 +12,11 @@ Get your workspace **OneReady** for Genie in Databricks One. An AI-powered Terra
 - **Masking functions** — Auto-deploy SQL UDFs to enforce column-level data masking (e.g., mask SSN, redact PII, hash emails).
 - **Genie Space** — Auto-create a new Genie Space from your tables, or bring an existing one. New spaces include AI-generated config:
   - **Sample questions** — Conversation starters tailored to your data domain
-  - **Instructions** — Domain-specific LLM guidance (metric definitions, date conventions, terminology)
-  - **Benchmarks** — Ground-truth question + SQL pairs for evaluating Genie accuracy
+  - **Instructions** — Domain-specific LLM guidance with business defaults (e.g., "customer" means active by default)
+  - **Benchmarks** — Unambiguous ground-truth question + SQL pairs for evaluating Genie accuracy
+  - **SQL filters** — Default WHERE clauses (e.g., active customers, completed transactions) that guide Genie's SQL generation
+  - **SQL measures & expressions** — Standard metrics (total revenue, avg risk score) and computed dimensions (transaction year)
+  - **Join specs** — Table relationships with join conditions so Genie knows how to combine tables
   - **Title & description** — Contextual naming based on your tables and domain
   - For existing spaces, set `genie_space_id` in `env.auto.tfvars` to apply `CAN_RUN` ACLs for all configured business groups
 - **SQL warehouse** — Auto-create a serverless warehouse or reuse an existing one.
@@ -64,6 +67,9 @@ Get your workspace **OneReady** for Genie in Databricks One. An AI-powered Terra
 │  │                         │  │  genie_sample_questions (5–10)    │   │
 │  │                         │  │  genie_instructions               │   │
 │  │                         │  │  genie_benchmarks (3–5 w/ SQL)    │   │
+│  │                         │  │  genie_sql_filters / measures     │   │
+│  │                         │  │  genie_sql_expressions            │   │
+│  │                         │  │  genie_join_specs                 │   │
 │  └────────────┬────────────┘  └─────────────────┬─────────────────┘   │
 └───────────────┼─────────────────────────────────┼─────────────────────┘
                 │             ▲  TUNE & VALIDATE  │
@@ -94,8 +100,10 @@ Get your workspace **OneReady** for Genie in Databricks One. An AI-powered Terra
 │  │ (auto-deploy UDFs) │  │ USE_CATALOG    │  │ • sample questions │   │
 │  │                    │  │ USE_SCHEMA     │  │ • instructions     │   │
 │  │ + SQL Warehouse    │  │ SELECT         │  │ • benchmarks       │   │
-│  │ (auto-created if   │  │                │  │ • CAN_RUN ACLs     │   │
-│  │  needed)           │  │                │  │   for all groups   │   │
+│  │ (auto-created if   │  │                │  │ • sql filters /    │   │
+│  │  needed)           │  │                │  │   measures / joins │   │
+│  │                    │  │                │  │ • CAN_RUN ACLs     │   │
+│  │                    │  │                │  │   for all groups   │   │
 │  └────────────────────┘  └────────────────┘  └────────────────────┘   │
 └───────────────────────────────────────────────────────────────────────┘
 ```
@@ -113,7 +121,7 @@ make validate-generated     # 3. (Optional) Tune generated/ files, validate afte
 make apply                  #    Validates → promotes → terraform apply
 ```
 
-That's it. `make apply` creates groups, tags, masking functions, FGAC policies, UC grants, and a Genie Space (with AI-generated sample questions, instructions, and benchmarks) — all in one command.
+That's it. `make apply` creates groups, tags, masking functions, FGAC policies, UC grants, and a Genie Space (with AI-generated sample questions, instructions, benchmarks, SQL filters/measures/expressions, and join specs) — all in one command.
 
 To tear everything down: `make destroy`.
 
@@ -166,16 +174,20 @@ Managed automatically based on `genie_space_id` in `env.auto.tfvars`:
 When `make generate` creates the ABAC config, it also generates Genie Space config in `abac.auto.tfvars`:
 
 
-| Variable                  | Purpose                                                                                 |
-| ------------------------- | --------------------------------------------------------------------------------------- |
-| `genie_space_title`       | AI-generated title for the Genie Space (e.g., "Financial Compliance Analytics")         |
-| `genie_space_description` | 1–2 sentence summary of the space's scope and audience                                  |
-| `genie_sample_questions`  | Natural-language questions shown as conversation starters in the Genie UI               |
-| `genie_instructions`      | Domain-specific guidance for the Genie LLM (metric definitions, date conventions, etc.) |
-| `genie_benchmarks`        | Ground-truth question + SQL pairs for evaluating Genie accuracy                         |
+| Variable                  | Purpose                                                                                                    |
+| ------------------------- | ---------------------------------------------------------------------------------------------------------- |
+| `genie_space_title`       | AI-generated title for the Genie Space (e.g., "Financial Compliance Analytics")                            |
+| `genie_space_description` | 1–2 sentence summary of the space's scope and audience                                                     |
+| `genie_sample_questions`  | Natural-language questions shown as conversation starters in the Genie UI                                  |
+| `genie_instructions`      | Domain-specific guidance including business defaults (e.g., "customer" = active by default)                |
+| `genie_benchmarks`        | Unambiguous ground-truth question + SQL pairs for evaluating Genie accuracy                                |
+| `genie_sql_filters`       | Default WHERE clauses (e.g., active customers, completed transactions) that guide Genie's SQL generation   |
+| `genie_sql_measures`      | Standard aggregate metrics (e.g., total revenue, average risk score)                                       |
+| `genie_sql_expressions`   | Computed dimensions (e.g., transaction year, age bucket)                                                   |
+| `genie_join_specs`        | Table relationships with join conditions (e.g., accounts to customers on CustomerID)                       |
 
 
-All five fields are included in the `serialized_space` when a new Genie Space is created. Review and tune them in `generated/abac.auto.tfvars` alongside the ABAC policies before applying.
+All nine fields are included in the `serialized_space` when a new Genie Space is created. Review and tune them in `generated/abac.auto.tfvars` alongside the ABAC policies before applying.
 
 ## Make Targets