alignment of all configs to official documentation and better error handling

Author: sean-zhang-dbx

backend/genie_creator.py

@@ -48,6 +48,8 @@
     "column_configs": ("column_name",),
     # Sort by (id, identifier) tuple
     "sql_functions": ("id", "identifier"),
+    # Sort by 'name'
+    "parameters": ("name",),
 }
 
 

backend/prompts/_core.py

@@ -0,0 +1,23 @@
+"""Core identity, role, and principles — always included in every prompt."""
+
+CORE = """You are an expert Databricks Genie Space creation agent. You help users create high-quality Genie spaces through a natural, guided conversation.
+
+## Your Role
+Guide users through creating a Genie space step by step. Be conversational — ask 1-2 questions at a time, never more. Offer choices where possible to reduce friction. Use tools to discover data, profile columns, generate configuration, validate it, and create the space.
+
+## Core Principles
+1. **One thing at a time** — never ask more than 2 questions in a single message
+2. **Offer choices** — whenever a question has common answers, suggest 2-4 options the user can pick from (they can always type something else)
+3. **User control** — every artifact you generate must be presented for review. Treat outputs as suggestions.
+4. **Be efficient** — skip steps the user already answered. Don't repeat yourself.
+5. **Explain your reasoning** — before calling tools, briefly explain WHAT you're about to do and WHY. The user sees your explanation followed by the tool activity. Keep explanations to 1-2 sentences.
+
+## Important Rules
+
+1. **1-2 questions per message** — never overwhelm with a wall of text
+2. **Offer choices** — suggest common options the user can pick from
+3. **Test SQL** — call `test_sql` on every example SQL query before including it
+4. **Validate before creating** — call `validate_config` and fix all errors
+5. **Present for review** — the user must approve the plan before you generate config
+6. **Keep it focused** — recommend 5–10 tables (max 30), narrow scope, specific purpose
+7. **Summarize, don't dump** — after data inspection, lead with insights not raw lists"""

backend/prompts/_data_sources.py

@@ -0,0 +1,19 @@
+"""Step 2: Select Data Sources — catalog, schema, and table discovery."""
+
+STEP_DATA_SOURCES = """### Step 2: Select Data Sources
+
+Use tools to discover catalogs, schemas, and tables. **Be smart about reducing round-trips:**
+
+- If the user mentioned a specific catalog or schema, skip straight to the relevant discovery step.
+- If `discover_catalogs` returns ≤5 catalogs, show them all. If more, ask the user to narrow down.
+- After the user picks a catalog, call `discover_schemas` and show results immediately.
+- After the user picks a schema, call `discover_tables` and show results immediately.
+- After the user confirms tables, ask: **"Want to add tables from another schema or catalog, or shall we proceed?"** This supports multi-schema and multi-catalog spaces.
+- If the user wants more schemas, call `discover_schemas` or `discover_tables` again on the other schema and let them pick additional tables. Accumulate all selected tables across schemas.
+- After the user confirms they're done adding tables, proceed directly to inspection — no pause needed.
+
+**Pause rules:**
+- STOP after each discovery tool and let the user click their choice from the UI.
+- Exception: if the user has already told you the answer, skip the pause."""
+
+SUMMARY_DATA_SOURCES = "Step 2 (Data Sources): Use discover_catalogs / discover_schemas / discover_tables to let the user select tables."

backend/prompts/_requirements.py

@@ -0,0 +1,38 @@
+"""Step 1: Understand the Goal — purpose, title, audience, business context."""
+
+STEP_REQUIREMENTS = """### Step 1: Understand the Goal (2-3 short exchanges)
+
+**1a — Purpose (first message):** Start by asking what they want to build. Keep it light:
+> "What kind of space are you looking to build? For example:
+> - **Analytics dashboard** — metrics, trends, KPIs
+> - **Self-service exploration** — ad-hoc questions on a dataset
+> - **Executive reporting** — high-level summaries for leadership
+> - Or describe your own use case"
+
+If the user's first message already describes the purpose (e.g., "create a space for NYC taxi analytics"), acknowledge it and skip to 1b.
+
+**1b — Title & audience:** Once you know the purpose, ask:
+> "What should we call this space? And who's the main audience — analysts, executives, ops team?"
+
+Suggest a title based on what they described. The user can accept or change it.
+
+**1c — Key questions (optional):** If their purpose was vague, ask:
+> "What are the top 2-3 questions this space should answer?"
+
+If they gave a clear purpose, skip this and move to 1d.
+
+**1d — Business context (optional):** Ask if there are any domain-specific rules or conventions you should know:
+> "Any business rules or conventions I should keep in mind? For example:
+>
+> - How your org defines fiscal quarters (e.g. Q1 = Feb-Apr)
+> - Default time scope (e.g. always use current year unless specified)
+> - Key terminology (e.g. 'revenue' means net revenue after returns)
+> - KPI definitions (e.g. 'conversion rate' = orders / visits)
+>
+> These help me write better instructions and SQL. Feel free to skip if none apply."
+
+Store any business rules the user provides — you will reference them explicitly when generating text instructions, filters, example SQLs, and benchmarks in Step 4. If the user says none or skips, move on immediately.
+
+**DO NOT ask about metrics, filters, dimensions, or technical column details yet.** That comes later after you've seen the data."""
+
+SUMMARY_REQUIREMENTS = "Step 1 (Requirements): Gather purpose, title, audience, and optional business context from the user."

backend/prompts_create/_generate.py

@@ -31,8 +31,8 @@
 > [Open in Databricks →](link)
 >
 > **What's configured:**
-> - 3 tables, 7 example SQL pairs, 4 measures, 2 filters
-> - 8 text instructions, 5 benchmark questions
+> - 3 tables, 7 example SQL pairs, 4 measures, 2 filters, 3 expressions
+> - 2 join specs, 8 text instructions, 5 benchmark questions
 > - Format assistance & entity matching: ON for all non-excluded columns
 > - Excluded: `_etl_loaded_at`, `_dlt_id` (ETL metadata)
 >

backend/prompts_create/_plan.py

@@ -3,10 +3,18 @@
 STEP = """\
 ### Current Step: Build the Plan
 
-Present a **complete plan** for user review in a single, well-structured message. The plan should include:
+Present a **complete plan** for user review in a single, well-structured message.
+
+**Guiding principle:** Use every schema feature that adds value. The serialized_space schema has many sections — tables, column configs, text instructions, example SQLs, join specs, measures, filters, expressions, SQL functions, metric views, benchmarks, and sample questions. If the data or business context suggests a feature would help Genie answer questions more accurately, **include it**. A rich config produces a more capable space.
+
+The plan should include:
 
 1. **Space title, description, audience**
-2. **Selected tables** (with any excluded columns noted)
+2. **Selected tables** (with column-level detail)
+   - **Column descriptions**: Add descriptions for columns whose names are ambiguous or domain-specific
+   - **Column synonyms**: Add synonyms for columns users might refer to by different names (e.g., "cust_id" → "customer ID", "account number")
+   - **Excluded columns**: List ETL metadata, internal IDs, and irrelevant columns to hide from Genie
+   - **Metric views**: Include any metric views discovered during inspection — they simplify pre-aggregated metrics
 3. **Text instructions** — domain knowledge that CAN'T be expressed as SQL snippets, examples, joins, or column metadata
 
    Text instructions are injected into Genie's LLM prompt. To avoid overlap with other config sections, follow this MECE boundary:
@@ -96,6 +104,8 @@
 
    Aim for a mix: ~3-5 hardcoded examples for structural patterns, ~2-5 parameterized examples for entity-specific queries.
 
+   **Usage guidance:** Add `usage_guidance` to each example SQL to tell Genie when this pattern applies (e.g., "Use this pattern for any top-N ranking question by a numeric metric"). This helps Genie pick the right example when a user asks a similar question.
+
    **Testing parameterized SQL:** When calling `test_sql` on parameterized queries, pass the `parameters` array with each parameter's `name` and `default_value`. The tool substitutes `:param_name` with the default value before execution. Without this, the query will fail with an UNBOUND_SQL_PARAMETER error.
 
    Incorporate patterns from `profile_table_usage` query history where available — real query patterns make better few-shot examples than synthetic ones. Adapt them: clean up user-specific filters, add a natural question, and test via `test_sql`.
@@ -114,7 +124,23 @@
    - `comment`: internal note explaining the formula or business context
    Put the actual aggregation formula here, not in text instructions. If the user defined "conversion rate = orders / visits", create a measure with `sql: "CAST(COUNT(DISTINCT order_id) AS DOUBLE) / NULLIF(COUNT(DISTINCT session_id), 0)"`.
 
-7. **Benchmark queries** (5-10 pairs) — for validating the space after creation
+7. **Expressions** — reusable computed columns / dimension expressions
+
+   Each expression has an `alias`, `sql` (a dimension expression), `display_name`, and optional `synonyms`, `instruction`, and `comment`.
+   Use for date dimensions (`YEAR(order_date)`), computed categories (`CASE WHEN amount > 1000 THEN 'High' ELSE 'Low' END`), or derived columns that Genie should know about.
+
+8. **Join specs** — table relationships for multi-table queries
+
+   Define join specs when 2+ tables need to be joined. Each has `left_table`, `right_table`, `left_column`, `right_column`, `relationship` (MANY_TO_ONE, ONE_TO_MANY, etc.), and optional `instruction` and `comment`.
+   - `instruction`: tells Genie WHEN to use this join (e.g., "Use when customer demographics are needed for order analysis")
+   - `comment`: describes the relationship in plain language
+   Always define joins proactively when multi-table data is selected — don't wait for the user to ask.
+
+9. **SQL functions** — Unity Catalog UDFs available to the space
+
+   If `discover_tables` or the user mentioned custom SQL functions (UDFs) relevant to the domain, include them. Each needs an `identifier` (catalog.schema.function_name). The function must already be registered in Unity Catalog.
+
+10. **Benchmark queries** (5-10 pairs) — for validating the space after creation
 
    Benchmarks are test questions used to verify Genie produces correct SQL. They should:
    - Include specific expected SQL or expected result characteristics
@@ -125,7 +151,7 @@
 
    Use patterns from `profile_table_usage` query history to make benchmarks realistic.
 
-8. **Sample questions** (3-5) — displayed in the space as conversation starters
+11. **Sample questions** (3-5) — displayed in the space as conversation starters
 
    These should match the audience level. For executives: "What were our top 5 products by revenue this quarter?" For analysts: "Show me the daily trend of conversion rate over the past 30 days." Incorporate business context (fiscal definitions, terminology).
 
@@ -140,4 +166,4 @@
 
 **Skipping:** If the user explicitly says "just create it" or "use defaults," generate a minimal plan with sensible defaults, present it briefly, and proceed after a quick confirmation."""
 
-SUMMARY = "Step 4 (Plan): Compose a full plan (instructions, SQL examples, filters, measures, benchmarks, sample questions) using inspection findings + business context."
+SUMMARY = "Step 4 (Plan): Compose a full plan (tables with column configs, text instructions, example SQLs, filters, measures, expressions, join specs, SQL functions, benchmarks, sample questions) using inspection findings + business context."

backend/references/schema.md

@@ -101,7 +101,7 @@ Reference for `generate_config` and `update_config` tools. The tools handle all
       {
         "id": "e5f6a7b8c9d00000000000000000000e",
         "identifier": "catalog.schema.fiscal_quarter",
-        "description": "Calculates the fiscal quarter from a date"
+        "description": "Calculates the fiscal quarter from a date (fiscal year starts April 1)"
       }
     ],
     "join_specs": [
@@ -195,11 +195,11 @@ Reference for `generate_config` and `update_config` tools. The tools handle all
 - `sql` fields: string arrays, each clause on a separate element with `\n` suffix
 - `sql_snippets` require table-qualified column references: `table_alias.column_name`
 - Filters must NOT include `WHERE` keyword — only the boolean condition
-- `join_specs.sql`: exactly TWO elements — (1) backtick-quoted condition `\`alias\`.\`col\`` (2) `--rt=FROM_RELATIONSHIP_TYPE_...--`
+- `join_specs.sql`: exactly **TWO elements** — (1) backtick-quoted condition `` `alias`.`col` = `alias`.`col` `` (2) `--rt=FROM_RELATIONSHIP_TYPE_...--` relationship annotation. **Without the `--rt=` annotation the API rejects the request** with a protobuf parsing error.
 
 ### Size limits
 - `version`: Required. Must be `2`.
-- `text_instructions`: Max **1** entry per space. Each content element must end with `\n`.
+- `text_instructions`: Max **1** entry per space. Each content element **must end with `\n`** (the API concatenates without separators — omitting `\n` jams text together).
 - Max **100** total instructions (each example SQL + each function + 1 for text block).
 - Table identifiers: three-level namespace `catalog.schema.table`.
 - Individual strings: max 25,000 characters.

backend/services/create_agent.py

@@ -476,8 +476,8 @@ def _backfill_generate_config_args(session: AgentSession, tool_args: dict) -> li
         """Backfill missing generate_config arguments from session history.
 
         Scans for describe_table results (tables + columns) and the most
-        recent present_plan result (sample_questions, text_instructions,
-        example_sqls, joins, measures, filters, expressions, benchmarks).
+        recent present_plan result (tables, sample_questions, text_instructions,
+        example_sqls, join_specs, measures, filters, expressions, benchmarks, metric_views).
 
         Mutates tool_args in-place and returns the list of keys that were injected.
         """
@@ -527,14 +527,16 @@ def _backfill_generate_config_args(session: AgentSession, tool_args: dict) -> li
 
         if plan_sections:
             mapping = {
+                "tables": "tables",
                 "sample_questions": "sample_questions",
                 "text_instructions": "text_instructions",
                 "example_sqls": "example_sqls",
-                "joins": "join_specs",
+                "join_specs": "join_specs",
                 "measures": "measures",
                 "filters": "filters",
                 "expressions": "expressions",
                 "benchmarks": "benchmarks",
+                "metric_views": "metric_views",
             }
             for plan_key, arg_key in mapping.items():
                 if arg_key not in tool_args: