docs(skills): adopt create_or_update_*_monitor tools and preview-then-confirm flow (#85)

## Summary

- Renames every `create_*_monitor_mac` reference in skill docs to the
new `create_or_update_{table,metric,validation,sql,comparison}_monitor`
tools introduced in
[ai-agent#998](monte-carlo-data/ai-agent#998).
- Documents the **two-call preview-then-confirm flow** that those tools
enforce: `dry_run=True` (the default) returns rendered MaC YAML in
`result.yaml`; `dry_run=False` actually creates/updates the monitor and
returns `result.monitor_uuid` + a deep link in `result.instructions`
(with `result.yaml=None` by design).
- Documents `monitor_uuid` for **update-in-place**, `is_draft` for draft
mode, and the parity params (`schedule_type`, `interval_minutes`,
`audiences`, `failure_audiences`, `notes`, `priority`, `tags`, plus
per-type extras like
`aggregate_time_sql`/`segment_sql`/`sensitivity`/`collection_lag_hours`
on metric,
`query_result_type`/`custom_sampling_sql`/`variable_definitions` on
SQL).
- Migrates the **tune-monitor** skill off the now-extended-only
direct-GraphQL `create_metric_monitor` / `create_custom_sql_monitor` /
`create_validation_monitor` tools (hidden from the public default
toolset) onto `create_or_update_*_monitor` with `monitor_uuid` for
in-place updates.
- Updates
`plugins/claude-code/evals/{monitoring-advisor,prevent}/live-evals-*.yaml`
and the eval README example to assert the new tool names in `must_call`
/ `must_not_call`.
- Bumps `monitoring-advisor` to **2.1.0** and `tune-monitor` to
**1.1.0**.

Plugin-level CHANGELOGs are left for the next release cut (no Unreleased
section convention in this repo).

## Test plan

- [ ] Spot-check the rendered skill docs to confirm tool names +
preview/confirm flow read coherently end-to-end.
- [ ] Run the `monitoring-advisor` live evals
(`plugins/claude-code/evals/monitoring-advisor/live-evals-{dev,prod}.yaml`)
against the updated dev MCP server — confirms the new tool names match
what the server actually exposes in the default toolset.
- [ ] Run the `prevent` live eval (`live-04-monitor-delegation`) and
confirm the turn-2 trace shows one of the new
`create_or_update_*_monitor` calls.
- [ ] Manually exercise the tune-monitor apply step against a sandbox
monitor: `dry_run=True` returns YAML, `dry_run=False` updates in place
and returns the deep link.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Author: santiagoaguiar

plugins/claude-code/evals/README.md

@@ -99,7 +99,7 @@ Edit `<skill>/live-evals-<env>.yaml`. All cases use the `turns` format:
     - prompt: "Your prompt here"
       criteria:
         must_call: [get_warehouses]
-        must_not_call: [create_table_monitor_mac]
+        must_not_call: [create_or_update_table_monitor]
         output_must_not_contain: ["MCON++"]
   criteria:
     judge_rubric: |

plugins/claude-code/evals/monitoring-advisor/live-evals-dev.yaml

@@ -4,7 +4,7 @@ cases:
       - prompt: "What warehouses do I have access to?"
         criteria:
           must_call: [get_warehouses]
-          must_not_call: [create_table_monitor_mac, create_metric_monitor_mac]
+          must_not_call: [create_or_update_table_monitor, create_or_update_metric_monitor]
           output_must_not_contain: ["MCON++"]
     criteria:
       judge_rubric: |

plugins/claude-code/evals/monitoring-advisor/live-evals-prod.yaml

@@ -4,7 +4,7 @@ cases:
       - prompt: "What warehouses do I have access to?"
         criteria:
           must_call: [get_warehouses]
-          must_not_call: [create_table_monitor_mac, create_metric_monitor_mac]
+          must_not_call: [create_or_update_table_monitor, create_or_update_metric_monitor]
           output_must_not_contain: ["MCON++"]
     criteria:
       judge_rubric: |

plugins/claude-code/evals/prevent/live-evals-dev.yaml

@@ -74,9 +74,9 @@ cases:
         monitor generation to the monitoring-advisor skill — not generate
         monitor YAML inline as part of prevent. The handoff is observable
         via the monitoring-advisor's MCP calls (one of:
-        get_validation_predicates, create_validation_monitor_mac,
-        create_metric_monitor_mac, create_comparison_monitor_mac,
-        create_custom_sql_monitor_mac) appearing in the turn 2 trace.
+        get_validation_predicates, create_or_update_validation_monitor,
+        create_or_update_metric_monitor, create_or_update_comparison_monitor,
+        create_or_update_sql_monitor) appearing in the turn 2 trace.
 
         The output should include monitor YAML (or describe the YAML) that
         targets the is_active column specifically — for example a validation

skills/monitoring-advisor/SKILL.md

@@ -2,7 +2,7 @@
 name: monte-carlo-monitoring-advisor
 description: Analyze data coverage, create monitors for warehouse tables and AI agents. Covers coverage gaps, use-case analysis, data monitor creation, and agent observability.
 bucket: Monitoring
-version: 2.0.0
+version: 2.1.1
 ---
 
 # Monte Carlo Monitoring Advisor Skill
@@ -82,13 +82,15 @@ All tools are available via the `monte-carlo` MCP server.
 
 ### Data monitor creation tools
 
+All five tools follow a **two-call preview-then-confirm pattern**: the first call (with the default `dry_run=True`) returns rendered MaC YAML for review; the second call (`dry_run=False`) deploys the monitor live and returns a deep link to it. Pass `monitor_uuid` on either call to update an existing monitor in place instead of creating a new one. See `references/data-monitor-creation.md` for the full flow.
+
 | Tool | Purpose |
 | --- | --- |
-| `create_table_monitor_mac` | Generate table monitor YAML (dry-run) |
-| `create_metric_monitor_mac` | Generate metric monitor YAML (dry-run) |
-| `create_validation_monitor_mac` | Generate validation monitor YAML (dry-run) |
-| `create_custom_sql_monitor_mac` | Generate custom SQL monitor YAML (dry-run) |
-| `create_comparison_monitor_mac` | Generate comparison monitor YAML (dry-run) |
+| `create_or_update_table_monitor` | Create or update a table monitor (preview YAML on `dry_run=True`, deploy on `dry_run=False`) |
+| `create_or_update_metric_monitor` | Create or update a metric monitor (preview YAML on `dry_run=True`, deploy on `dry_run=False`) |
+| `create_or_update_validation_monitor` | Create or update a validation monitor (preview YAML on `dry_run=True`, deploy on `dry_run=False`) |
+| `create_or_update_sql_monitor` | Create or update a custom SQL monitor (preview YAML on `dry_run=True`, deploy on `dry_run=False`) |
+| `create_or_update_comparison_monitor` | Create or update a comparison monitor (preview YAML on `dry_run=True`, deploy on `dry_run=False`) |
 
 ### Agent monitoring tools
 
@@ -210,7 +212,7 @@ When coverage analysis leads to monitor creation, gather this context before rea
 
 ### Use-case tag monitors
 
-The most common output of coverage analysis is a **table monitor scoped by use-case tags** via `create_table_monitor_mac`. The `asset_selection` parameter uses this structure:
+The most common output of coverage analysis is a **table monitor scoped by use-case tags** via `create_or_update_table_monitor`. The `asset_selection` parameter uses this structure:
 
 ```json
 {

skills/monitoring-advisor/references/data-comparison-monitor.md

@@ -1,6 +1,6 @@
 # Comparison Monitor Reference
 
-Detailed reference for building `create_comparison_monitor_mac` tool calls.
+Detailed reference for building `create_or_update_comparison_monitor` tool calls. The tool follows the **two-call preview-then-confirm pattern** — see `data-monitor-creation.md` for the full flow.
 
 ## Critical Constraints
 
@@ -49,6 +49,16 @@ Before constructing alert conditions, you MUST verify that both tables exist and
 | `target_warehouse` | string | Warehouse name or UUID for the target table. Required if `target_table` is not an MCON. |
 | `segment_fields` | array of string | Fields to segment the comparison by. Must exist in BOTH tables with the same name. |
 | `domain_uuids` | array of string (uuid) | Domain UUIDs (use `get_domains` to list). Data monitors accept exactly one UUID in the list. |
+| `schedule_type` | string | Schedule type: `"fixed"` (default), `"dynamic"`, `"manual"`. |
+| `interval_minutes` | int | Schedule interval in minutes (only for `schedule_type="fixed"`). |
+| `audiences` | array of string | Notification audience **names** (not UUIDs) to alert when the monitor triggers. |
+| `failure_audiences` | array of string | Notification audience names to alert on query execution failures. |
+| `notes` | string | Free-text notes shown in the UI (separate from `description`). |
+| `priority` | string | Monitor priority (e.g. `"P1"`, `"P2"`). |
+| `tags` | array of `{name, value}` | Key-value tags to attach. |
+| `is_draft` | bool | When `True`, saves the monitor as a draft (not active). Default `False`. |
+| `monitor_uuid` | string (uuid) | UUID of an existing monitor to update in place. Omit to create a new monitor. **PUT semantics:** the call fully replaces the monitor's configuration — fields you omit revert to tool defaults, they are NOT left untouched. Before editing, read the current config with `get_monitors(monitor_ids=[<uuid>], include_fields=["config"])` and re-pass every field you want to keep. See `data-monitor-creation.md` (Step 7) for the safe-edit workflow. |
+| `dry_run` | bool | Default `True`. Preview mode. When omitted or `True`, returns YAML preview in `result.yaml`. When `False`, actually creates/updates the monitor and returns `result.monitor_uuid` + a deep link in `result.instructions`. See `data-monitor-creation.md`. |
 
 ---
 

skills/monitoring-advisor/references/data-custom-sql-monitor.md

@@ -1,6 +1,6 @@
 # Custom SQL Monitor Reference
 
-Detailed reference for building `create_custom_sql_monitor_mac` tool calls.
+Detailed reference for building `create_or_update_sql_monitor` tool calls. The tool follows the **two-call preview-then-confirm pattern** — see `data-monitor-creation.md` for the full flow.
 
 ## Critical Constraints
 
@@ -48,6 +48,19 @@ If you find yourself contorting another monitor type to fit the user's intent, s
 | Parameter | Type | Description |
 |-----------|------|-------------|
 | `domain_uuids` | array of string (uuid) | Domain UUIDs (use `get_domains` to list). Data monitors accept exactly one UUID in the list. |
+| `query_result_type` | string | What the SQL query returns — see the tool's enum for accepted values (single numeric is the most common). |
+| `custom_sampling_sql` | string | Optional SQL used to sample rows that contributed to the result (shown in alert detail). |
+| `variable_definitions` | object | Named variables that can be referenced in `sql` / `custom_sampling_sql`. |
+| `schedule_type` | string | Schedule type: `"fixed"` (default), `"dynamic"`, `"manual"`. |
+| `interval_minutes` | int | Schedule interval in minutes (only for `schedule_type="fixed"`). |
+| `audiences` | array of string | Notification audience **names** (not UUIDs) to alert when the monitor triggers. |
+| `failure_audiences` | array of string | Notification audience names to alert on query execution failures. |
+| `notes` | string | Free-text notes shown in the UI (separate from `description`). |
+| `priority` | string | Monitor priority (e.g. `"P1"`, `"P2"`). |
+| `tags` | array of `{name, value}` | Key-value tags to attach. |
+| `is_draft` | bool | When `True`, saves the monitor as a draft (not active). Default `False`. |
+| `monitor_uuid` | string (uuid) | UUID of an existing monitor to update in place. Omit to create a new monitor. **PUT semantics:** the call fully replaces the monitor's configuration — fields you omit revert to tool defaults, they are NOT left untouched. Before editing, read the current config with `get_monitors(monitor_ids=[<uuid>], include_fields=["config"])` and re-pass every field you want to keep. See `data-monitor-creation.md` (Step 7) for the safe-edit workflow. |
+| `dry_run` | bool | Default `True`. Preview mode. When omitted or `True`, returns YAML preview in `result.yaml`. When `False`, actually creates/updates the monitor and returns `result.monitor_uuid` + a deep link in `result.instructions`. See `data-monitor-creation.md`. |
 
 ---
 

skills/monitoring-advisor/references/data-metric-monitor.md

@@ -1,6 +1,6 @@
 # Metric Monitor Reference
 
-Detailed reference for building `create_metric_monitor_mac` tool calls.
+Detailed reference for building `create_or_update_metric_monitor` tool calls. The tool follows the **two-call preview-then-confirm pattern** — see `data-monitor-creation.md` for the full flow.
 
 ## Critical Constraints
 
@@ -35,12 +35,25 @@ Use a metric monitor when the user wants to:
 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
 | `aggregate_time_field` | string | none | Timestamp/datetime column for time-windowed aggregation. **When provided, MUST be a real column from the table — NEVER guess this value.** When omitted, the monitor queries all rows on each run (whole-table scan). Omit for tables without a suitable timestamp column. |
+| `aggregate_time_sql` | string | none | SQL expression that produces the timestamp for bucketing (e.g. `CAST(payload:event_time AS TIMESTAMP)`). Use when the timestamp is embedded in a variant/object column or needs transformation. Mutually exclusive with `aggregate_time_field`. |
 | `warehouse` | string | auto-resolved | Warehouse name or UUID. Required if `table` is not an MCON. |
 | `segment_fields` | array of string | none | Fields to group/segment metrics by (e.g., `["country", "status"]`). |
+| `segment_sql` | array of string | none | SQL expressions to segment by (e.g. `["CASE WHEN amount > 100 THEN 'high' ELSE 'low' END"]`). |
 | `aggregate_by` | string | `"day"` | Time interval: `"hour"`, `"day"`, `"week"`, `"month"`. |
 | `where_condition` | string | none | SQL WHERE clause (without `WHERE` keyword) to filter rows before computing metrics. |
+| `sensitivity` | string | none | Anomaly detection sensitivity for AUTO operators: `"low"`, `"medium"`, `"high"`. |
+| `collection_lag_hours` | int | none | Hours to wait after expected data arrival before running the monitor. |
+| `schedule_type` | string | `"fixed"` | Schedule type: `"fixed"`, `"dynamic"`, `"manual"`. |
 | `interval_minutes` | int | auto | Schedule interval in minutes. Must be compatible with `aggregate_by` (see note below). If not specified, the tool defaults to the minimum valid interval for the chosen `aggregate_by`. |
 | `domain_uuids` | array of string (uuid) | none | Domain UUIDs (use `get_domains` to list). Data monitors accept exactly one UUID in the list. |
+| `audiences` | array of string | none | Notification audience **names** (not UUIDs) to alert when the monitor triggers. |
+| `failure_audiences` | array of string | none | Notification audience names to alert on query execution failures. |
+| `notes` | string | none | Free-text notes shown in the UI (separate from `description`). |
+| `priority` | string | none | Monitor priority (e.g. `"P1"`, `"P2"`). |
+| `tags` | array of `{name, value}` | none | Key-value tags to attach. |
+| `is_draft` | bool | `False` | When `True`, saves the monitor as a draft (not active). |
+| `monitor_uuid` | string (uuid) | none | UUID of an existing monitor to update in place. Omit to create a new monitor. **PUT semantics:** the call fully replaces the monitor's configuration — fields you omit revert to tool defaults, they are NOT left untouched. Before editing, read the current config with `get_monitors(monitor_ids=[<uuid>], include_fields=["config"])` and re-pass every field you want to keep. See `data-monitor-creation.md` (Step 7) for the safe-edit workflow. |
+| `dry_run` | bool | `True` | Preview mode. When omitted or `True`, returns YAML preview in `result.yaml`. When `False`, actually creates/updates the monitor and returns `result.monitor_uuid` + a deep link in `result.instructions`. See `data-monitor-creation.md`. |
 
 ---