diff --git a/README.md b/README.md index 9a5599a..c81a943 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ Skills are grouped by the job they help you do. Orchestrated workflows sequence |---|---|---| | **Proactive Monitoring** _(workflow)_ | Sequences coverage analysis → gap identification → monitor creation into a guided flow. | [SKILL](skills/proactive-monitoring/SKILL.md) | | **Monitoring Advisor** | Identifies coverage gaps and creates monitors for warehouse tables or AI agents — validates tables and fields against your live workspace, emits monitors-as-code YAML. | [README](skills/monitoring-advisor/README.md) | +| **Manage MaC** | Create, edit, validate, and import Monitors-as-Code YAML files — authors new monitors from scratch, modifies existing files, validates against the published JSON Schema, and exports live monitors to YAML. | [SKILL](skills/manage-mac/SKILL.md) | | **Tune Monitor** | Recommends sensitivity, segment, and schedule changes to reduce alert noise on an existing metric monitor. | [SKILL](skills/tune-monitor/SKILL.md) | ### Prevent — catch issues before they ship diff --git a/plugins/claude-code/.claude-plugin/plugin.json b/plugins/claude-code/.claude-plugin/plugin.json index dabbb42..24190ba 100644 --- a/plugins/claude-code/.claude-plugin/plugin.json +++ b/plugins/claude-code/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "mc-agent-toolkit", - "version": "1.11.1", + "version": "1.12.0", "description": "Monte Carlo Agent Toolkit — data observability for AI coding agents. Surfaces blast radius, active alerts, and monitor coverage gaps before you edit dbt SQL. Includes skills for incident response, proactive monitoring, storage cost analysis, and push ingestion. Hooks enforce data-safety checks automatically as you work.", "author": { "name": "Monte Carlo", diff --git a/plugins/claude-code/CHANGELOG.md b/plugins/claude-code/CHANGELOG.md index 4f65a5a..6fca5c0 100644 --- a/plugins/claude-code/CHANGELOG.md +++ b/plugins/claude-code/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to the Monte Carlo Agent Toolkit plugin for Claude Code will Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.12.0] - 2026-05-20 + +### Added + +- New `/manage-mac` skill: create, edit, validate, and import Monitors-as-Code YAML files — CLI-first: uses `montecarlo monitors compile` to validate and `apply` to deploy; falls back to MC MCP tools then manual validation +- Schema validation gates injected into `monitoring-advisor` and `tune-monitor` — agents now validate generated YAML against the published schema before presenting it to the user + ## [1.11.1] - 2026-05-13 ### Added diff --git a/plugins/claude-code/skills/manage-mac b/plugins/claude-code/skills/manage-mac new file mode 120000 index 0000000..2b61899 --- /dev/null +++ b/plugins/claude-code/skills/manage-mac @@ -0,0 +1 @@ +../../../skills/manage-mac \ No newline at end of file diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json index 677794e..a8bdf7e 100644 --- a/plugins/codex/.codex-plugin/plugin.json +++ b/plugins/codex/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "mc-agent-toolkit", - "version": "1.11.1", + "version": "1.12.0", "description": "Monte Carlo Agent Toolkit — data observability skills and enforcement hooks for AI coding agents.", "author": { "name": "Monte Carlo", diff --git a/plugins/codex/CHANGELOG.md b/plugins/codex/CHANGELOG.md index 482437b..ab13994 100644 --- a/plugins/codex/CHANGELOG.md +++ b/plugins/codex/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to the Monte Carlo Agent Toolkit plugin for Codex will be do Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.12.0] - 2026-05-20 + +### Added + +- New `/manage-mac` skill: create, edit, validate, and import Monitors-as-Code YAML files — CLI-first: uses `montecarlo monitors compile` to validate and `apply` to deploy; falls back to MC MCP tools then manual validation +- Schema validation gates injected into `monitoring-advisor` and `tune-monitor` — agents now validate generated YAML against the published schema before presenting it to the user + ## [1.11.1] - 2026-05-13 ### Added diff --git a/plugins/codex/skills/manage-mac b/plugins/codex/skills/manage-mac new file mode 120000 index 0000000..2b61899 --- /dev/null +++ b/plugins/codex/skills/manage-mac @@ -0,0 +1 @@ +../../../skills/manage-mac \ No newline at end of file diff --git a/plugins/copilot/CHANGELOG.md b/plugins/copilot/CHANGELOG.md index 233e4fb..1ee2373 100644 --- a/plugins/copilot/CHANGELOG.md +++ b/plugins/copilot/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to the Monte Carlo Agent Toolkit plugin for Copilot CLI will Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.12.0] - 2026-05-20 + +### Added + +- New `/manage-mac` skill: create, edit, validate, and import Monitors-as-Code YAML files — CLI-first: uses `montecarlo monitors compile` to validate and `apply` to deploy; falls back to MC MCP tools then manual validation +- Schema validation gates injected into `monitoring-advisor` and `tune-monitor` — agents now validate generated YAML against the published schema before presenting it to the user + ## [1.11.1] - 2026-05-13 ### Added diff --git a/plugins/copilot/plugin.json b/plugins/copilot/plugin.json index d7cdeb8..f93e020 100644 --- a/plugins/copilot/plugin.json +++ b/plugins/copilot/plugin.json @@ -1,6 +1,6 @@ { "name": "mc-agent-toolkit", - "version": "1.11.1", + "version": "1.12.0", "description": "Monte Carlo Agent Toolkit — data observability for AI coding agents.", "author": { "name": "Monte Carlo", diff --git a/plugins/copilot/skills/manage-mac b/plugins/copilot/skills/manage-mac new file mode 120000 index 0000000..2b61899 --- /dev/null +++ b/plugins/copilot/skills/manage-mac @@ -0,0 +1 @@ +../../../skills/manage-mac \ No newline at end of file diff --git a/plugins/cursor/.cursor-plugin/plugin.json b/plugins/cursor/.cursor-plugin/plugin.json index 955554a..26ac608 100644 --- a/plugins/cursor/.cursor-plugin/plugin.json +++ b/plugins/cursor/.cursor-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "mc-agent-toolkit", - "version": "1.11.1", + "version": "1.12.0", "description": "Monte Carlo Agent Toolkit — data observability skills and enforcement hooks for AI coding agents.", "author": { "name": "Monte Carlo", diff --git a/plugins/cursor/CHANGELOG.md b/plugins/cursor/CHANGELOG.md index 5c5cc58..ceb7497 100644 --- a/plugins/cursor/CHANGELOG.md +++ b/plugins/cursor/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to the Monte Carlo Agent Toolkit plugin for Cursor will be d Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.12.0] - 2026-05-20 + +### Added + +- New `/manage-mac` skill: create, edit, validate, and import Monitors-as-Code YAML files — CLI-first: uses `montecarlo monitors compile` to validate and `apply` to deploy; falls back to MC MCP tools then manual validation +- Schema validation gates injected into `monitoring-advisor` and `tune-monitor` — agents now validate generated YAML against the published schema before presenting it to the user + ## [1.11.1] - 2026-05-13 ### Added diff --git a/plugins/cursor/skills/manage-mac b/plugins/cursor/skills/manage-mac new file mode 120000 index 0000000..2b61899 --- /dev/null +++ b/plugins/cursor/skills/manage-mac @@ -0,0 +1 @@ +../../../skills/manage-mac \ No newline at end of file diff --git a/plugins/opencode/CHANGELOG.md b/plugins/opencode/CHANGELOG.md index f51a1c6..5aa082b 100644 --- a/plugins/opencode/CHANGELOG.md +++ b/plugins/opencode/CHANGELOG.md @@ -5,6 +5,13 @@ All notable changes to the Monte Carlo Agent Toolkit plugin for OpenCode will be Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.12.0] - 2026-05-20 + +### Added + +- New `/manage-mac` skill: create, edit, validate, and import Monitors-as-Code YAML files — CLI-first: uses `montecarlo monitors compile` to validate and `apply` to deploy; falls back to MC MCP tools then manual validation +- Schema validation gates injected into `monitoring-advisor` and `tune-monitor` — agents now validate generated YAML against the published schema before presenting it to the user + ## [1.11.1] - 2026-05-13 ### Added diff --git a/plugins/opencode/package.json b/plugins/opencode/package.json index b5f39d6..58438cf 100644 --- a/plugins/opencode/package.json +++ b/plugins/opencode/package.json @@ -1,6 +1,6 @@ { "name": "@montecarlo/mc-agent-toolkit", - "version": "1.11.1", + "version": "1.12.0", "description": "Monte Carlo Agent Toolkit — data observability skills and enforcement hooks for AI coding agents.", "type": "module", "main": "src/index.ts", diff --git a/plugins/opencode/skills/manage-mac b/plugins/opencode/skills/manage-mac new file mode 120000 index 0000000..2b61899 --- /dev/null +++ b/plugins/opencode/skills/manage-mac @@ -0,0 +1 @@ +../../../skills/manage-mac \ No newline at end of file diff --git a/skills/README.md b/skills/README.md index 05bbf1c..fe27ed3 100644 --- a/skills/README.md +++ b/skills/README.md @@ -18,6 +18,7 @@ Skills are platform-agnostic instruction sets that tell an AI coding agent what | **[Storage Cost Analysis](storage-cost-analysis/)** | Identifies storage waste patterns (unread, zombie, dead-end tables) and recommends safe cleanup with cost estimates. | | **[Performance Diagnosis](performance-diagnosis/)** | Diagnoses slow pipelines and expensive queries across Airflow, dbt, and Databricks with tiered investigation. | | **[Remediation](remediation/)** | Investigates and remediates data quality alerts — runs TSA root cause analysis, discovers available tools, executes fixes (or escalates), and documents the resolution. | +| **[Manage MaC](manage-mac/)** | Create, edit, validate, and import Monitors-as-Code YAML files — authors new monitors from scratch, modifies existing files, validates against the published JSON Schema, and exports live monitors to YAML. | | **[Tune Monitor](tune-monitor/)** | Analyzes a Monte Carlo metric monitor's alert history and recommends configuration changes to reduce noise — sensitivity, WHERE conditions, segment exclusions, schedule, and aggregation. | | **[Connection Auth Rules](connection-auth-rules/)** | Build a Connection Auth Rules configuration for a Monte Carlo connection type. Fetches live connector schemas and transform steps from the apollo-agent repo. | | **[Instrument Agent](instrument-agent/)** | Instruments a Python AI agent for Monte Carlo Agent Observability — detects AI libraries, installs the Monte Carlo OpenTelemetry SDK, sets up tracing, and verifies traces in Monte Carlo. Asks before editing. | diff --git a/skills/manage-mac/SKILL.md b/skills/manage-mac/SKILL.md new file mode 100644 index 0000000..5527b80 --- /dev/null +++ b/skills/manage-mac/SKILL.md @@ -0,0 +1,397 @@ +--- +name: monte-carlo-manage-mac +description: Create, edit, validate, and import Monitors-as-Code YAML files. CLI-first; falls back to MC MCP tools, then manual validation. +when_to_use: | + Invoke when the user has a MaC YAML file they want to create, edit, or validate, or when they + want to export live monitors into a MaC YAML file. + Example triggers: "create a monitors YAML for this table", "add a metric monitor to my MaC file", + "validate my monitors.yaml before I apply it", "what's wrong with my MaC file", + "export my existing monitors to YAML", "get my monitors into a file so I can commit them", + "import my live monitors to YAML", "get a MaC file from my existing monitors". + Do NOT invoke when the user wants to discover what to monitor or generate monitors from scratch + via table exploration — use monitoring-advisor for that. +bucket: Monitoring +version: 1.0.0 +--- + +# Manage MaC: Monitors-as-Code YAML Authoring + +You are a Monitors-as-Code (MaC) YAML authoring agent. Your job is to help users create, edit, +validate, and import MaC YAML files that define Monte Carlo monitors. + +**Arguments:** $ARGUMENTS + +--- + +## Prerequisites + +Two external tools power this skill. Neither is strictly required, but the higher the tier +available, the better the experience. + +**MC CLI (Tier 1)** +- Docs: https://docs.getmontecarlo.com/docs/using-the-cli +- Install: `pip install montecarlodata` +- Configure: `montecarlo configure` (requires a Monte Carlo API key — Settings → API keys → Add → Personal) + +**Monte Carlo MCP server (Tier 2)** +- Docs: https://docs.getmontecarlo.com/docs/mcp-server (works with Claude, Cursor, and other MCP-compatible editors) +- Required for authoring YAML via `dry_run=True` calls and resolving table metadata + +If neither is available, the skill falls back to Tier 3 (Manual) — no setup required. + +--- + +## Tooling tiers + +Use the highest available tier: + +| Tier | Tool | Used for | +|---|---|---| +| 1 — CLI | `montecarlo` binary | Validate (`compile`), apply, import (`convert-to-mac`, `export`) | +| 2 — MCP | Monte Carlo MCP server | Author YAML shapes via `dry_run=True`, resolve table metadata | +| 3 — Manual | No external tools | Validate field names/enums/types when CLI unavailable | + +### CLI check + +Before starting any workflow, run: + +```bash +montecarlo --version +``` + +If the command fails or is not found, inform the user: +> "MC CLI is not installed. It enables local validation and streamlined apply/import. +> Install: `pip install montecarlodata` +> Configure: `montecarlo configure` (requires a Monte Carlo API key — Settings → API keys → Add → Personal) +> Would you like to set it up, or continue without it?" + +If the user accepts, give the full install and configure steps, then resume the workflow once setup is complete. +If the user declines, proceed using Tier 2 (MCP) and Tier 3 (Manual) only. + +--- + +## Entry point detection + +| User intent | Workflow | +|---|---| +| No existing file; wants monitors for a table or use case | **Create** | +| Has an existing file; wants to add, modify, or remove monitors | **Edit** | +| Has an existing file; wants to check it before applying | **Validate** | +| Wants to export live monitors into a MaC YAML file | **Import** | +| Wants to discover what to monitor or explore a table | Redirect to `monitoring-advisor` — do not proceed | + +If ambiguous, ask which workflow is needed. + +--- + +## MCP tools reference + +| Tool | Used for | +|---|---| +| `search` | Resolve a table name to its MCON and `full_table_id` | +| `get_table` | Verify column names and retrieve table schema | +| `get_warehouses` | Resolve warehouse UUID | +| `create_or_update_metric_monitor` | Author `metric` monitors (`dry_run=True`) | +| `create_or_update_sql_monitor` | Author `custom_sql` monitors (`dry_run=True`) | +| `create_or_update_validation_monitor` | Author `validation` monitors (`dry_run=True`) | +| `create_or_update_table_monitor` | Author `table` monitors (`dry_run=True`) | +| `create_or_update_comparison_monitor` | Author `metric_comparison` monitors (`dry_run=True`) | +| `get_validation_predicates` | List valid predicates for `validation` monitors | +| `get_monitors` | Fetch live monitors in YAML format (Import fallback) | + +For monitor types without a dedicated MCP tool (`json_schema`, `query_performance`, `bulk_monitor`), +fall back to schema-based authoring. Never guess field names — derive them from the schema: + +```bash +curl -s https://clidocs.getmontecarlo.com/mac/schema.json +``` + +--- + +## Create workflow + +### Step 1: Gather context + +Ask for any information not already provided: + +1. **Table(s):** fully qualified name (database.schema.table or equivalent) +2. **Monitor type(s):** what kind of monitoring — metric, validation, custom SQL, etc. + Do not suggest deprecated types: `field_health`, `dimension_tracking`, `field_quality`, + `comparison`, `freshness`, or `volume`. If the user explicitly requests one of these, + decline: inform them it is no longer supported, and suggest the closest valid alternative + (e.g. `freshness` or `volume` → `metric` monitor tracking recency or row count; + `field_quality` → `validation` monitor; `comparison` → `metric_comparison`; `field_health` → `metric`). + Note: `comparison` (deprecated) and `metric_comparison` (current) are distinct — never + decline a request for a `metric_comparison` monitor. + Common phrases → monitor type: "null rate / percent null / zero rate / column distribution" → `metric`; + "validate email format / check values in set / regex match" → `validation`; + "query taking too long / slow queries" → `query_performance`. +3. **Namespace:** used with `montecarlo monitors apply --namespace ` +4. **Notification audiences:** optional — ask only if the user mentions alerting +5. **Type-specific required inputs:** + - `metric`: ask for the metric to track if not provided (e.g. row count, null rate, freshness, custom metric expression) + - `custom_sql`: ask for the SQL query if not provided + - `json_schema`: ask for the field name to check if not provided + +### Step 2: Resolve table and field metadata (Tier 2 — MCP) + +Follow steps 1–3 from `../monitoring-advisor/references/data-monitor-creation.md` to: +- Resolve the MCON and `full_table_id` via `search` +- Verify column names via `get_table` +- Resolve domain UUID and warehouse UUID + +Never guess column names, warehouse UUIDs, or domain UUIDs. + +For `validation` monitors, call `get_validation_predicates` to confirm the predicate names +available in the user's workspace before proceeding. If the result is empty, inform the user +that no validation predicates are configured in their workspace and stop. + +### Step 3: Author YAML blocks (Tier 2 — MCP) + +For each monitor, call the appropriate `create_or_update_*_monitor` with `dry_run=True` and the +parameters the user specified. The backend returns a canonical YAML block — use that output as +the YAML for the file rather than authoring it by hand. + +Call the tool once per monitor. Complete all dry_run calls before assembling the file. If an +MCP tool returns an error, stop and surface the error message to the user. Do not proceed with +a partial result. + +### Step 4: Assemble the YAML file + +1. Add the yaml-language-server header as the first line: + ```yaml + # yaml-language-server: $schema=https://clidocs.getmontecarlo.com/mac/schema.json + ``` +2. Open with `montecarlo:` as the root key +3. Group the dry_run output blocks by monitor type under their respective keys +4. If the user specified notification audiences, add the `audiences` field (array of strings) + directly on each monitor object + +### Step 5: Validate and apply + +Prompt for namespace if not already provided. + +**Tier 1 — CLI (preferred):** +```bash +montecarlo monitors compile --namespace # validate +montecarlo monitors apply --namespace # deploy +``` + +**Tier 3 fallback (CLI unavailable):** Run the Validate workflow against the assembled YAML, +then present the apply command for the user to run manually when CLI is available. + +--- + +## Edit workflow + +### Step 1: Read the file + +Use the Read tool to load the user's file. Ask for the path if not provided. If the Read tool +returns an error (file not found), report it and ask for the correct path — do not create a new +file silently. + +### Step 2: Understand the requested change + +**Adding a monitor:** Follow the Create workflow (Steps 1–4) to generate the new monitor block +via `dry_run=True`, then append it to the correct type list in the file. + +**Modifying a monitor:** Call `create_or_update_*_monitor(dry_run=True, name=, ...)` +with the updated parameters, preserving the existing `name` value. Use the returned YAML block +to replace the existing monitor entry. Do not look up or pass a UUID — in the MaC realm, +identity is the `name` field plus namespace. + +**Removing a monitor:** Delete the monitor object and preserve all other monitors in the type +list. If it is the only item under its type key, remove the entire type key — do not leave +an empty list. + +**Deprecated field names:** While reading the file, check for fields marked `deprecated: true` +in the schema. Scope this scan to the `montecarlo:` block only. If found, list all occurrences +and offer to migrate them in a single operation before applying other changes. Apply only after +explicit user confirmation. If the user declines, proceed with the requested edit without +migrating. The schema's `description` encodes the canonical replacement name +(e.g. "Deprecated. Use `warehouse` instead.") — never guess. If both the deprecated field and +its replacement are present with different values, flag the conflict and ask the user which to keep. + +**Deprecated monitor types** (`field_health`, `dimension_tracking`, `field_quality`, `comparison`, +`freshness`, `volume`): cannot be mechanically migrated — offer to re-author with a supported type +via the Create workflow, then delete the deprecated block. + +**YAML-level fields** (not part of the monitor definition sent to the backend): add or modify +these directly in the YAML without calling the MCP tool. Common examples: `is_paused`, `labels`, +`tags`, `priority`, `audiences`, `data_quality_dimension`, `domains`. Refer to the schema to +confirm others. + +### Step 3: Write and validate + +Show only what changed (before/after for modifications, new block for additions). Write the +updated file using the Edit tool. + +Ensure the `# yaml-language-server: $schema=https://clidocs.getmontecarlo.com/mac/schema.json` +header is the first line. Add it if missing. + +If removing the last monitor of the last type, the file should contain only the +yaml-language-server header and `montecarlo: {}`. + +**Tier 1 — CLI (preferred):** +```bash +montecarlo monitors compile --namespace # validate +montecarlo monitors apply --namespace # deploy +``` + +**Tier 3 fallback (CLI unavailable):** Run the Validate workflow against the updated file. + +--- + +## Validate workflow + +### Step 1: Try CLI first (Tier 1) + +```bash +montecarlo monitors compile --namespace +``` + +If this succeeds, report the output to the user and stop — no further LLM validation needed. + +### Step 2: Manual validation fallback (Tier 3 — no external tools) + +Use this path only if CLI is unavailable. + +Fetch the schema — it is ~50KB and WebFetch truncates it, so use Bash: + +```bash +curl -s https://clidocs.getmontecarlo.com/mac/schema.json +``` + +If Bash is unavailable, fall back to WebFetch — but coverage of `validation`, `table`, +`query_performance`, and `bulk_monitor` types may be incomplete. + +If the schema cannot be fetched, stop and report: +> Cannot fetch the MaC schema from `https://clidocs.getmontecarlo.com/mac/schema.json`. Please +> check your network connection and try again. + +### Step 3: Read the file + +Use the Read tool to load the user's file. Ask for the path if not provided. + +### Step 4: Validate against the schema + +For each monitor in the file, check: + +1. **Required fields present:** every field marked `required` in the schema items is present +2. **No unknown fields:** no field names that don't appear in the schema for that monitor type +3. **Enum values valid:** validate against the schema, not memory. Enums are case-sensitive and + vary by field: `sensitivity` is lowercase (`high`/`medium`/`low`), `priority` is uppercase + (`P1`–`P5`), `data_quality_dimension` is uppercase (`ACCURACY`, `COMPLETENESS`, `CONSISTENCY`, + `TIMELINESS`, `UNIQUENESS`, `VALIDITY`), `alert_conditions[].operator` is uppercase (`GT`, + `GTE`, `LT`, `LTE`, `EQ`, `NEQ`, `AUTO`, `AUTO_HIGH`, `AUTO_LOW`, `INSIDE_RANGE`, + `OUTSIDE_RANGE`, `NOOP`). +4. **Type correctness:** string fields are strings, integer fields are integers, etc. +5. **Top-level structure:** `montecarlo:` must be present; its sub-keys must be valid monitor + type keys or `notifications:`. Extra top-level keys (e.g. dbt `version:`, `models:`) are + allowed and must not be flagged. + +**Schema scope disclaimer:** The schema validates field names, types, and enum values only. +Cross-field semantic constraints are enforced by the backend — a file that passes schema +validation may still be rejected by `montecarlo monitors apply`. + +**Type-specific reminders:** +- `metric` monitors use a nested `data_source` object (`data_source.table`), not a flat `table` + field. `alert_conditions` is required. `sensitivity` is only valid on `metric`. +- `custom_sql` monitors require both `sql` (the query string) and `schedule`. +- `validation` monitors have a singular `alert_condition` field whose value is a predicate tree. + The minimal valid structure requires `type: GROUP`, `operator`, and `conditions` with at least + one `BINARY` or `UNARY` node. Binary predicates require both `left` (field) and `right` + (value) nodes; unary predicates (`not_null`, `is_not_empty`) require only `left`. +- `query_performance` monitors have no `table` field — asset targeting uses a `selection` array. + `alert_conditions` items require `threshold` and `metric` fields; `additionalProperties: false` + applies — unknown fields like `threshold_value` or `type` will be flagged. +- `table` monitors have no flat `table` field — asset targeting uses `asset_selection`. +- `notifications:` is the NaC block — do not validate or modify its contents. +- `bulk_monitor` monitors use `asset_selection` for targeting, not a `tables` field. + Required fields: `description`, `asset_selection`, `monitor_type`, `alert_conditions`, + `schedule`. `monitor_type` enum: `bulk_metric` or `bulk_pii` — `metric` is not valid. + +Do not author new monitors of deprecated types. If the file contains them, validate what is +present but do not add new instances. + +### Step 5: Report findings + +If the file is valid: +> The file is valid. Apply with: `montecarlo monitors apply --namespace ` + +If issues exist, report all in a single pass: + +``` +Validation issues found: + +1. metric[0] ("orders_row_count") + - Missing required field: `description` + - Fix: add `description: "Row count for orders table"` + +2. custom_sql[0] ("status_check") + - Unknown field: `sensitivity` + - Fix: remove — `sensitivity` is only valid on `metric` monitors +``` + +**Deprecated field migration:** List all occurrences of deprecated fields found (every instance, +not just unique field names) and offer to migrate them. Apply only after explicit user confirmation. + +--- + +## Import workflow + +### Step 1: Identify the source + +Ask what to import: +- A specific table: "Which table? Provide the full name (database.schema.table)" +- A namespace or group: "Any filters? (table name pattern, monitor type, namespace)" + +### Step 2: Fetch monitors + +**Tier 1 — CLI (preferred):** +```bash +montecarlo monitors export # export all +montecarlo monitors convert-to-mac # convert UI monitors to MaC YAML +``` + +**Tier 2 — MCP fallback:** +``` +get_monitors(full_table_id="database.schema.table", config_format="yaml") +``` +For broader imports, omit `full_table_id` and filter by other criteria (e.g. `namespace`). + +If no monitors are returned, inform the user and stop — do not create an empty file. + +### Step 3: Assemble the YAML file + +1. Add the yaml-language-server header as the first line +2. Group returned monitors by type under a single `montecarlo:` block +3. Deduplicate: two monitors are duplicates if they share the same `name` field. Keep the one + with a `uuid` (deployed version). If neither or both have UUIDs, keep the first and flag + the conflict. +4. Scan for deprecated field names; offer to migrate before saving +5. Prompt for a namespace if not provided + +### Step 4: Present and save + +Show the assembled YAML and ask for a file path if not provided. If the user specifies an +existing file, read it first, merge by type list (deduplicating by `name`), and write the result. +For a new file, use the Write tool. + +Remind the user: +> These monitors are now defined in your repo. Once you run `montecarlo monitors apply`, +> Monte Carlo will manage them as MaC resources identified by their `name` field. Future edits +> should be made in this file, not in the UI. + +If the user wants to validate before saving, run the Validate workflow first. To add monitors +immediately after importing, transition to the Edit workflow retaining the file path and namespace. + +--- + +## File format rules + +- Always include `# yaml-language-server: $schema=https://clidocs.getmontecarlo.com/mac/schema.json` + as the first line +- Use 2-space indentation +- Quote string values that contain special characters or colons +- Do not add inline comments explaining field values diff --git a/skills/monitoring-advisor/references/data-monitor-creation.md b/skills/monitoring-advisor/references/data-monitor-creation.md index 241261b..9a8df00 100644 --- a/skills/monitoring-advisor/references/data-monitor-creation.md +++ b/skills/monitoring-advisor/references/data-monitor-creation.md @@ -121,6 +121,17 @@ Before calling the creation tool, present the monitor configuration in plain lan Ask: "Does this look correct? I'll generate the monitor configuration." +Also ask how the user wants to deploy it: + +> **Deployment preference:** Deploy live now (via MCP), or save as a Monitors-as-Code YAML file to apply through your repo? +> +> - **Live (MCP):** I'll call the creation tool and the monitor will be active immediately. +> - **MaC YAML:** I'll generate the YAML definition so you can commit it to your repo and apply it with `montecarlo monitors apply`. Use `/monte-carlo-manage-mac` if you want to validate or edit the file first. + +If the user chooses MaC YAML: generate the preview YAML (dry_run=True) as usual, present it wrapped in the standard MaC structure (see MaC YAML Format), and stop -- do not call with `dry_run=False`. The user takes the YAML from there. + +If the user chooses live or does not express a preference, proceed with the standard two-call sequence in Step 7. + ### Step 7: Create the monitor This step is a **two-call sequence**. Do NOT skip the preview call. @@ -204,6 +215,22 @@ If the user prefers to deploy via CLI/CI rather than the live tool call: --- +## Schema Validation + +Always add the following comment as the **first line** of any MaC YAML file you create or edit: + +```yaml +# yaml-language-server: $schema=https://clidocs.getmontecarlo.com/mac/schema.json +``` + +The published schema is available at `https://clidocs.getmontecarlo.com/mac/schema.json`. Use WebFetch to inspect it if you're uncertain whether a field name or value is valid for a given monitor type. + +Generated YAML must not include fields that don't appear in the schema for that monitor type. Unknown fields are silently ignored by the CLI but indicate a misconfiguration and may break future validation. + +**Schema scope:** The schema validates field names, types, and enum values only. Cross-field semantic constraints (e.g. required field combinations, mutually exclusive options, conditional required fields) are NOT checked by the schema — they are enforced by the Monte Carlo backend at apply time. A file that passes schema validation may still fail on `montecarlo monitors apply`. + +--- + ## Available MCP Tools All tools are available via the `monte-carlo` MCP server. diff --git a/skills/tune-monitor/SKILL.md b/skills/tune-monitor/SKILL.md index 9540905..62af8f1 100644 --- a/skills/tune-monitor/SKILL.md +++ b/skills/tune-monitor/SKILL.md @@ -234,6 +234,8 @@ Phase 1.5. Each reference specifies the correct tool and constraints for that mo General rules for all types: 1. **Always preview first** — show the user what will change before applying. 2. **Get explicit confirmation** before applying any change. +3. **Validate the preview YAML against the schema** — before presenting the preview YAML to the user, fetch the published MaC JSON Schema from `https://clidocs.getmontecarlo.com/mac/schema.json` (WebFetch) and check the preview YAML against it. If any field in the YAML does not appear in the schema for the given monitor type, flag it and correct it. Note: the schema validates field names, types, and enum values only — cross-field semantic constraints are enforced by the backend at apply time, not by the schema. +4. **MaC-managed monitors** — if `get_monitors` returns a `mac_name` or the user mentions the monitor is managed via a MaC YAML file, note this before applying: changes made via the API will be overwritten the next time `montecarlo monitors apply` runs. Offer to hand off to `/manage-mac` (edit workflow) instead so the YAML file stays the source of truth. --- @@ -245,3 +247,4 @@ General rules for all types: - **Cite evidence.** Reference specific incident dates, segment values, and counts from the report. - **Degrade gracefully.** If troubleshooting runs are missing, note the limited context and reason from alert patterns alone. +- **Add `$schema` when saving YAML to a file.** If the user asks to save the MaC YAML to a file, add `# yaml-language-server: $schema=https://clidocs.getmontecarlo.com/mac/schema.json` as the first line of that file.