docs(skills): adopt create_or_update_*_monitor tools and preview-then-confirm flow#85
Merged
Merged
Conversation
…-confirm flow
ai-agent PR #998 renames the MaC creation tools to
create_or_update_{table,metric,validation,sql,comparison}_monitor and
makes dry_run=True the default. Update the monitoring-advisor and
tune-monitor skills (plus claude-code eval YAMLs) to reflect the new
names and the two-call flow, document monitor_uuid for in-place
updates, and the dry_run=False live-deploy response shape.
The direct-GraphQL create_*_monitor tools used by tune-monitor are now
EXTENDED_ONLY (hidden from the public default toolset). Migrate
tune-monitor's apply step to create_or_update_*_monitor with
monitor_uuid for update-in-place. Bumps monitoring-advisor to 2.1.0 and
tune-monitor to 1.1.0.
…ith monitor_uuid Follow-up to ai-agent PR #998 commit c2f451c, which clarifies that passing monitor_uuid to create_or_update_*_monitor has PUT semantics — the call fully replaces the monitor's configuration, and fields you omit revert to tool defaults rather than being left untouched. Adds the safe-edit recipe (read current config with get_monitors(..., include_fields=["config"]), re-pass every kept field, diff the dry_run YAML against the original) to: - monitoring-advisor's data-monitor-creation.md (intro + Step 7) - monitoring-advisor per-type references — augments the monitor_uuid row in the parameters table for all five monitor types - tune-monitor references (metric, custom-sql, validation) — strengthens the "Common mistakes" section and cross-references the Phase 1 config fetch as the source of truth for the re-pass Bumps monitoring-advisor to 2.1.1 and tune-monitor to 1.1.1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
create_*_monitor_macreference in skill docs to the newcreate_or_update_{table,metric,validation,sql,comparison}_monitortools introduced in ai-agent#998.dry_run=True(the default) returns rendered MaC YAML inresult.yaml;dry_run=Falseactually creates/updates the monitor and returnsresult.monitor_uuid+ a deep link inresult.instructions(withresult.yaml=Noneby design).monitor_uuidfor update-in-place,is_draftfor draft mode, and the parity params (schedule_type,interval_minutes,audiences,failure_audiences,notes,priority,tags, plus per-type extras likeaggregate_time_sql/segment_sql/sensitivity/collection_lag_hourson metric,query_result_type/custom_sampling_sql/variable_definitionson SQL).create_metric_monitor/create_custom_sql_monitor/create_validation_monitortools (hidden from the public default toolset) ontocreate_or_update_*_monitorwithmonitor_uuidfor in-place updates.plugins/claude-code/evals/{monitoring-advisor,prevent}/live-evals-*.yamland the eval README example to assert the new tool names inmust_call/must_not_call.monitoring-advisorto 2.1.0 andtune-monitorto 1.1.0.Plugin-level CHANGELOGs are left for the next release cut (no Unreleased section convention in this repo).
Test plan
monitoring-advisorlive evals (plugins/claude-code/evals/monitoring-advisor/live-evals-{dev,prod}.yaml) against the updated dev MCP server — confirms the new tool names match what the server actually exposes in the default toolset.preventlive eval (live-04-monitor-delegation) and confirm the turn-2 trace shows one of the newcreate_or_update_*_monitorcalls.dry_run=Truereturns YAML,dry_run=Falseupdates in place and returns the deep link.🤖 Generated with Claude Code