feat(trogonstack-datadog): add dashboard naming conventions skill (#33)

Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>

Author: yordis

plugins/trogonstack-datadog/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "trogonstack-datadog",
-  "description": "Datadog observability skills for designing dashboards with proper widget selection, layout patterns, and template variables using the pup CLI",
+  "description": "Datadog observability skills for designing, redesigning, and auditing dashboards with proper widget selection, layout patterns, template variables, and operational readiness validation using the pup CLI",
   "version": "0.1.0",
   "author": {
     "name": "TrogonStack",

plugins/trogonstack-datadog/README.md

@@ -1,6 +1,6 @@
 # trogonstack-datadog
 
-Datadog observability skills for designing dashboards with proper widget selection, layout patterns, and template variables using the pup CLI.
+Datadog observability skills for designing, redesigning, and auditing dashboards with proper widget selection, layout patterns, template variables, and operational readiness validation using the pup CLI.
 
 ```bash
 claude plugin install trogonstack-datadog@trogonstack

plugins/trogonstack-datadog/skills/datadog-design-dashboard/SKILL.md

@@ -1,12 +1,15 @@
 ---
 name: datadog-design-dashboard
 description: >-
-  Design Datadog dashboards with proper widget selection, layout organization,
-  template variables, and group structure. Uses pup CLI for inspecting existing
-  dashboards and validating designs. Use when designing new dashboards, auditing
-  existing ones, or planning observability layouts for services. Do not use for:
-  (1) Datadog agent installation or configuration, (2) monitor/alert rule design,
-  (3) APM instrumentation or tracing setup, (4) log pipeline configuration.
+  Design new Datadog dashboards, redesign existing ones, or audit dashboards
+  for operational readiness. Covers widget selection, layout organization,
+  template variables, group structure, alert threshold validation, and
+  zero-knowledge readability. Uses pup CLI for inspecting dashboards and
+  validating designs. Use when designing new dashboards, redesigning existing
+  ones, auditing before on-call handoff, or reviewing after dashboard changes.
+  Do not use for: (1) Datadog agent installation or configuration,
+  (2) monitor/alert rule design, (3) APM instrumentation or tracing setup,
+  (4) log pipeline configuration.
 allowed-tools: AskUserQuestion, Write, Read, Shell
 ---
 
@@ -18,8 +21,43 @@ Design a dashboard layout that tells a clear story — from high-level health si
 
 **Philosophy**: The frameworks, layouts, and widget guides in this skill are starting points — not rigid rules. Every product and business is different. Understand the domain first, then adapt the frameworks to fit. The best dashboards reflect how the business actually works, not how a generic template says they should.
 
+---
+
+## Interview
+
+First, determine the mode:
+- **Design mode** — user wants to create or redesign a dashboard → ask design questions, then run the full workflow
+- **Audit mode** — user wants to review an existing dashboard with no intent to redesign → skip to [Audit](#audit)
+
+### Design Questions
+
+**Skip if ALL of these are already specified**: dashboard purpose, target audience, data sources, template variable needs, dashboard strategy.
+
+1. **Purpose** — "What is this dashboard for? Service overview, infrastructure, executive KPIs, debugging, or SLO tracking?"
+2. **Audience** — "Who will use this? On-call engineers, platform team, leadership, or mixed?"
+3. **Data Sources** — "Which Datadog products are involved? Metrics only, APM + Metrics, Logs + Metrics, or full stack?"
+4. **Scope** — "Is this for a single service, a group of services, or infrastructure-wide?"
+5. **Dashboard Strategy** — "One dashboard per service, or a consolidated view?" — share the trade-offs from [references/layouts.md](references/layouts.md) to help them decide. If unsure, ask: "During an outage, does your team investigate one service at a time, or do they need to see all services simultaneously?"
+6. **Existing Dashboard** — "Is there an existing dashboard to audit or redesign?" If yes, fetch with `pup dashboards get <id>` before designing.
+
+### Audit Questions
+
+**Skip if ALL of these are already specified**: dashboard ID or URL, service name or team context.
+
+**Always interview if**: No dashboard ID is provided or multiple dashboards may be relevant.
+
+1. **Dashboard** — "Which dashboard should I review? Provide a dashboard ID, URL, or service name to search for."
+2. **Business Context** — "Can you tell me what this service does for customers? Are there codebases or docs I can read to understand the product?"
+   - Impact: Understanding the domain lets the review focus on whether the right metrics are being tracked, not just whether generic rules are followed
+3. **Focus** — "Is there anything specific you want me to focus on? (A) Full review, (B) Alert thresholds only, (C) Business section, (D) Layout and readability"
+   - Default to full review if unspecified
+
+---
+
 ## Domain Discovery
 
+*Applies to design mode. Skip if auditing only.*
+
 Before designing, understand what you are building observability for. The metrics that matter depend entirely on the product and business context.
 
 **Ask the user**:
@@ -33,34 +71,36 @@ Before designing, understand what you are building observability for. The metric
 
 **Skip domain discovery if**: You already have deep context about the service from prior conversations or the user has provided detailed specifications.
 
-## Interview
-
-**Skip if ALL of these are already specified**: dashboard purpose, target audience, data sources, template variable needs, dashboard strategy.
-
-**Always interview if**: Auditing or redesigning an existing dashboard (needs current state review first).
-
-1. **Purpose** — "What is this dashboard for? Service overview, infrastructure, executive KPIs, debugging, or SLO tracking?"
-2. **Audience** — "Who will use this? On-call engineers, platform team, leadership, or mixed?"
-3. **Data Sources** — "Which Datadog products are involved? Metrics only, APM + Metrics, Logs + Metrics, or full stack?"
-4. **Scope** — "Is this for a single service, a group of services, or infrastructure-wide?"
-5. **Dashboard Strategy** — "One dashboard per service, or a consolidated view?" — share the trade-offs from [references/layouts.md](references/layouts.md) to help them decide. If unsure, ask: "During an outage, does your team investigate one service at a time, or do they need to see all services simultaneously?"
-6. **Existing Dashboard** — "Is there an existing dashboard to audit or redesign?" If yes, fetch with `pup dashboards get <id>` before designing.
-
 ---
 
-## Workflow
+## Design
+
+*Skip to [Audit](#audit) if the user only wants to review an existing dashboard.*
 
 ### 1. Gather existing context
 
 ```bash
 pup dashboards list
 pup dashboards get <dashboard-id>
-pup dashboards url <dashboard-id>
 ```
 
 If auditing an existing dashboard, fetch its definition first and analyze its current structure before redesigning.
 
-### 2. Choose a framework
+### 2. Explore available telemetry
+
+Before designing widgets, check what metrics and tag values actually exist for the service. This prevents designing around metrics that don't exist or using the wrong tag values in queries.
+
+```bash
+# See what metrics are available for the service
+pup metrics list --filter="<service-name>.*"
+
+# Verify the service tag is active and see what metrics are flowing
+pup metrics list --filter="trace.*" --tag-filter="service:<service-name>"
+```
+
+Use the actual metric names and tag values you find here when writing widget queries — do not guess or invent them. If a metric you expect does not appear, flag it to the user before building widgets around it.
+
+### 3. Choose a framework
 
 Match the dashboard purpose to a framework. Read [references/frameworks.md](references/frameworks.md) for detailed metric mappings and group structures.
 
@@ -72,22 +112,15 @@ Match the dashboard purpose to a framework. Read [references/frameworks.md](refe
 | SLO tracking | SLI/SLO |
 | Debugging | Drill-down |
 
-### 3. Design the layout
+### 4. Design the layout
 
-Using your domain understanding and the chosen framework, design the group structure and select widgets. Read these references as needed:
+Using your domain understanding and the chosen framework, design the group structure and select widgets. Read these references before designing:
 
 - **[layouts.md](references/layouts.md)** — Template variable conventions, group structure patterns, dashboard strategy trade-offs, grid sizing, anti-patterns
 - **[widgets.md](references/widgets.md)** — Widget selection guide, display options, sizing, naming conventions
 - **[thresholds.md](references/thresholds.md)** — Alert threshold markers, threshold proximity, Y-axis configuration
 
-**Key principles** (not rigid rules — use judgment):
-- **Prefix every widget title** with its layer and priority: `I0:` (most critical infra), `P0:` (most critical platform), `D0:` (most critical domain), `B0:` (most critical business). See [widgets.md](references/widgets.md) for the full prefix system.
-- Start with a **Business** group (5-8 `B`-prefixed metrics) so someone with zero service knowledge can tell if customers are affected within 5 seconds. Tailor the metrics to the domain.
-- Timeseries widgets should have **alert threshold markers** (red lines) with thresholds close to normal traffic. If a metric doesn't warrant an alert, question whether it belongs — but context-providing metrics can earn their place.
-- Set **Y-axis max** explicitly near the threshold — don't let auto-scaling compress the normal range.
-- Order groups macro-to-micro: business → overview → domain-specific → infrastructure.
-
-### 4. Write the output
+### 5. Write the design output
 
 Present the design using this template:
 
@@ -110,38 +143,188 @@ Present the design using this template:
 | ... | ... | ... | ... | ... |
 
 [Repeat for each group]
-
-## Quality Validation
-[Run quality principles below]
 ```
 
-### 5. Validate
+---
+
+## Audit
+
+Applies to both modes. Run after design, or directly if auditing an existing dashboard.
+
+The core principles are: graphs should earn their place with alert thresholds, thresholds should sit close to normal traffic, a business section should exist at the top, and the dashboard should be readable by someone with zero service knowledge.
+
+These are guiding principles — not a rigid checklist. Apply judgment based on the product and business context. A context-providing metric (like deployment events) may earn its place without a threshold. A service with unusual traffic patterns may need different proximity rules.
+
+### 1. Fetch the dashboard
 
 ```bash
+# If given a service name, list all dashboards and identify the relevant one by title
 pup dashboards list
+
+# If given a URL, extract the dashboard ID from the path (e.g., /dashboard/abc-def-ghi/...)
+
+# Get the full dashboard definition (includes the dashboard URL in the response)
 pup dashboards get <dashboard-id>
+
+# Verify real metric names exist
 pup metrics list --filter="trace.http.request.*"
 ```
 
+Parse the response to build an inventory of all widgets, groups, and their configurations.
+
+### 2. Build widget inventory
+
+Read [references/widgets.md](references/widgets.md) for the full widget prefix system before cataloging.
+
+Catalog every widget in the dashboard:
+
+| Widget Title | Prefix | Type | Group | Has Alert Threshold | Threshold Value | Notes |
+|-------------|--------|------|-------|--------------------:|----------------|-------|
+| ... | I0/P1/D0/B0/— | ... | ... | ... | ... | ... |
+
+Focus on timeseries and query value widgets — these are the primary candidates for alert threshold markers.
+
+### 3. Audit alert thresholds
+
+Read [references/thresholds.md](references/thresholds.md) for threshold marker principles, configuration details, and findings format.
+
+For each timeseries widget, check:
+- Does it have a marker/threshold line configured?
+- Is the marker colored red for visibility?
+- Does the threshold correspond to an actual monitor/alert?
+
+### 4. Audit threshold proximity
+
+Read [references/thresholds.md](references/thresholds.md) for proximity guidance, Y-axis configuration rules, and findings format.
+
+For each widget with a threshold, check:
+- What is the typical (normal) value range?
+- Where is the threshold set?
+- Is there excessive whitespace between the normal line and the alert line?
+- Is the Y-axis auto-scaled or explicitly set?
+
+### 5. Audit business section
+
+**Principle**: A dedicated Business (`B`) group should exist at the top of the dashboard with 5-8 key metrics for immediate outage identification. Business metrics are customer-visible outcomes — not infrastructure or domain internals. The specific metrics should reflect the product's business transactions, not generic traffic and error rates.
+
+Check:
+- Does a Business group exist (named "Business", "B", or equivalent)?
+- Is it the first group on the dashboard?
+- Do its widgets use the `B0-N:` prefix?
+- Does it contain 5-8 metrics covering: customer-visible success rates, key transaction flows, and SLA-impacting latency?
+- Can someone determine "are customers affected?" within 5 seconds of opening the dashboard?
+
+**Findings format**:
+
+```markdown
+#### Business Section Audit
+
+**Status**: MISSING / INCOMPLETE / OK
+
+**Current state**: [Description of what exists]
+
+**Recommended metrics** (if missing or incomplete):
+1. B0: Key transaction success rate (are critical flows completing?)
+2. B0: Customer-facing error rate (are requests failing for customers?)
+3. B1: API p99 latency (are responses slow for customers?)
+4. B1: Total request rate (are we receiving traffic?)
+5. B2: Queue depth or processing lag (is async work backing up?)
+6. B2: Key business event throughput (e.g. orders created, payments processed)
+```
+
+### 6. Apply zero-knowledge viewer test
+
+**Principle**: Someone with zero knowledge of the service should be able to spot problems by looking for red indicators.
+
+Evaluate:
+- Can you identify a problem in under 10 seconds without reading widget titles?
+- Are thresholds visible as red lines on every graph?
+- Is conditional formatting applied to query value widgets (green/yellow/red)?
+- Are group names self-explanatory?
+- Is there a note widget with runbook links or team ownership?
+
+**Findings format**:
+
+```markdown
+#### Zero-Knowledge Readability Audit
+
+| Check | Status | Finding |
+|-------|--------|---------|
+| Problems visible in <10s | FAIL | No red lines on 8 of 12 graphs |
+| Conditional formatting on QV widgets | PARTIAL | 2 of 4 QV widgets have thresholds |
+| Group names self-explanatory | OK | All groups use clear names |
+| Runbook/ownership note | MISSING | No note widget with team info |
+```
+
+### 7. Generate audit report
+
+Compile all findings into a structured report:
+
+```markdown
+# Dashboard Audit: [Dashboard Title]
+
+**Dashboard ID**: [id]
+**URL**: [url]
+**Review date**: [date]
+
+## Summary
+
+[2-3 sentence summary: overall health of the dashboard, critical issues count]
+
+## Critical Issues
+
+[List issues that must be fixed before the dashboard is production-ready]
+
+## Alert Threshold Audit
+[From step 3]
+
+## Threshold Proximity Audit
+[From step 4]
+
+## Business Section Audit
+[From step 5]
+
+## Zero-Knowledge Readability Audit
+[From step 6]
+
+## Recommended Actions
+
+### Must Fix
+1. [Action item with specific widget and group reference]
+
+### Should Fix
+1. [Action item]
+
+### Nice to Have
+1. [Action item]
+```
+
 ---
 
 ## Quality Principles
 
+- [ ] Widget queries use real metric names verified via `pup metrics list` — no invented metric names
 - [ ] Dashboard reflects the actual product and business — metrics tailored to the domain
-- [ ] Dashboard title is concise (no environment, region, or version)
-- [ ] Template variables defined for env, service, and relevant scopes (default `*`)
+- [ ] Dashboard title follows `[service] Purpose` pattern — no "Dashboard" suffix, no environment in the title
+- [ ] `title` field updated in the JSON (not just the filename) — redeploy after any title change
+- [ ] Template variables match the dashboard type — see [references/layouts.md](references/layouts.md)
+- [ ] Widget queries use template variable scopes verified via `pup metrics list` — no hardcoded env, service, or host values; use the variable set appropriate for the dashboard type (see [references/layouts.md](references/layouts.md))
 - [ ] **Business group** with 5-8 `B`-prefixed metrics tailored to the service's customer-visible outcomes
 - [ ] Groups ordered macro-to-micro (business → overview → details)
-- [ ] **Timeseries widgets have alert threshold markers** (red lines) where the metric is alertable
+- [ ] **Every widget title uses the layer-priority prefix** (`I0:`, `P1:`, `D0:`, `B0:`, etc.) — see [references/widgets.md](references/widgets.md)
+- [ ] Widget titles use sentence case, don't repeat group name
+- [ ] **Timeseries widgets have alert threshold markers** (red lines) where the metric is alertable — see [references/thresholds.md](references/thresholds.md)
 - [ ] **Thresholds close to normal traffic** — no excessive whitespace
 - [ ] **Zero-knowledge readability** — someone with no service knowledge can spot problems via red indicators
-- [ ] **Widget titles prefixed** with layer and priority (`I0:`, `P1:`, `D0:`, `B0:`, etc.)
-- [ ] Widget titles use sentence case, don't repeat group name
+- [ ] Query Value widgets have conditional formatting (green/yellow/red)
 - [ ] Every metric earns its place — if it spikes, someone can act on it
+- [ ] All audit findings include specific widget names and group references
+- [ ] Recommended actions categorized by priority (must/should/nice-to-have)
+- [ ] Dashboard URL included in audit report
 
 ## References
 
 - **[Observability Frameworks](references/frameworks.md)** — RED, USE, Golden Signals, SLI/SLO with metric mappings
 - **[Layout & Structure](references/layouts.md)** — Template variables, group patterns, dashboard strategy, grid sizing, anti-patterns
-- **[Widgets](references/widgets.md)** — Widget types, display options, sizing, naming conventions
+- **[Widgets](references/widgets.md)** — Widget prefix system, types, display options, sizing, naming conventions
 - **[Alert Thresholds](references/thresholds.md)** — Threshold markers, proximity guide, Y-axis configuration

plugins/trogonstack-datadog/skills/datadog-design-dashboard/references/layouts.md

@@ -8,17 +8,26 @@ Template variable conventions, group structure patterns, dashboard strategy trad
 
 Template variables make one dashboard serve many contexts. Define them before laying out widgets.
 
-**Standard variables** (include when applicable):
+**`env` is always required.** The remaining variables depend on the dashboard type:
+
+| Dashboard Type | Template Variables |
+|---------------|-------------------|
+| Service Overview (RED), Debugging | `env` · `service` |
+| Infrastructure (USE) | `env` · `host` · `availability_zone` |
+| Executive / Golden Signals | `env` · `team` · `region` |
+
+**Optional variables** (add when relevant to the dashboard type):
 
 | Variable | Tag | Use Case |
 |----------|-----|----------|
-| `env` | `env` | Environment filtering (prod, staging, dev) |
-| `service` | `service` | Service scoping |
 | `region` | `region` | Regional filtering |
 | `availability_zone` | `availability_zone` | AZ-level drill-down |
 | `host` | `host` | Host-level investigation |
+| `endpoint` | `http.url` | Per-route investigation (Debugging dashboards) |
 
-- Start with the broadest scope (`env`) and narrow down
+- For **service dashboards** (RED, Debugging): scope every widget query with `{$service,$env}`
+- For **infrastructure dashboards**: scope queries with `{$host,$env}` or `{$availability_zone,$env}` instead
+- For **executive dashboards**: scope queries with `{$team,$env}` or `{$region,$env}` instead
 - Use `*` as the default value so dashboards load with full scope
 - Never put environment or region in the dashboard title — that is what template variables are for
 - Name variables after the tag they filter on