Skip to content

Commit d8eb8df

Browse files
style: apply /style-guide pass to models/automations (#2684)
## Summary This PR applies the `/style-guide` skill (Google Developer Style Guide + CoreWeave conventions) to the `models/automations` section. The run was fully automated — no technical content changes were intended. ## Files edited - `models/automations/api.mdx` - `models/automations/automation-events.mdx` - `models/automations/create-automations.mdx` - `models/automations/create-automations/slack.mdx` - `models/automations/create-automations/webhook.mdx` - `models/automations/project-automation-tutorial.mdx` - `models/automations/registry-automation-tutorial.mdx` - `models/automations/tutorial.mdx` - `models/automations/view-automation-history.mdx` ## Recommendations for technical review ### Prerequisites - Consolidate implicit role and permissions prerequisites (team admin, Registry admin, W&B admin, IAM, Python tooling/`wandb login`) into explicit **Prerequisites** sections in `create-automations.mdx`, `create-automations/webhook.mdx`, `project-automation-tutorial.mdx`, `registry-automation-tutorial.mdx`, and `view-automation-history.mdx`. - In `create-automations.mdx`, confirm the role-requirement asymmetry between the Registry tab (no admin restatement) and the Project tab ("W&B admin" restated) for view/manage flows. - In `tutorial.mdx`, surface shared prerequisites (W&B account, project, Slack workspace, webhook endpoint) on the hub page instead of only inside each sub-tutorial. - In `automation-events.mdx:34`, consider a prerequisite or cross-reference for readers landing directly on the Registry webhook section. - In `create-automations/webhook.mdx`, consolidate the implicit prereqs (roles, an existing W&B secret, access to **Team Settings**) into a single block. - In `tutorial.mdx`, surface the **Enterprise Cloud** tier requirement alongside the **Requirements** list rather than only in the `EnterpriseCloudOnly` callout. - In `registry-automation-tutorial.mdx`, flag a minimum `wandb` Python SDK version for `api.artifact_collection`, `collection.versions()`, and `version.aliases.append`. - In `view-automation-history.mdx:12`, verify the `W&B Self-Managed v0.75.0` minimum version is still accurate and consider moving it into a Prerequisites section; also document which roles can view automation history. ### Verification steps - Add expected-outcome cues across procedures so readers can confirm success: where a new automation appears in the UI after **Create automation** (`create-automations.mdx`, `create-automations/webhook.mdx`, `tutorial.mdx`), what a successful webhook test looks like (`create-automations/webhook.mdx`), and what the test Slack message contains (sender, formatting, fields) in `project-automation-tutorial.mdx`. - In `api.mdx`, the Note says `list`, `get`, and `delete` "might still work" — add a minimal example or link into the Python API reference so readers can attempt them. - In `automation-events.mdx:165-167`, add an example or expected-behavior description for run-level evaluation (per-run windows, max once per 24 hours) so users can verify firing. - In `create-automations.mdx`, verify the literal Slack handoff message (`[YOUR-SLACK-HANDLE] added an integration to this channel: Weights & Biases`) against actual Slack output. - In `create-automations/webhook.mdx`, clarify what a successful vs. failed response from the W&B-posted response looks like in the troubleshooting flow. - In `registry-automation-tutorial.mdx`, point readers to an in-product verification path (e.g., the **Automations** tab run history) and add brief troubleshooting guidance for common failure modes (alias regex mismatch, webhook auth failure, scope confusion). - In `view-automation-history.mdx`, walk both Tab procedures (Registry and Project) in the UI to confirm click paths, tab names ("Automations history" vs. "History"), the **Last execution** column label, the `collection-automation-execution-icon.png` icon, and whether the Registry **Last execution** timestamp navigates to the **Automations history** tab or a separate detail view. Also confirm whether the single Project-tab step should be split. ### Technical accuracy - In `api.mdx`, confirm the Note's framing (create/update fail due to client-side feature check) still matches engineering guidance, that `WB-34263` is the correct internal reference, and whether exposing that internal Jira ID publicly is acceptable. - In `automation-events.mdx:57-60`, decide whether the `MultiTenantCloudOnly` availability notice and the **Killed** behavioral caveat belong in the same `<Note>` or should be split. - In `automation-events.mdx:75`, confirm whether the "every 15 seconds" system metrics interval is configurable or version-dependent and consider linking the system metrics reference. - In `automation-events.mdx:109`, consider a diagram or worked example showing how the current and prior windows align. - In `create-automations.mdx`, confirm the side effects on dependent automations when a Slack destination is deleted, and document whether the run-filter username field accepts entity usernames vs. display names. - In `create-automations/webhook.mdx`, validate the rewording of "the webhook's service determines your webhook's requirements"; confirm that nested artifact metadata is genuinely unsupported via `${artifact_metadata.<KEY>}`; decide between "GHA workflow" and the expanded "GitHub Actions workflow"; confirm **Bot User OAuth Token** is still the recommended secret type for the legacy Slack webhook flow and whether it should be bolded as a UI label; confirm whether the Bash shell script accurately mirrors W&B's actual `POST` or should be flagged as an approximation. - In `project-automation-tutorial.mdx`, verify whether the commented-out Python tab (blocked by WB-34263) should be restored, removed, or updated, and confirm the regression is still open. - In `registry-automation-tutorial.mdx`, confirm whether the **Alias regex** field accepts true regex or a literal/glob (the example `production` is ambiguous); confirm whether registry-wide scope is achievable via the Python SDK (vs. the commented `OnAddArtifactAlias(scope=collection, ...)`); verify that `next(collection.versions())` yields the newest version first; and confirm `run.wait()` is still the recommended pattern vs. `artifact.wait()` or context-manager finalization. - In `tutorial.mdx`, decide whether the unused imports `TutorialDiagramProject` and `TutorialDiagramRegistry` should be rendered or removed. - In `view-automation-history.mdx:21`, confirm the `#execution-status` anchor target is intended (vs. `#execution-details`); in line 51, confirm whether a **Failed** automation always halts entirely or may have partially completed actions; in lines 50–52, confirm Finished/Failed/Pending are the complete status set (no Running, Cancelled, Retrying); in line 59, verify event-type strings ("New artifact version", "Run completed") match exact UI labels. ### Missing content - In `api.mdx`, add a one-sentence definition of "W&B automations" rather than relying on the overview link, and confirm the hidden HTML comment's restoration plan (list/get/create/update/delete examples + Automations tutorial notebook from `wandb/examples` PR 618) is still accurate; if the SDK fix is imminent, consider a temporary redirect to `/models/automations/create-automations`. - In `automation-events.mdx`, document boundary behavior during the change-threshold warm-up before the prior window is fully populated (line 104), and confirm whether the z-score window has a minimum run count before evaluation begins (line 131). - In `create-automations.mdx`, preview the nested numbered list in the Project tab (lines 80–86) to confirm Mintlify renders sub-steps correctly; if not, flatten to a single numbered list with embedded prose. - In `create-automations/webhook.mdx`, re-validate the cross-reference in the Secret step that points to a "next step" about specifying the secret; confirm the Slack notifications "new Slack integration" link target `/models/automations/create-automations/slack`; reformat YAML (`on/repository_dispatch`) and JSON (`client_payload`) examples for consistent indentation; convert angle-bracket placeholders (`<wandb-user>`, `<entity>`, `<registered_model_name>`, `<alias>`) in the template-string output block to the `[ALL-CAPS-HYPHENATED]` convention; and flatten the four-level-deep nesting under "Choose the event to watch for." - In `project-automation-tutorial.mdx`, add a brief link or note distinguishing "failed" runs from other terminal states (crashed, killed) since the filter depends on it. - In `registry-automation-tutorial.mdx`, add **Clean up** or **Next steps** with teardown guidance (removing the test alias, disabling the automation); track restoring the commented Python tab once WB-34263 ships; confirm whether the existing payload-variables link is sufficient. - In `tutorial.mdx`, verify that the `<AutomationsMentalModel/>` snippet provides sufficient framing; if not, add a brief "Who this is for" or "Before you begin" pointer on the hub. - In `view-automation-history.mdx`, document conditions/typical duration for the **Pending** status (and what to do if stuck); clarify which event types include user attribution for the "User who triggered the event (if applicable)" line; add a parent description sentence for **Result information** for parallel structure; document the retention period for execution history; add guidance (link to troubleshooting or retry) for failed executions; and confirm or replace the **Next steps** link to "Create a secret," which is not clearly related to automation history. ## How to review - Each file's changes are style edits only. Compare side-by-side and flag any that change technical meaning. - Approve and merge to accept the edits, or close to reject them.
1 parent f516163 commit d8eb8df

9 files changed

Lines changed: 213 additions & 139 deletions

File tree

models/automations/api.mdx

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,19 @@
11
---
22
title: Manage automations with the API
3-
description: Programmatic automation management with the Python API (create and update may be affected on some client versions; prefer the W&B App until the SDK fix ships).
3+
description: Programmatic automation management with the Python API. Create and update may be affected on some client versions. Prefer the W&B App until the SDK fix ships.
4+
keywords: ["Python API", "programmatic automation", "wandb.Api", "automation scripting", "SDK"]
45
---
56

7+
Manage W&B automations programmatically with the `wandb` Python API when you script automation workflows instead of using the W&B App.
8+
69
<Note>
7-
Programmatic **create** and **update** for automations through `wandb.Api().create_automation()` and related helpers can fail on recent `wandb` Python clients because of a client-side feature check (for example, run-state automations). Use the [W&B App](/models/automations/create-automations) to create and edit automations until a fixed SDK ships. List, get, and delete may still work for your version; if you hit errors, rely on the App. Internal tracking: WB-34263.
10+
Programmatic **create** and **update** for automations through `wandb.Api().create_automation()` and related helpers can fail on some `wandb` Python client versions because of a client-side feature check (for example, run-state automations). Until a fixed SDK ships, use the [W&B App](/models/automations/create-automations) to create and edit automations. The `list`, `get`, and `delete` methods aren't affected. Internal tracking: WB-34263.
811
</Note>
912

1013
## Next steps
1114

15+
Refer to the following resources to manage automations and learn more about how they work:
16+
1217
- [Automations overview](/models/automations)
1318
- [Create an automation](/models/automations/create-automations) (W&B App)
1419
- [Automation events and scopes](/models/automations/automation-events)

models/automations/automation-events.mdx

Lines changed: 45 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
title: Automation events and scopes
33
description: "Learn about events and scopes that trigger W&B Automations, including artifact changes, run status, and metric conditions."
4+
keywords: ["automation triggers", "artifact alias", "run status", "metric threshold", "z-score"]
45
---
56
import EnterpriseCloudOnly from "/snippets/_includes/enterprise-cloud-only.mdx";
67
import MultiTenantCloudOnly from "/snippets/_includes/multi-tenant-cloud-only.mdx";
@@ -9,41 +10,48 @@ import MultiTenantCloudOnly from "/snippets/_includes/multi-tenant-cloud-only.md
910
<EnterpriseCloudOnly/>
1011
</Info>
1112

12-
An automation can start when a specific event occurs within a project or registry. This page describes the events that can trigger an automation within each scope. Learn more about automations in the [Automations overview](/models/automations) or [Create an automation](/models/automations/create-automations).
13+
An automation can start when a specific event occurs within a project or registry. This page describes the events that can trigger an automation within each scope, so you can choose the right trigger when you configure an automation. Learn more about automations in the [Automations overview](/models/automations) or [Create an automation](/models/automations/create-automations).
1314

1415
## Registry
15-
This section describes the scopes and events for an automation in a [Registry](/models/registry).
16+
17+
The following sections describe the scopes and events for an automation in a [Registry](/models/registry).
1618

1719
### Scopes
20+
1821
A [Registry](/models/registry) automation watches for the event taking place on any collection within a specific registry, including collections added in the future.
1922

2023
### Events <a id="registry-events" aria-label="Registry events"></a>
24+
2125
A Registry automation can watch for these events:
22-
- **A new version is linked to a collection**: Test and validate new models or datasets when they are added to a registry.
26+
- **A new version is linked to a collection**: Test and validate new models or datasets when they're added to a registry.
2327
- **An artifact alias is added**: Trigger a specific step of your workflow when a new artifact version has a specific alias applied. For example, deploy a model when it has the `production` alias applied.
2428

2529
When the automation calls a webhook, it can access the same team-level webhook configurations and [team secrets](/platform/secrets) as project-scoped automations.
2630

2731
## Project
28-
This section describes the scopes and events for an automation in a [project](/models/track/project-page).
32+
33+
The following sections describe the scopes and events for an automation in a [project](/models/track/project-page).
2934

3035
### Scopes
36+
3137
A project-level automation watches for the event taking place on any collection in the project. Depending on the event you specify, you can further limit the scope of the automation.
3238

3339
### Artifact events
40+
3441
This section describes the events related to an artifact that can trigger an automation.
3542

3643
- **A new version is added to an artifact**: Apply recurring actions to each version of an artifact. For example, start a training job when a new dataset artifact version is created.
37-
- **An artifact alias is added**: Trigger a specific step of your workflow when a new artifact version in a project has an alias applied that matches the **Alias regex** you specify. For example, run a series of downstream processing steps when an artifact has the `test-set-quality-check` alias applied, or run a workflow each time a new artifact version has the `latest` alias. Only one artifact version can have a given alias at a point in time.
38-
- **An artifact tag is added**: Trigger a specific step of your workflow when an artifact version in a project has a tag applied that matches the **Tag regex** you specify. For example, specify `^europe.*` to trigger a geo-specific workflow when a tag beginning with the string `europe` is added to an artifact version. Artifact tags are used for grouping and filtering, and a given tag can be assigned to multiple artifact versions simultaneously.
44+
- **An artifact alias is added**: Trigger a specific step of your workflow when a new artifact version in a project has an alias applied that matches the **Alias regex** you specify. For example, run a series of downstream processing steps when an artifact has the `test-set-quality-check` alias applied, or run a workflow each time a new artifact version has the `latest` alias. Only one artifact version can have a given alias at a time.
45+
- **An artifact tag is added**: Trigger a specific step of your workflow when an artifact version in a project has a tag applied that matches the **Tag regex** you specify. For example, specify `^europe.*` to trigger a geo-specific workflow when a tag beginning with the string `europe` is added to an artifact version. Use artifact tags for grouping and filtering. You can assign the same tag to multiple artifact versions.
3946

4047
### Run events
41-
An automation can be triggered by a change in a [run's status](/models/runs/run-states) or a change in a [metric value](/models/track/log#what-data-is-logged-with-specific-wb-api-calls).
48+
49+
The following sections describe how to configure an automation that starts based on a change in a [run's status](/models/runs/run-states) or a change in a [run's metric value](/models/track/log#what-data-is-logged-with-specific-wb-api-calls).
4250

4351
#### Run status change
4452
<Note>
4553
- <MultiTenantCloudOnly/>
46-
- A run with **Killed** status cannot trigger an automation. This status indicates that the run was stopped forcibly by an admin user.
54+
- A run with **Killed** status can't trigger an automation. This status indicates that an admin forcibly stopped the run.
4755
</Note>
4856

4957
Trigger a workflow when a run changes its [status](/models/runs/run-states) to **Running**, **Finished**, or **Failed**. Optionally, you can further limit the runs that can trigger an automation by specifying a user or run name filter.
@@ -63,12 +71,13 @@ Trigger a workflow based on a logged value for a metric, either a metric in a ru
6371

6472
You can create a run metrics automation from the project's **Automations** tab or directly from a line plot panel in a workspace.
6573

66-
To set up a run metric automation, you configure how to compare the metric's value with the threshold you specify. Your choices depend on the event type and on any filters you specify.
74+
To set up a run metric automation, configure how to compare the metric's value with the threshold you specify. Your choices depend on the event type and on any filters you specify.
6775

6876
Optionally, you can further limit the runs that can trigger an automation by specifying a user or run name filter.
6977

7078
##### Threshold
71-
For **Run metrics threshold met** events, you configure:
79+
80+
Use a threshold event to start an automation when a metric crosses a fixed value. For **Run metrics threshold met** events, configure:
7281
1. The window of most recently logged values to consider (defaults to 5).
7382
1. Whether to evaluate the **Average**, **Min**, or **Max** value within the window.
7483
1. The comparison to make:
@@ -79,21 +88,22 @@ For **Run metrics threshold met** events, you configure:
7988
- Not equal to
8089
- Equal to
8190

82-
For example, trigger an automation when average `accuracy` is above `.6`.
91+
For example, trigger an automation when average `accuracy` is above `0.6`.
8392

8493
<Frame>
8594
![Screenshot showing a run metrics threshold automation](/images/automations/run_metrics_threshold_automation.png)
8695
</Frame>
8796

8897
##### Change threshold
89-
For **Run metrics change threshold met** events, the automation uses two "windows" of values to check whether to start:
98+
99+
Use a change threshold event to start an automation when a metric shifts between two recent windows of values. For **Run metrics change threshold met** events, the automation uses two "windows" of values to check whether to start:
90100

91101
- The _current window_ of recently logged values to consider (defaults to 10).
92102
- The _prior window_ of recently logged values to consider (defaults to 50).
93103

94-
The current and prior windows are consecutive and do not overlap.
104+
The current and prior windows are consecutive and don't overlap.
95105

96-
To create the automation, you configure:
106+
To create the automation, configure:
97107
1. The current window of logged values (defaults to 10).
98108
1. The prior window of logged values (defaults to 50).
99109
1. Whether to evaluate the values as relative or absolute (defaults to **Relative**).
@@ -102,7 +112,7 @@ To create the automation, you configure:
102112
- Decreases by at least
103113
- Increases or decreases by at least
104114

105-
For example, trigger an automation when average `loss` decreases by at least `.25`.
115+
For example, trigger an automation when average `loss` decreases by at least `0.25`.
106116

107117
<Frame>
108118
![Screenshot showing a run metrics change threshold automation](/images/automations/run_metrics_change_threshold_automation.png)
@@ -113,41 +123,49 @@ For example, trigger an automation when average `loss` decreases by at least `.2
113123
<MultiTenantCloudOnly/>
114124
</Note>
115125

116-
W&B can trigger an automation when a metrics z-score (standard score) exceeds a specified threshold. A z-score measures how many standard deviations the value is from the mean for that metric across a configurable window of runs in the project (defaults to 30 runs).
126+
W&B can trigger an automation when a metric's z-score (standard score) exceeds a specified threshold. A z-score measures how many standard deviations the value is from the mean for that metric across a configurable window of runs in the project (defaults to 30 runs).
117127

118128
To use a z-score as an event trigger, select the **Run metrics z-score threshold met** event.
119129

120130
Automations based on z-score keep your team informed about unusual performance without checking absolute thresholds, which may vary as your model or training process evolves.
121131

122132
You can create a run metrics z-score automation from the project's **Automations** tab or directly from a line plot panel in a workspace.
123133

124-
To create a z-score automation, you configure:
134+
To create a z-score automation, configure:
125135
1. The target z-score threshold, expressed as a positive float value (for example, 2.0).
126136
1. The window of logged values that determine the mean value (defaults to 30).
127137
1. The comparison to make:
128-
- Above (trigger when performance is unusually high)
129-
- Below (trigger when performance is unusually low)
130-
- Either above or below
138+
- Above (triggers when performance is unusually high).
139+
- Below (triggers when performance is unusually low).
140+
- Either above or below.
131141

132-
For example, trigger an automation when `accuracy` has a z-score above 2, which means the run is performing significantly better than other runs in the project.
142+
For example, trigger an automation when `accuracy` has a z-score above 2, which means the run is performing well above other runs in the project.
143+
144+
Z-score values have the following meanings:
133145

134-
**Understanding z-score values:**
135146
- A z-score of 0 means the metric is at the average.
136147
- A z-score of +2.0 means the metric is 2 standard deviations above average.
137148
- A z-score of -2.0 means the metric is 2 standard deviations below average.
138149
- Values beyond ±2 are often considered statistically significant outliers.
139150

140151
#### Run filters
152+
141153
This section describes how the automation selects runs to evaluate.
142154

143-
- By default, any run in the project triggers the automation when the event occurs. You can limit which runs trigger an automation by configuring one of the following filters:
144-
- **Filter to one user's runs**: Include only runs created by the specified user.
145-
- **Filter on run name**: Include only runs whose names match the given regular expression.
155+
By default, any run in the project triggers the automation when the event occurs. You can limit which runs trigger an automation by configuring one of the following filters:
156+
157+
| Filter | Description |
158+
|--------|-------------|
159+
| **Filter to one user's runs** | Include only runs created by the specified user. |
160+
| **Filter on run name** | Include only runs whose names match the given regular expression. |
161+
162+
The automation evaluates each run as follows:
146163

147-
For details, see [Create automations](/models/automations/create-automations).
148164
- Each run is considered individually and can potentially trigger the automation.
149165
- Each run's values are put into a separate window and compared to the threshold separately.
150-
- In a 24 hour period, a particular automation can fire at most once per run.
166+
- In a 24-hour period, a particular automation can fire at most once per run.
167+
168+
For details, see [Create automations](/models/automations/create-automations).
151169

152170
## Next steps
153171
- [Create a Slack automation](/models/automations/create-automations/slack)

0 commit comments

Comments
 (0)