Skip to content

Commit 5a6f612

Browse files
lukegalbraithrussellmwbrooksWilliamBergaminClaude
authored
feat: adds docs skill (#55)
* initial draft * Apply suggestions from code review Co-authored-by: William Bergamin <william.bergamin.coen@gmail.com> * feat: Use category filter in slack-docs skill discovery The docs search endpoint now supports a `category` param. Lead with it in Step 1 rather than treating it as optional: an uncategorized search skews heavily toward SDK reference pages (queries like "socket mode" or "oauth" can return a top 10 that is entirely Bolt/Node pages), burying the conceptual and reference content most questions are about. - Step 1 (discovery): make `category` the default path, with guidance on picking the starting category (guides for conceptual, reference for a named method/event/scope, a tool category once the SDK is known) and how to widen the search when a categorized query misses. - Step 3 (tool/SDK scoping): map each tool to its matching `category` so the agent can scope discovery to a language family. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com> * refactor: Consolidate page-fetch logic into Step 2 of slack-docs Make Step 2 the single home for the page-reference → .md → fetch → cite routine, accepting both a site-relative path from search and a full URL the developer pasted. The Fast Path now hands off to Step 2 instead of duplicating the append-.md/WebFetch logic. Less to maintain, and Step 2 now covers the anchor-stripping case that only the Fast Path mentioned before. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com> * fix: resolve slack-cli/slack-docs doc-search overlap Route general documentation questions to the slack-docs skill and scope slack-cli's docs search to terminal-based lookups, removing the overlap between the two skill descriptions. Update the tool-selection eval to match the redrawn boundary: the doc-search scenario now expects slack-docs, plus new scenarios for a conceptual docs question, a docs.slack.dev URL fetch, and a slack-cli local-run to guard slack-cli routing. Also remove the orphaned tests/eval/skills/test_slack_docs.py (referenced a nonexistent Ollama harness and never imported) and add the missing code-fence language that was failing rumdl lint. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com> * Apply suggestions from code review Co-authored-by: William Bergamin <william.bergamin.coen@gmail.com> * Apply suggestion from @lukegalbraithrussell --------- Co-authored-by: Michael Brooks <mbrooks@slack-corp.com> Co-authored-by: William Bergamin <wbergamin@salesforce.com> Co-authored-by: William Bergamin <william.bergamin.coen@gmail.com> Co-authored-by: Claude <svc-devxp-claude@slack-corp.com>
1 parent e75b0cf commit 5a6f612

5 files changed

Lines changed: 151 additions & 8 deletions

File tree

‎.changeset/clarify-docs-search-boundary.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"slack": minor
3+
---
4+
5+
Route general Slack documentation questions to the slack-docs skill. The slack-cli skill's docs search is now scoped to terminal-based lookups via `slack docs search`, removing the overlap with slack-docs.

‎skills/slack-cli/SKILL.md‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: slack-cli
3-
description: Use the Slack CLI to create, run, and manage Slack apps from the terminal. Use whenever the developer wants to log in, add a team, switch workspaces, or authenticate with Slack; whenever Slack CLI commands are needed (local development with `slack run`, managing app lifecycle, the manifest); or when searching the Slack developer documentation for any topic (socket mode, the Events API, OAuth, manifests, Bolt).
3+
description: Use the Slack CLI to create, run, and manage Slack apps from the terminal. Use whenever the developer wants to log in, add a team, switch workspaces, or authenticate with Slack; whenever Slack CLI commands are needed (local development with `slack run`, managing app lifecycle, the manifest).
44
---
55

66
# Slack CLI
@@ -90,12 +90,11 @@ Search Slack's developer documentation directly from the terminal.
9090
SLACK_CMD docs search "<query>" --output=text --limit=5
9191
```
9292

93-
Use `--output=text` for concise terminal-readable results. Use this:
93+
Use `--output=text` for concise terminal-readable results. Use this when you are already running CLI commands and want to:
9494

95-
- Before implementing a Slack feature you have not used before
96-
- When the developer asks "how does X work in Slack?"
97-
- To verify API behavior or required scopes
98-
- To look up Block Kit elements, event types, or method parameters
95+
- Check a Slack feature before implementing it
96+
- Verify API behavior or required scopes
97+
- Look up Block Kit elements, event types, or method parameters
9998

10099
---
101100

‎skills/slack-docs/SKILL.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
name: slack-docs
3+
description: "Search and read the official Slack platform documentation at docs.slack.dev. Use this skill to answer conceptual or how-to questions about Slack features. You can also use it to look up, fetch, or summarize specific guide pages from provided docs.slack.dev links."
4+
argument-hint: "[topic or docs.slack.dev URL]"
5+
---
6+
7+
# Slack Platform Documentation
8+
9+
Help the developer **find** the right page on the official Slack documentation site (`https://docs.slack.dev`) and **read** it as clean markdown, so answers come from the live docs rather than memory. The site exposes three machine-readable surfaces an agent can use directly, with no authentication and no Slack workspace:
10+
11+
- A **search API method**: `GET https://docs.slack.dev/api/v1/search?query=<q>&category=<c>` returns ranked page hits as JSON. Scope every search with a `category` (guides, reference, a specific SDK, etc.); an uncategorized search skews heavily toward SDK pages.
12+
- **Per-page markdown**: every page is available at its URL **+ `.md`** (e.g. `/quickstart.md`).
13+
- **Index files**: `/llms.txt` (a curated overview) and `/llms-sitemap.md` (a list of every markdown page).
14+
15+
If `$0` is provided, it is either a `docs.slack.dev` URL (jump to the **Fast Path**) or a topic to search (start at **Step 1**). For a broad "how do I build X on Slack?" question rather than one specific page, read `https://docs.slack.dev/llms.txt` first; it is a curated, LLM-oriented overview of the platform and the recommended build path.
16+
17+
> **Critical rules:**
18+
>
19+
> - The docs are the **source of truth**. Do not answer a factual question about the Slack platform from memory; discover the page, fetch it, then answer from what it says.
20+
> - Prefer the **`.md` version** of any page over the HTML version. It is cleaner for reading and quoting.
21+
> - Every fetched markdown page begins with a `Source: <url>` line. Keep that URL so you can cite the page back to the developer.
22+
23+
> **DO NOT rules:**
24+
>
25+
> - DO NOT invent documentation URLs. Get them from the search API, the sitemap, or a link the developer gave you then verify by fetching.
26+
> - DO NOT paraphrase a page you have not actually fetched. If a fetch fails, say so rather than filling the gap from memory.
27+
> - DO NOT assume every URL has a `.md` version. If a fetch returns an error, fall back to the search API or the sitemap (Step 1) rather than guessing another URL.
28+
29+
---
30+
31+
## Fast Path (the developer already has a URL)
32+
33+
If the developer pasted a `https://docs.slack.dev/...` link, skip discovery and go straight to **Step 2** to fetch and read it.
34+
35+
---
36+
37+
## Step 1: Discover the Page (search)
38+
39+
Use the docs **search API** to find candidate pages. WebFetch this URL, with the query URL-encoded and a `category` to scope the results:
40+
41+
```text
42+
https://docs.slack.dev/api/v1/search?query=<url-encoded query>&category=<category>&limit=5
43+
```
44+
45+
**Always start with a `category`.** An uncategorized search is dominated by SDK reference pages (a query like `socket mode` or `oauth` can return a top 10 that is entirely Bolt/Node pages), which buries the conceptual and reference content most questions are actually about. Pick the starting category from the developer's intent:
46+
47+
- **`guides`** — the default for "how do I…", "what is…", and conceptual platform questions (Events API, OAuth, Socket Mode, manifests, modals, App Home). Start here when in doubt.
48+
- **`reference`** — for a specific method, event, scope, object, or Block Kit element, especially an exact name like `chat.postMessage`.
49+
- **A tool/SDK category** (`python`, `javascript`, `java`, `slack_cli`, `slack_github_action`, `deno_slack_sdk`) — only once you know the developer's tool. See **Step 3**.
50+
51+
So `socket mode` as a concept → `https://docs.slack.dev/api/v1/search?query=socket%20mode&category=guides&limit=5`.
52+
53+
The response is JSON:
54+
55+
```json
56+
{
57+
"total_results": 12,
58+
"results": [
59+
{ "url": "/apis/events-api/using-socket-mode", "title": "Using Socket Mode" }
60+
],
61+
"limit": 5
62+
}
63+
```
64+
65+
Scan `results` for the best `title`/`url` match, then read it in **Step 2**. A query is **required**; calling the endpoint with no `query` returns a `400` with an `error` field.
66+
67+
Full set of categories:
68+
69+
| Category | Scopes results to |
70+
|----------|-------------------|
71+
| `guides` | Conceptual and how-to guides |
72+
| `reference` | API reference: methods, events, scopes, objects, Block Kit |
73+
| `changelog` | Changelog and release notes |
74+
| `python` | Python tools (Bolt for Python, Python Slack SDK) |
75+
| `javascript` | JavaScript tools (Bolt for JS, Node Slack SDK) |
76+
| `java` | Java tools (Bolt for Java, Java Slack SDK) |
77+
| `slack_cli` | Slack CLI docs |
78+
| `slack_github_action` | Slack Send GitHub Action docs |
79+
| `deno_slack_sdk` | Deno Slack SDK docs |
80+
81+
If a categorized search returns no good hit, widen it: try the other likely category (`guides` ⇄ `reference`), then drop `category` entirely as a last resort. An unrecognized value returns a `400` with an `error` field listing the valid categories, so re-run with one of those or with no category.
82+
83+
**Fallbacks** when search does not surface a good hit, or returns a `500`/temporary error (the endpoint is rate-limited and cached ~5 minutes):
84+
85+
- WebFetch `https://docs.slack.dev/llms-sitemap.md`, a flat list of every documentation page's `.md` URL, and scan it for the relevant path.
86+
- For API reference lookups, the enriched index pages are often faster: `https://docs.slack.dev/reference/methods.md`, `.../events.md`, `.../scopes.md`, `.../objects.md`, and `.../block-kit.md` each list every item with a one-line description and a `.md` link.
87+
- If the developer has the Slack CLI, `slack docs search "<query>"` does the same discovery from the terminal (see the `slack:slack-cli` skill).
88+
89+
---
90+
91+
## Step 2: Read the Page (fetch markdown)
92+
93+
Given a page reference (a `url` from Step 1, or a link the developer pasted), read its markdown:
94+
95+
1. Normalize the reference to its `.md` URL:
96+
- A site-relative path from Step 1 (e.g. `/apis/events-api/using-socket-mode`): prepend `https://docs.slack.dev`.
97+
- A full URL the developer pasted: drop any `#anchor` first.
98+
- Append `.md`, e.g. `…/using-socket-mode` → `https://docs.slack.dev/apis/events-api/using-socket-mode.md`.
99+
The server lowercases `.md` requests, so casing does not matter: `chat.postMessage.md` and `chat.postmessage.md` both resolve.
100+
2. **WebFetch** it. The page opens with `Source: <original-url>`; the rest is the page body in markdown.
101+
3. Answer the developer from the fetched content, and cite the `Source` URL.
102+
103+
If a page is long and the developer asked something narrow, fetch it and quote only the relevant section rather than dumping the whole page.
104+
105+
---
106+
107+
## Step 3: Tool and SDK Documentation
108+
109+
Implementation details differ significantly between the official tools, so **establish which one the developer is using first**, then scope your reading to that tool's doc subtree. Each lives under `https://docs.slack.dev/tools/<name>` and its pages are fetchable as `.md` like any other (e.g. `https://docs.slack.dev/tools/bolt-js/concepts.md`). If the developer has not said, ask before assuming.
110+
111+
| Tool | Docs path | Search `category` | Use when the developer… |
112+
|------|-----------|-------------------|--------------------------|
113+
| **Slack CLI** | `/tools/slack-cli` | `slack_cli` | scaffolds, runs, or manages an app from the terminal; mentions `slack` commands or app manifests |
114+
| **Bolt for JavaScript** | `/tools/bolt-js` | `javascript` | builds an app in Node/TypeScript with the Bolt framework |
115+
| **Bolt for Python** | `/tools/bolt-python` | `python` | builds an app in Python with the Bolt framework |
116+
| **Bolt for Java** | `/tools/java-slack-sdk` | `java` | builds an app in Java with Bolt (Bolt for Java lives in the Java SDK docs) |
117+
| **Node Slack SDK** | `/tools/node-slack-sdk` | `javascript` | wants lower-level Node clients (`@slack/web-api`, `@slack/socket-mode`) without the full Bolt framework |
118+
| **Python Slack SDK** | `/tools/python-slack-sdk` | `python` | wants the lower-level Python client without Bolt |
119+
| **Java Slack SDK** | `/tools/java-slack-sdk` | `java` | wants Java clients, or is using Bolt for Java |
120+
| **Slack Send GitHub Action** | `/tools/slack-github-action` | `slack_github_action` | sends data to Slack from a GitHub Actions workflow |
121+
122+
Once you know the tool, narrow discovery with the matching `category` from Step 1, e.g. `…?query=middleware&category=javascript`. Note the languages group: both Bolt for JavaScript and the Node Slack SDK fall under `javascript` (likewise `python` and `java` each cover their Bolt framework plus lower-level SDK), so the category scopes to the language family, not a single subtree.
123+
124+
**Bolt** is the framework built upon the matching language **SDK**. When unsure which subtree a topic lives in, fall back to the search API (Step 1) as it indexes all of these.

‎tests/config.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ def get_gemini_api_key_pool(environ: Mapping[str, str]) -> list[str]:
2121
PLUGIN_NAME = json.loads(PLUGIN_MANIFEST.read_text())["name"]
2222

2323
# Skill inventory (single source of truth)
24-
EXPECTED_SKILLS = ("create-slack-app", "block-kit", "slack-api", "slack-cli")
24+
EXPECTED_SKILLS = ("create-slack-app", "block-kit", "slack-api", "slack-cli", "slack-docs")
2525

2626
# Gemini judge model. Any env var whose name starts with GEMINI_API_KEY contributes a
2727
# key to the pool; blank values are skipped so an empty GEMINI_API_KEY= doesn't sneak in.

‎tests/eval/test_tool_selection.py‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,8 +70,23 @@ class ToolChoice(BaseModel):
7070
"accepted_tools": ["slack_search_channels"],
7171
},
7272
{
73-
"id": "skill-slack-cli-socket-mode",
73+
"id": "skill-slack-docs-socket-mode",
7474
"prompt": "Search the Slack developer documentation for how to use socket mode",
75+
"accepted_tools": ["slack-docs"],
76+
},
77+
{
78+
"id": "skill-slack-docs-events-api-concept",
79+
"prompt": "How does the Events API delivery model work?",
80+
"accepted_tools": ["slack-docs"],
81+
},
82+
{
83+
"id": "skill-slack-docs-guide-url",
84+
"prompt": "Summarize this docs page for me: https://docs.slack.dev/apis/events-api/using-socket-mode",
85+
"accepted_tools": ["slack-docs"],
86+
},
87+
{
88+
"id": "skill-slack-cli-run-local",
89+
"prompt": "Run my Slack app locally for development from the terminal",
7590
"accepted_tools": ["slack-cli"],
7691
},
7792
{

0 commit comments

Comments
 (0)