diff --git a/.changeset/clarify-docs-search-boundary.md b/.changeset/clarify-docs-search-boundary.md new file mode 100644 index 0000000..8b14cbd --- /dev/null +++ b/.changeset/clarify-docs-search-boundary.md @@ -0,0 +1,5 @@ +--- +"slack": minor +--- + +Route general Slack documentation questions to the slack-docs skill. The slack-cli skill's docs search is now scoped to terminal-based lookups via `slack docs search`, removing the overlap with slack-docs. diff --git a/skills/slack-cli/SKILL.md b/skills/slack-cli/SKILL.md index e7e7825..334552d 100644 --- a/skills/slack-cli/SKILL.md +++ b/skills/slack-cli/SKILL.md @@ -1,6 +1,6 @@ --- name: slack-cli -description: Use the Slack CLI to create, run, and manage Slack apps from the terminal. Use whenever the developer wants to log in, add a team, switch workspaces, or authenticate with Slack; whenever Slack CLI commands are needed (local development with `slack run`, managing app lifecycle, the manifest); or when searching the Slack developer documentation for any topic (socket mode, the Events API, OAuth, manifests, Bolt). +description: Use the Slack CLI to create, run, and manage Slack apps from the terminal. Use whenever the developer wants to log in, add a team, switch workspaces, or authenticate with Slack; whenever Slack CLI commands are needed (local development with `slack run`, managing app lifecycle, the manifest). --- # Slack CLI @@ -90,12 +90,11 @@ Search Slack's developer documentation directly from the terminal. SLACK_CMD docs search "" --output=text --limit=5 ``` -Use `--output=text` for concise terminal-readable results. Use this: +Use `--output=text` for concise terminal-readable results. Use this when you are already running CLI commands and want to: -- Before implementing a Slack feature you have not used before -- When the developer asks "how does X work in Slack?" -- To verify API behavior or required scopes -- To look up Block Kit elements, event types, or method parameters +- Check a Slack feature before implementing it +- Verify API behavior or required scopes +- Look up Block Kit elements, event types, or method parameters --- diff --git a/skills/slack-docs/SKILL.md b/skills/slack-docs/SKILL.md new file mode 100644 index 0000000..5e6baf3 --- /dev/null +++ b/skills/slack-docs/SKILL.md @@ -0,0 +1,124 @@ +--- +name: slack-docs +description: "Search and read the official Slack platform documentation at docs.slack.dev. Use this skill to answer conceptual or how-to questions about Slack features. You can also use it to look up, fetch, or summarize specific guide pages from provided docs.slack.dev links." +argument-hint: "[topic or docs.slack.dev URL]" +--- + +# Slack Platform Documentation + +Help the developer **find** the right page on the official Slack documentation site (`https://docs.slack.dev`) and **read** it as clean markdown, so answers come from the live docs rather than memory. The site exposes three machine-readable surfaces an agent can use directly, with no authentication and no Slack workspace: + +- A **search API method**: `GET https://docs.slack.dev/api/v1/search?query=&category=` returns ranked page hits as JSON. Scope every search with a `category` (guides, reference, a specific SDK, etc.); an uncategorized search skews heavily toward SDK pages. +- **Per-page markdown**: every page is available at its URL **+ `.md`** (e.g. `/quickstart.md`). +- **Index files**: `/llms.txt` (a curated overview) and `/llms-sitemap.md` (a list of every markdown page). + +If `$0` is provided, it is either a `docs.slack.dev` URL (jump to the **Fast Path**) or a topic to search (start at **Step 1**). For a broad "how do I build X on Slack?" question rather than one specific page, read `https://docs.slack.dev/llms.txt` first; it is a curated, LLM-oriented overview of the platform and the recommended build path. + +> **Critical rules:** +> +> - The docs are the **source of truth**. Do not answer a factual question about the Slack platform from memory; discover the page, fetch it, then answer from what it says. +> - Prefer the **`.md` version** of any page over the HTML version. It is cleaner for reading and quoting. +> - Every fetched markdown page begins with a `Source: ` line. Keep that URL so you can cite the page back to the developer. + +> **DO NOT rules:** +> +> - DO NOT invent documentation URLs. Get them from the search API, the sitemap, or a link the developer gave you then verify by fetching. +> - DO NOT paraphrase a page you have not actually fetched. If a fetch fails, say so rather than filling the gap from memory. +> - DO NOT assume every URL has a `.md` version. If a fetch returns an error, fall back to the search API or the sitemap (Step 1) rather than guessing another URL. + +--- + +## Fast Path (the developer already has a URL) + +If the developer pasted a `https://docs.slack.dev/...` link, skip discovery and go straight to **Step 2** to fetch and read it. + +--- + +## Step 1: Discover the Page (search) + +Use the docs **search API** to find candidate pages. WebFetch this URL, with the query URL-encoded and a `category` to scope the results: + +```text +https://docs.slack.dev/api/v1/search?query=&category=&limit=5 +``` + +**Always start with a `category`.** An uncategorized search is dominated by SDK reference pages (a query like `socket mode` or `oauth` can return a top 10 that is entirely Bolt/Node pages), which buries the conceptual and reference content most questions are actually about. Pick the starting category from the developer's intent: + +- **`guides`** — the default for "how do I…", "what is…", and conceptual platform questions (Events API, OAuth, Socket Mode, manifests, modals, App Home). Start here when in doubt. +- **`reference`** — for a specific method, event, scope, object, or Block Kit element, especially an exact name like `chat.postMessage`. +- **A tool/SDK category** (`python`, `javascript`, `java`, `slack_cli`, `slack_github_action`, `deno_slack_sdk`) — only once you know the developer's tool. See **Step 3**. + +So `socket mode` as a concept → `https://docs.slack.dev/api/v1/search?query=socket%20mode&category=guides&limit=5`. + +The response is JSON: + +```json +{ + "total_results": 12, + "results": [ + { "url": "/apis/events-api/using-socket-mode", "title": "Using Socket Mode" } + ], + "limit": 5 +} +``` + +Scan `results` for the best `title`/`url` match, then read it in **Step 2**. A query is **required**; calling the endpoint with no `query` returns a `400` with an `error` field. + +Full set of categories: + +| Category | Scopes results to | +|----------|-------------------| +| `guides` | Conceptual and how-to guides | +| `reference` | API reference: methods, events, scopes, objects, Block Kit | +| `changelog` | Changelog and release notes | +| `python` | Python tools (Bolt for Python, Python Slack SDK) | +| `javascript` | JavaScript tools (Bolt for JS, Node Slack SDK) | +| `java` | Java tools (Bolt for Java, Java Slack SDK) | +| `slack_cli` | Slack CLI docs | +| `slack_github_action` | Slack Send GitHub Action docs | +| `deno_slack_sdk` | Deno Slack SDK docs | + +If a categorized search returns no good hit, widen it: try the other likely category (`guides` ⇄ `reference`), then drop `category` entirely as a last resort. An unrecognized value returns a `400` with an `error` field listing the valid categories, so re-run with one of those or with no category. + +**Fallbacks** when search does not surface a good hit, or returns a `500`/temporary error (the endpoint is rate-limited and cached ~5 minutes): + +- WebFetch `https://docs.slack.dev/llms-sitemap.md`, a flat list of every documentation page's `.md` URL, and scan it for the relevant path. +- For API reference lookups, the enriched index pages are often faster: `https://docs.slack.dev/reference/methods.md`, `.../events.md`, `.../scopes.md`, `.../objects.md`, and `.../block-kit.md` each list every item with a one-line description and a `.md` link. +- If the developer has the Slack CLI, `slack docs search ""` does the same discovery from the terminal (see the `slack:slack-cli` skill). + +--- + +## Step 2: Read the Page (fetch markdown) + +Given a page reference (a `url` from Step 1, or a link the developer pasted), read its markdown: + +1. Normalize the reference to its `.md` URL: + - A site-relative path from Step 1 (e.g. `/apis/events-api/using-socket-mode`): prepend `https://docs.slack.dev`. + - A full URL the developer pasted: drop any `#anchor` first. + - Append `.md`, e.g. `…/using-socket-mode` → `https://docs.slack.dev/apis/events-api/using-socket-mode.md`. + The server lowercases `.md` requests, so casing does not matter: `chat.postMessage.md` and `chat.postmessage.md` both resolve. +2. **WebFetch** it. The page opens with `Source: `; the rest is the page body in markdown. +3. Answer the developer from the fetched content, and cite the `Source` URL. + +If a page is long and the developer asked something narrow, fetch it and quote only the relevant section rather than dumping the whole page. + +--- + +## Step 3: Tool and SDK Documentation + +Implementation details differ significantly between the official tools, so **establish which one the developer is using first**, then scope your reading to that tool's doc subtree. Each lives under `https://docs.slack.dev/tools/` and its pages are fetchable as `.md` like any other (e.g. `https://docs.slack.dev/tools/bolt-js/concepts.md`). If the developer has not said, ask before assuming. + +| Tool | Docs path | Search `category` | Use when the developer… | +|------|-----------|-------------------|--------------------------| +| **Slack CLI** | `/tools/slack-cli` | `slack_cli` | scaffolds, runs, or manages an app from the terminal; mentions `slack` commands or app manifests | +| **Bolt for JavaScript** | `/tools/bolt-js` | `javascript` | builds an app in Node/TypeScript with the Bolt framework | +| **Bolt for Python** | `/tools/bolt-python` | `python` | builds an app in Python with the Bolt framework | +| **Bolt for Java** | `/tools/java-slack-sdk` | `java` | builds an app in Java with Bolt (Bolt for Java lives in the Java SDK docs) | +| **Node Slack SDK** | `/tools/node-slack-sdk` | `javascript` | wants lower-level Node clients (`@slack/web-api`, `@slack/socket-mode`) without the full Bolt framework | +| **Python Slack SDK** | `/tools/python-slack-sdk` | `python` | wants the lower-level Python client without Bolt | +| **Java Slack SDK** | `/tools/java-slack-sdk` | `java` | wants Java clients, or is using Bolt for Java | +| **Slack Send GitHub Action** | `/tools/slack-github-action` | `slack_github_action` | sends data to Slack from a GitHub Actions workflow | + +Once you know the tool, narrow discovery with the matching `category` from Step 1, e.g. `…?query=middleware&category=javascript`. Note the languages group: both Bolt for JavaScript and the Node Slack SDK fall under `javascript` (likewise `python` and `java` each cover their Bolt framework plus lower-level SDK), so the category scopes to the language family, not a single subtree. + +**Bolt** is the framework built upon the matching language **SDK**. When unsure which subtree a topic lives in, fall back to the search API (Step 1) as it indexes all of these. diff --git a/tests/config.py b/tests/config.py index f7b5ba2..1ab5020 100644 --- a/tests/config.py +++ b/tests/config.py @@ -21,7 +21,7 @@ def get_gemini_api_key_pool(environ: Mapping[str, str]) -> list[str]: PLUGIN_NAME = json.loads(PLUGIN_MANIFEST.read_text())["name"] # Skill inventory (single source of truth) -EXPECTED_SKILLS = ("create-slack-app", "block-kit", "slack-api", "slack-cli") +EXPECTED_SKILLS = ("create-slack-app", "block-kit", "slack-api", "slack-cli", "slack-docs") # Gemini judge model. Any env var whose name starts with GEMINI_API_KEY contributes a # key to the pool; blank values are skipped so an empty GEMINI_API_KEY= doesn't sneak in. diff --git a/tests/eval/test_tool_selection.py b/tests/eval/test_tool_selection.py index 90123f4..f6573f7 100644 --- a/tests/eval/test_tool_selection.py +++ b/tests/eval/test_tool_selection.py @@ -70,8 +70,23 @@ class ToolChoice(BaseModel): "accepted_tools": ["slack_search_channels"], }, { - "id": "skill-slack-cli-socket-mode", + "id": "skill-slack-docs-socket-mode", "prompt": "Search the Slack developer documentation for how to use socket mode", + "accepted_tools": ["slack-docs"], + }, + { + "id": "skill-slack-docs-events-api-concept", + "prompt": "How does the Events API delivery model work?", + "accepted_tools": ["slack-docs"], + }, + { + "id": "skill-slack-docs-guide-url", + "prompt": "Summarize this docs page for me: https://docs.slack.dev/apis/events-api/using-socket-mode", + "accepted_tools": ["slack-docs"], + }, + { + "id": "skill-slack-cli-run-local", + "prompt": "Run my Slack app locally for development from the terminal", "accepted_tools": ["slack-cli"], }, {