diff --git a/docs/mcp_server/index.md b/docs/mcp_server/index.md new file mode 100644 index 0000000000..6b7d9ae7e7 --- /dev/null +++ b/docs/mcp_server/index.md @@ -0,0 +1,82 @@ +# SC4S MCP Server + +The **SC4S MCP Server** is a [Model Context Protocol](https://modelcontextprotocol.io) server +that exposes Splunk Connect for Syslog (SC4S) knowledge and a safe management +API to any MCP-compatible AI assistant or agent (for example, Cursor, +Claude Desktop, or Visual Studio Code with an MCP extension). It lets the +assistant help you discover supported vendors, author and validate +syslog-ng parsers, inspect a running SC4S instance, and apply +configuration changes through well-defined MCP tools, resources, and +prompts. + +## What it is + +The server is shipped as a small containerized Python application based on +[FastMCP](https://github.com/jlowin/fastmcp). It provides three categories of +capabilities: + +| Category | Purpose | +|---|----------------------------------------------------------------------------------------------------------------------------------| +| **Tools** | Callable functions the assistant can invoke (for example: upload a new parser, modify environment variables, check SC4S health). | +| **Resources** | Read-only documents the assistant can load on demand (parser creation guide, troubleshooting guide, vendor docs). | +| **Prompts** | Guided workflows (for example: `create_parser`, `troubleshoot_sc4s`) that orient the assistant to a specific task. | + +The full list of tools, resources, and prompts is documented in +[Tools](tools.md) and [Resources and prompts](resources_and_prompts.md). + +## Architecture + +The MCP server runs in its own OCI container (Docker or Podman) and +communicates with: + +1. Your **MCP client** (the AI assistant or agent) over `stdio` on the + local machine, or over `SSE` (HTTP) when the client is remote. +2. The **SC4S container's management API**, an HTTP REST API exposed by + the SC4S container (default port `8080`) that handles configuration + reads and writes for `env_file`, custom parsers, and Splunk metadata. + +![MCP-server-diagram.svg](../resources/images/MCP-server-diagram.svg) + +## Security model + +!!! important "The MCP server never runs commands outside its container" + The SC4S MCP server does **not** execute shell commands, scripts, or + binaries on your host. It does **not** invoke `docker`, `podman`, + `systemctl`, `syslog-ng`, `bash`, or any other process outside the + container in which it runs. + + All management actions are performed by calling a well-defined HTTP + REST API exposed by the SC4S container itself. That API, and only + that API, decides what configuration is accepted, validates syntax, + and restarts `syslog-ng` inside the SC4S container when needed. + +Concretely, the MCP server only does two kinds of I/O: + +* **Reads** a small, read-only set of files that are baked into its own + container image at build time: the documentation under `docs/`, the + parser library under `package/lite/etc/addons/`, and the parser-creator + knowledge base. These are static; the MCP server does not reach into + your host filesystem. +* **Makes HTTP(S) requests** to `SC4S_API_URL`, the REST API running + inside the SC4S container. Every "management" tool is a thin wrapper + over a single HTTP call. There is no shell, no `exec`, and no process + spawning. + +Additional the shipped image: + +* Runs as a non-root user (`uid 10001`), as declared in the Dockerfile + (`USER mcp`). +* Does not mount or connect to the Docker or Podman socket; the MCP + server has no container-management capabilities. +* The MCP server container itself does not require any host-filesystem + bind mounts. (The `env_file` bind mount described in + [Installation](installation.md#prepare-your-sc4s-instance) is applied + to the SC4S container, not to the MCP server container.) + +## Next steps + +* **[Installation](installation.md)**, build the image, run it with + Docker or Podman, and configure your MCP client. +* **[Tools](tools.md)**, reference of every callable tool. +* **[Resources and prompts](resources_and_prompts.md)**, reference of + the available MCP resources and guided prompts. diff --git a/docs/mcp_server/installation.md b/docs/mcp_server/installation.md new file mode 100644 index 0000000000..4eaba9b8c0 --- /dev/null +++ b/docs/mcp_server/installation.md @@ -0,0 +1,186 @@ +# Installing the SC4S MCP Server + +The SC4S MCP server is distributed as a container image. It can be run on +your local workstation for use with a local AI assistant (using the +`stdio` transport), or on a shared host, such as the same machine as your +SC4S instance, where remote assistants and agents connect to it over the +`SSE` transport. + +!!! note "No host commands are executed" + Regardless of how you run the container, the MCP server itself never + runs commands outside the container. See + [Security model](index.md#security-model-no-host-command-execution). + +## Prerequisites + +SC4S MCP server is currently available only for Podman or Docker runtimes. + +* A running SC4S instance exposing the management REST API (default port + `8080`). +* Docker or Podman on the host where the MCP server will run. +* An MCP-compatible AI assistant or agent that can connect to the server + over `stdio` or `SSE` (for example, Cursor, Claude Desktop, or Visual + Studio Code with an MCP extension). + +## Configuration reference + +The MCP server is configured through environment variables. + +| Variable | Default | Description | +|---|---|---| +| `MCP_TRANSPORT` | `stdio` | Transport mode: `stdio` for local clients, `sse` for remote clients. | +| `MCP_HOST` | `0.0.0.0` | Bind address used in `sse` mode. | +| `MCP_PORT` | `8000` | TCP port used in `sse` mode. | +| `SC4S_API_URL` | `http://localhost:8080` | URL of the SC4S management REST API. The MCP server calls this URL for all management tools. | + +## Build the image + +The Dockerfile is part of the SC4S repository. Build from the repository +root so that the `docs/`, `package/lite/etc/addons/`, and +`.agents/skills/parser-creator/` directories are available to the build +context. + +Docker: + +```bash +docker build -t sc4s-mcp -f sc4s_mcp/Dockerfile . +``` + +Podman: + +```bash +podman build -t sc4s-mcp -f sc4s_mcp/Dockerfile . +``` + +## Run the container + +The examples below assume the SC4S container is reachable at +`http://:8080`. If SC4S runs on the same host, use +`--network host` and point `SC4S_API_URL` to `http://127.0.0.1:8080`. + +Docker: + +```bash +docker run -d \ + -p 8000:8000 \ + -e SC4S_API_URL=http://:8080 \ + --name sc4s-mcp \ + sc4s-mcp +``` + +Same host as SC4S (Linux): + +```bash +docker run -d --network host \ + -e SC4S_API_URL=http://127.0.0.1:8080 \ + --name sc4s-mcp \ + sc4s-mcp +``` + +Podman (locally built images are stored under the `localhost/` prefix): + +```bash +podman run -d \ + -p 8000:8000 \ + -e SC4S_API_URL=http://:8080 \ + --name sc4s-mcp \ + localhost/sc4s-mcp +``` + +Same host as SC4S (Linux): + +```bash +podman run -d --network host \ + -e SC4S_API_URL=http://127.0.0.1:8080 \ + --name sc4s-mcp \ + localhost/sc4s-mcp +``` + +The image ships a healthcheck that verifies the SSE endpoint is up. Check +the container status with: + +```bash +docker ps # or: podman ps +``` + +## Prepare your SC4S instance + +The MCP management tools (`sc4s_set_env`, `sc4s_add_parser`, and others) +need to read and write SC4S's `env_file` at runtime. The `--env-file` +Docker/Podman flag alone is **not sufficient**: it injects variables at +startup but does not make the file visible inside the container. + +Bind-mount the file in addition to passing `--env-file`: + +``` +-v /opt/sc4s/env_file:/opt/sc4s/env_file:z +``` + +### systemd unit example + +If you run SC4S via systemd, add the mount variable: + +```ini +Environment="SC4S_ENV_FILE_MOUNT=/opt/sc4s/env_file:/opt/sc4s/env_file:z" +``` + +and include `-v "$SC4S_ENV_FILE_MOUNT"` in your `ExecStart` alongside the +other `-v` flags: + +```ini +ExecStart=/usr/bin/podman run \ + -e "SC4S_CONTAINER_HOST=${SC4SHOST}" \ + -v "$SC4S_PERSIST_MOUNT" \ + -v "$SC4S_LOCAL_MOUNT" \ + -v "$SC4S_ARCHIVE_MOUNT" \ + -v "$SC4S_TLS_MOUNT" \ + -v "$SC4S_ENV_FILE_MOUNT" \ + --env-file=/opt/sc4s/env_file \ + --network host \ + --name SC4S \ + --rm $SC4S_IMAGE +``` + +Then reload and restart: + +```bash +sudo systemctl daemon-reload +sudo systemctl restart sc4s +``` + +### Generic MCP client configuration + +Most MCP clients accept one of two connection styles. Consult your +client's documentation for the exact configuration file and its location; +the shape of the configuration is typically the same across clients. + +**Local process (stdio)**: the client launches the server as a child +process and communicates via standard input/output. Provide: + +* a `command` to execute (for example `docker` or `podman`), +* an `args` array that starts the MCP server, +* optional environment variables (`SC4S_API_URL`, `MCP_TRANSPORT=stdio`). + +**Remote endpoint (SSE)**: the client connects to an HTTP URL exposing +the MCP SSE endpoint. Provide: + +* a `url` pointing at `http://:8000/sse`, +* any additional headers required by your deployment (for example, a + reverse-proxy auth header). + +## Verify the installation + +1. Confirm the container is running: `docker ps` or `podman ps`. +2. Confirm the MCP client sees the server. Most clients list available + MCP servers in a dedicated panel or on startup. +3. From the assistant, call the `sc4s_health` tool. A healthy instance + returns a status payload from the SC4S management API. An error like + `"SC4S instance unreachable at http://..."` means the MCP server + could reach out but SC4S is not answering. Check `SC4S_API_URL`, the + SC4S container status, and network connectivity. + +## Upgrading + +To upgrade the MCP server, rebuild the image from a newer revision of +the repository and restart the container. Configuration is +environment-based, so no data migration is required. \ No newline at end of file diff --git a/docs/mcp_server/resources_and_prompts.md b/docs/mcp_server/resources_and_prompts.md new file mode 100644 index 0000000000..2ef7005b88 --- /dev/null +++ b/docs/mcp_server/resources_and_prompts.md @@ -0,0 +1,95 @@ +# Resources and prompts + +In addition to [tools](tools.md), the SC4S MCP server exposes two other +MCP primitives: + +* **Resources**: read-only documents that the assistant (or the user) + can request by URI. Resources are static content baked into the MCP + container image; they do not touch SC4S or the host. +* **Prompts**: pre-built workflows that seed a conversation with all + the context needed for a specific task. Prompts do not execute code + on their own; they just structure the conversation. + +## Resources + +All resources use the `sc4s://` URI scheme. They are assembled from the +project's markdown documentation at the time they are fetched, so they +always match the docs that ship with the image you are running. + +| URI | Contents | +|---|---| +| `sc4s://docs/creating_parsers` | The full parser-creation guide: the `index.md`, `filter_message.md`, `parse_message.md`, and `unit_tests.md` chapters, concatenated. Use this to give the assistant the conventions it needs to write a correct parser. | +| `sc4s://docs/troubleshooting` | The troubleshooting guide: startup/validation checks, logging resources, PCAP replay, and Splunk health checks, concatenated. Use this when diagnosing a problem. | +| `sc4s://docs/vendor/{vendor}` | All markdown documentation for a given vendor under `docs/sources/vendor/{vendor}/`. Replace `{vendor}` with a directory name; the list is available from the `list_vendors` tool. | + +Resources are read-only by construction: they return text, nothing +else. The server cannot use them to modify state. + +## Prompts + +Prompts are ready-made workflows that include the relevant knowledge +base and a set of diagnostic steps. Invoke them from your client's +prompt picker, most mcp clients should expose such menu. + +### `create_parser` + +Guided workflow for authoring a new SC4S syslog-ng parser from sample +logs. The prompt loads the project's parser-creation knowledge base and +asks the assistant to follow those conventions exactly. + +**Parameters** + +| Name | Description | +|---|---| +| `vendor` | Vendor name (free text, for example `cisco`). | +| `product` | Product name (free text, for example `asa`). | +| `sample_logs` | One or more raw log lines to base the parser on. | + +**Typical flow after invocation** + +1. The assistant reads the knowledge base and the sample logs. +2. It calls `list_vendor_parsers` / `get_parser` to learn existing + conventions for the same vendor (if any). +3. It drafts a new `.conf` parser and the corresponding unit tests. +4. With your approval, it uploads the parser via `sc4s_add_parser`. + +### `troubleshoot_sc4s` + +Guided workflow for diagnosing a live SC4S issue. The prompt loads the +troubleshooting docs and asks the assistant to run the standard +diagnostic sequence before making any changes. + +**Parameters** + +| Name | Description | +|---|---| +| `symptom` | Free-text description of what you are seeing (for example, *"no Cisco ASA events in Splunk"*). | + +**Diagnostic steps seeded by the prompt** + +1. Call `sc4s_health` to check the instance status. +2. Call `sc4s_get_env` to review the current configuration. +3. Call `sc4s_list_custom_parsers` to list deployed custom parsers. +4. Propose specific fixes. +5. Apply configuration changes via `sc4s_set_env` only after explaining + the reasoning. + +## Combining tools, resources, and prompts + +A typical interaction uses all three primitives together. For example, +adding support for a new vendor might look like this: + +1. Invoke the `create_parser` prompt with vendor, product, and sample + logs. +2. The assistant pulls context from `sc4s://docs/creating_parsers` and + calls read-only tools (`list_vendors`, `list_vendor_parsers`, + `get_parser`) to inspect existing work. +3. After you review the draft, the assistant calls `sc4s_add_parser` + to deploy it. +4. If the SC4S management API rejects the parser, the change is rolled + back automatically and the assistant iterates. + +Throughout this flow the MCP server itself only reads files from its own +container and makes HTTP calls to `SC4S_API_URL`. See +[Security model](index.md#security-model-no-host-command-execution) for +details. diff --git a/docs/mcp_server/tools.md b/docs/mcp_server/tools.md new file mode 100644 index 0000000000..ee3ec5bca9 --- /dev/null +++ b/docs/mcp_server/tools.md @@ -0,0 +1,108 @@ +# Tools + +Tools are callable functions that the MCP client (and the AI assistant +behind it) can invoke. The SC4S MCP server groups tools into three +categories: + +* **Repository / documentation**: read-only, safe to call at any time. + These tools only read content that is baked into the MCP container + image. +* **SC4S instance management**: thin wrappers over the SC4S management + REST API. These tools can change SC4S configuration and trigger a + `syslog-ng` restart **inside the SC4S container**. +* **Splunk and compliance metadata**: specialized management tools for + `splunk_metadata.csv` and `compliance_meta_by_source` overrides. + +!!! important "How tools make changes" + Management tools never execute shell commands. They send a single + HTTP request to the SC4S management API at `SC4S_API_URL`. The API + validates the input, stages the change, and restarts `syslog-ng` + inside the SC4S container. If validation fails, the SC4S API rolls + back the change automatically. + +## Repository and documentation tools + +These tools read static content from the MCP container. They make no +outbound calls and cannot modify anything. + +| Tool | Description | +|---|---| +| `list_vendors()` | Lists all vendors supported by SC4S, based on the subdirectories of `docs/sources/vendor/`. | +| `list_all_parsers()` | Lists all `.conf` parser files from `package/lite/etc/addons/`. | +| `list_vendor_parsers(vendor)` | Lists parser files whose contents reference a vendor name (case-insensitive whole-word match). | +| `get_parser(parser_name)` | Returns the content of a parser file. Accepts either the file name (`foo.conf`) or the stem (`foo`). Returns `{ "found": bool, "path": ..., "content": ... }`. | +| `search_docs(query)` | Regex search across every markdown file under `docs/`. Returns `path:line: snippet` entries. | +| `get_parser_creation_guide()` | Returns the full parser-creation guide (`SKILL.md` + testing reference). The assistant calls this automatically when a user asks to create a parser. | + +## SC4S instance management tools + +These tools require a reachable SC4S instance at `SC4S_API_URL`. If the +instance is unreachable, you will see an error payload +`{"status": "error", "message": "SC4S instance unreachable at ..."}` +instead of a failure inside your SC4S container. + +### Health and general configuration + +| Tool | Description | +|---|---| +| `sc4s_health()` | Returns the health payload from the SC4S management API. Use this first when troubleshooting. | +| `sc4s_get_env()` | Reads the current `env_file` from the running SC4S instance. | +| `sc4s_set_env(env_file_content)` | Uploads a new `env_file`. The SC4S API backs up the previous file, applies the new one, and restarts `syslog-ng`. On validation failure the previous file is restored. | + +### Custom parsers + +| Tool | Description | +|---|---| +| `sc4s_list_custom_parsers()` | Lists all custom parsers currently deployed on the SC4S instance. | +| `sc4s_get_custom_parser(name)` | Reads the content of a deployed custom parser. | +| `sc4s_add_parser(filename, content)` | Uploads a new `.conf` parser. The `.conf` extension is added if missing. SC4S validates syntax and restarts `syslog-ng`; invalid parsers are rolled back. | +| `sc4s_delete_parser(name)` | Deletes a custom parser. SC4S re-validates the remaining configuration and restarts `syslog-ng`; if validation fails, the parser is restored. | + +### Splunk metadata (`splunk_metadata.csv`) + +These tools manage per-vendor/product overrides that SC4S sends to Splunk +(index, source, sourcetype, host, template). + +| Tool | Description | +|---|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `sc4s_get_splunk_metadata()` | Reads `splunk_metadata.csv` entries. Each entry is `{ key, metadata, value }`, where `metadata` is one of `index, source, sourcetype, host, sc4s_template` and `key` is a `vendor_product` identifier. | +| `sc4s_set_splunk_metadata(entries)` | Overwrites `splunk_metadata.csv` with the provided list. SC4S restarts after applying. Example entry: `{"key": "juniper_netscreen", "metadata": "index", "value": "ns_index"}`. | +| `sc4s_delete_splunk_metadata()` | Clears all Splunk metadata overrides. SC4S restarts after clearing. | + +### Compliance metadata (`compliance_meta_by_source`) + +These tools manage the filter definitions and CSV rows used to redirect +events to different Splunk indexes (or add indexed fields) based on +host, IP, or subnet matching. + +| Tool | Description | +|---|---| +| `sc4s_get_compliance_overrides()` | Reads both the `.conf` filter definitions (`conf_content`) and the CSV rows (`csv_content`). | +| `sc4s_set_compliance_override(conf_content, csv_content)` | Overwrites both files. `csv_content` is a list of `{filter_name, field_name, value}` dicts, where `field_name` must be `.splunk.index`, `.splunk.source`, `.splunk.sourcetype`, or `fields.`. SC4S restarts after applying. | +| `sc4s_delete_compliance_override()` | Clears both files, removing all compliance overrides. SC4S restarts after clearing. | + +Example `conf_content`: + +``` +filter f_pci_zone { host("pci-*" type(glob)) or netmask(10.1.0.0/16) }; +``` + +Example `csv_content` entry: + +```json +{ + "filter_name": "f_pci_zone", + "field_name": ".splunk.index", + "value": "pci_idx" +} +``` + +## Error handling + +All management tools return structured JSON. When something goes wrong, +you will get a payload with a `status: "error"` field, and one of: + +| Field | Meaning | +|---|---| +| `message` | Human-readable error (connection refused, timeout, transport-level failure). | +| `http_status` + additional fields | The SC4S management API returned a non-2xx HTTP response. The extra fields come from the API's JSON body (typically `detail` or `error`). | \ No newline at end of file diff --git a/docs/resources/images/MCP-server-diagram.svg b/docs/resources/images/MCP-server-diagram.svg new file mode 100644 index 0000000000..b5e1d1e124 --- /dev/null +++ b/docs/resources/images/MCP-server-diagram.svg @@ -0,0 +1,4 @@ + + + +
 stdio / HTTP (SSE) 
 stdio / HTTP (SSE) 
MCP Client
(Cursor, Claude e.g.)
MCP Client...
HTTP
HTTP
SC4S MCP server
SC4S MCP server
SC4S
(syslog-ng +
REST API)
SC4S... \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index d303d84e6b..3adaf0ef49 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -83,6 +83,11 @@ nav: - Pluggable modules: "pluggable_modules.md" - Enterprise: enterprise.md - Edge Processor: "edge_processor.md" + - MCP Server: + - Overview: "mcp_server/index.md" + - Installation: "mcp_server/installation.md" + - Tools: "mcp_server/tools.md" + - Resources and Prompts: "mcp_server/resources_and_prompts.md" - Troubleshooting: - SC4S Startup and Validation: "troubleshooting/troubleshoot_SC4S_server.md" - SC4S Logging and Troubleshooting Resources: "troubleshooting/troubleshoot_resources.md"