|
| 1 | +# AI CSV/TSV Insights Sidebar |
| 2 | + |
| 3 | +Adds an **Insights** sidebar panel and an **Insights** context-menu action for |
| 4 | +CSV and TSV files. On demand it downloads the file via WebDAV, parses it |
| 5 | +client-side (headers plus up to the first 200 rows), and sends a compact |
| 6 | +structured preview to an admin-configured, OpenAI-compatible LLM endpoint. The |
| 7 | +panel renders detected column types, value ranges for numeric columns, and |
| 8 | +2–3 natural-language observations about the data. |
| 9 | + |
| 10 | +Files larger than 5 MB are rejected before fetching to avoid buffering large |
| 11 | +files in memory. The first analysis per session shows a consent disclosure |
| 12 | +informing the user that file contents will be sent to the configured AI service. |
| 13 | + |
| 14 | +No LLM provider is bundled and no API keys are embedded in the browser — |
| 15 | +the endpoint is fully operator-controlled (BYO-LLM). |
| 16 | + |
| 17 | +Requests flow **browser → `ai-llm-proxy` sidecar → LLM**. The sidecar |
| 18 | +lives in `packages/ai-llm-proxy`, validates the user's oCIS bearer token |
| 19 | +against the oCIS OIDC userinfo endpoint, and forwards the request to the |
| 20 | +configured LLM with the LLM API key injected server-side. The API key |
| 21 | +never reaches the browser. |
| 22 | + |
| 23 | +## Supported File Types |
| 24 | + |
| 25 | +CSV (`.csv`), TSV (`.tsv`) |
| 26 | + |
| 27 | +Files are fetched via WebDAV and parsed client-side with a lightweight |
| 28 | +RFC-4180 state machine. The parser handles quoted fields (including embedded |
| 29 | +newlines and commas), Windows line endings, and single-column files. At most |
| 30 | +30 columns and 5 sample values per column are sent to the LLM, capping prompt |
| 31 | +size regardless of file width. |
| 32 | + |
| 33 | +## Extension Points |
| 34 | + |
| 35 | +| ID | Type | |
| 36 | +|----|------| |
| 37 | +| `global.files.sidebar` | `sidebarPanel` — "Insights" tab, visible for single supported files | |
| 38 | +| `global.files.context-actions` | `action` — "Insights" entry that opens the Insights sidebar tab | |
| 39 | + |
| 40 | +Both extension points are hidden entirely when no LLM endpoint is configured, |
| 41 | +so users on unconfigured deployments never see empty or broken UI. |
| 42 | + |
| 43 | +## Configuration |
| 44 | + |
| 45 | +### Web App Config |
| 46 | + |
| 47 | +Admins set the proxy endpoint and model in the oCIS Web app config: |
| 48 | + |
| 49 | +```yaml |
| 50 | +ai-data-insights-sidebar: |
| 51 | + config: |
| 52 | + llm: |
| 53 | + endpoint: "https://your-ocis.example.com/ai-llm-proxy/v1" |
| 54 | + model: "llama3.1:70b" |
| 55 | +``` |
| 56 | +
|
| 57 | +The panel reads `applicationConfig.llm` at startup. If `endpoint` or `model` |
| 58 | +is absent both the sidebar panel and the context-menu action are hidden. |
| 59 | + |
| 60 | +### `ai-llm-proxy` Sidecar |
| 61 | + |
| 62 | +The sidecar is configured entirely via environment variables — the LLM API key |
| 63 | +stays server-side and never reaches the browser: |
| 64 | + |
| 65 | +| Variable | Required | Description | |
| 66 | +|----------|----------|-------------| |
| 67 | +| `OCIS_URL` | yes | oCIS base URL, used to discover the OIDC userinfo endpoint | |
| 68 | +| `LLM_ENDPOINT` | yes | LLM base URL (OpenAI-compatible, e.g. `http://localhost:11434/v1`) | |
| 69 | +| `LLM_API_KEY` | no | API key forwarded to the LLM in `Authorization: Bearer` | |
| 70 | +| `PORT` | no | Listening port, default `3030` | |
| 71 | +| `NODE_TLS_REJECT_UNAUTHORIZED` | no | Set to `0` for dev stacks with self-signed certs | |
| 72 | + |
| 73 | +In the dev docker-compose stack the sidecar is exposed at |
| 74 | +`https://host.docker.internal:9200/ai-llm-proxy/v1` via Traefik. |
| 75 | +Set `AI_LLM_ENDPOINT` and `AI_LLM_API_KEY` in a `.env` file or as shell |
| 76 | +variables before running `docker-compose up`. |
| 77 | + |
| 78 | +## Insights Output |
| 79 | + |
| 80 | +The LLM is prompted to return a JSON object with three fields: |
| 81 | + |
| 82 | +- **columnTypes** — an array of `{ column, type }` objects confirming the |
| 83 | + detected type (`number`, `date`, `boolean`, or `string`) for each column |
| 84 | +- **ranges** — an array of `{ column, min, max }` objects for numeric and date |
| 85 | + columns |
| 86 | +- **observations** — an array of 2–3 plain natural-language observations about |
| 87 | + the data |
| 88 | + |
| 89 | +The panel renders this as a column-type table with range annotations and a |
| 90 | +bullet list of observations. Observations are returned in the user's preferred |
| 91 | +oCIS language (via the `preferredLanguage` profile setting). |
| 92 | + |
| 93 | +## Panel States |
| 94 | + |
| 95 | +| State | Shown when | |
| 96 | +|-------|-----------| |
| 97 | +| Idle | Panel mounted, no analysis started yet — shows "Analyze" button | |
| 98 | +| Analyzing | LLM request in flight — shows "Analyzing…" placeholder | |
| 99 | +| Result | Column-type table, ranges, and observations rendered — shows "Re-analyze" button | |
| 100 | +| Error | Endpoint unreachable, auth failure, rate-limit, cross-origin block, or malformed response | |
| 101 | + |
| 102 | +Errors are shown only inside the panel and include admin-actionable guidance. |
| 103 | + |
| 104 | +## Security |
| 105 | + |
| 106 | +- The extension enforces a same-origin check on the configured LLM endpoint |
| 107 | + before attaching the user's oCIS bearer token. Cross-origin endpoints are |
| 108 | + rejected with an in-panel error message. |
| 109 | +- No API key is ever stored in or passed through extension config — the key |
| 110 | + lives exclusively in the `ai-llm-proxy` server environment. |
0 commit comments