diff --git a/skills/databricks-apps/references/appkit/files.md b/skills/databricks-apps/references/appkit/files.md index 108d9c0..27ca210 100644 --- a/skills/databricks-apps/references/appkit/files.md +++ b/skills/databricks-apps/references/appkit/files.md @@ -1,17 +1,17 @@ # Files: Unity Catalog Volume Operations -**For full Files plugin API (routes, types, config options)**: run `npx @databricks/appkit docs` → Files plugin. +**For full Files plugin API (routes, types, config options)**: run `npx @databricks/appkit docs ./docs/plugins/files.md`. Use the `files()` plugin when your app needs to **browse, upload, download, or manage files** in Databricks Unity Catalog Volumes. For analytics dashboards reading from a SQL warehouse, use `config/queries/` instead. For persistent CRUD storage, use Lakebase. ## When to Use Files vs Other Patterns -| Pattern | Use Case | Data Source | -| --- | --- | --- | -| Analytics | Read-only dashboards, charts, KPIs | Databricks SQL Warehouse | -| Lakebase | CRUD operations, persistent state, forms | PostgreSQL (Lakebase) | -| Files | File uploads, downloads, browsing, previews | Unity Catalog Volumes | -| Files + Analytics | Upload CSVs then query warehouse tables | Volumes + SQL Warehouse | +| Pattern | Use Case | Data Source | +| ----------------- | ------------------------------------------- | ------------------------ | +| Analytics | Read-only dashboards, charts, KPIs | Databricks SQL Warehouse | +| Lakebase | CRUD operations, persistent state, forms | PostgreSQL (Lakebase) | +| Files | File uploads, downloads, browsing, previews | Unity Catalog Volumes | +| Files + Analytics | Upload CSVs then query warehouse tables | Volumes + SQL Warehouse | ## Scaffolding @@ -43,33 +43,82 @@ The env var suffix (after `DATABRICKS_VOLUME_`) becomes the volume key, lowercas import { createApp, files, server } from "@databricks/appkit"; await createApp({ - plugins: [ - server(), - files(), - ], + plugins: [server(), files()], }); ``` +### Configuration Overrides + +Only add plugin config when you need to override defaults from the discovered `DATABRICKS_VOLUME_*` env vars: + +```typescript +files({ + maxUploadSize: 5_000_000_000, // plugin-level default + volumes: { + uploads: { + maxUploadSize: 100_000_000, + policy: files.policy.allowAll(), // required for writes + }, + user_data: { + auth: "on-behalf-of-user", // HTTP routes run SDK calls as end user + }, + }, +}); +``` + +Auto-discovered volumes merge with explicit config, so `volumes: {}` is only needed for overrides. Check the AppKit docs for the current `IFilesConfig` / `VolumeConfig` shape. + +## Permission Model + +Three layers gate file access: + +1. **Unity Catalog grants** — service-principal volumes need the app SP to hold `WRITE_VOLUME`; OBO volumes need each end user to hold it. +2. **Execution identity** — HTTP routes use the volume's `auth` mode. Programmatic user-driven handlers must call `.asUser(req)` to run SDK calls as the request user. +3. **File policies** — app-level allow/deny functions evaluated before every operation. + +For SP volumes, removing a user's UC grant has no effect on HTTP access because the SDK call uses the SP. Use policies for per-user restrictions. For OBO volumes, UC grants gate the end user and policies stack on top. + +## Access Policies + +Volumes without an explicit `policy` default to `files.policy.publicRead()` (reads allowed, writes denied) and log a startup warning. Set an explicit policy on every volume that accepts uploads, directory creation, or deletes. + +```typescript +import { files } from "@databricks/appkit"; + +files({ + volumes: { + public_data: { policy: files.policy.publicRead() }, + uploads: { policy: files.policy.allowAll() }, + archive: { policy: files.policy.denyAll() }, + }, +}); +``` + +Use custom policies when access depends on the requesting user or action. For exact built-ins, combinators, `FileAction`, `FileResource`, `FilePolicyUser`, and `PolicyDeniedError` behavior, check `npx @databricks/appkit docs ./docs/plugins/files.md`. + ## Server-Side API (Programmatic) -Access volumes through the `files()` callable, which returns a `VolumeHandle`: +Access volumes through the `files()` callable, which returns a `VolumeHandle`. Direct programmatic calls do not have request headers available, so they normally run as the service principal. Use `.asUser(req)` in user-driven route handlers when the SDK call must run as the request user. ```typescript -// ✅ CORRECT — OBO access (recommended) -const entries = await appkit.files("uploads").asUser(req).list(); -const content = await appkit.files("exports").asUser(req).read("report.csv"); - -// ❌ WRONG — omitting .asUser(req) -const entries = await appkit.files("uploads").list(); -// In dev: silently falls back to service principal credentials, bypassing user-level UC permissions -// In production: throws an error +// User-driven handler: SDK call runs as user; policy sees user.id from req. +await appkit.files("uploads").asUser(req).list(); + +// Background or trusted server code: runs as SP. +await appkit.files("uploads").list(); ``` -**ALWAYS use `.asUser(req)`** — without it, dev mode silently uses the app's service principal (masking permission issues that will crash in production). +**Use `.asUser(req)` in user-driven route handlers** when you want UC grants enforced against the actual user. In production, `asUser(req)` throws `AuthenticationError.missingToken` if the forwarded user or access-token header is missing; in dev (`NODE_ENV === "development"`) it logs a warning and falls back to SP. Policy denial throws `PolicyDeniedError`. + +For method signatures, path rules, cache behavior, and retry/timeout defaults, check `npx @databricks/appkit docs ./docs/plugins/files.md`. + +## HTTP Routes + +Mounted at `/api/files/*`. Routes use the volume's `auth` mode and run the policy before the operation. Use the AppKit docs for the exact route list, request bodies, response types, and `/raw` content-security behavior. ## Frontend Components -Import file browser components from `@databricks/appkit-ui/react`. Full component props: `npx @databricks/appkit docs "FileBreadcrumb"`. +Import file browser components from `@databricks/appkit-ui/react`. Full component props: `npx @databricks/appkit docs ./docs/api/appkit-ui/files/DirectoryList.md` and the related component pages. ### File Browser Example @@ -184,8 +233,8 @@ export function FilesPage() { ```typescript const handleUpload = async (file: File) => { const uploadPath = currentPath ? `${currentPath}/${file.name}` : file.name; - const response = await fetch(apiUrl('upload', { path: uploadPath }), { - method: 'POST', + const response = await fetch(apiUrl("upload", { path: uploadPath }), { + method: "POST", body: file, }); if (!response.ok) { @@ -203,7 +252,7 @@ const handleUpload = async (file: File) => { const handleDelete = async (filePath: string) => { const response = await fetch( `/api/files/${volumeKey}?path=${encodeURIComponent(filePath)}`, - { method: 'DELETE' }, + { method: "DELETE" }, ); if (!response.ok) { const data = await response.json().catch(() => ({})); @@ -217,23 +266,28 @@ const handleDelete = async (filePath: string) => { ```typescript const handleCreateDirectory = async (name: string) => { const dirPath = currentPath ? `${currentPath}/${name}` : name; - const response = await fetch(apiUrl('mkdir'), { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, + const response = await fetch(apiUrl("mkdir"), { + method: "POST", + headers: { "Content-Type": "application/json" }, body: JSON.stringify({ path: dirPath }), }); if (!response.ok) { const data = await response.json().catch(() => ({})); - throw new Error(data.error ?? `Create directory failed (${response.status})`); + throw new Error( + data.error ?? `Create directory failed (${response.status})`, + ); } }; ``` ## Resource Requirements -Each volume key requires a resource with `WRITE_VOLUME` permission. Declare in `databricks.yml`: +The plugin auto-generates volume resource requirements from `DATABRICKS_VOLUME_*` env vars. Setting them in `app.yaml` is usually all you need. + +Declare the volume explicitly in `databricks.yml` only when you need to pin it as a managed resource, then wire the env var via `valueFrom` in `app.yaml`: ```yaml +# databricks.yml resources: apps: my_app: @@ -246,11 +300,10 @@ resources: permission: WRITE_VOLUME ``` -> **Note:** The scaffolded HTTP routes (`/api/files/...`) execute as the service principal and do not require `user_api_scopes`. The scope is needed when using the programmatic `appkit.files("key").asUser(req)` API for per-user Volume access. - -Wire the env var in `app.yaml`: +> **Note:** `user_api_scopes` is required for OBO volumes (`auth: "on-behalf-of-user"`) and for any `appkit.files("key").asUser(req)` programmatic call. Pure SP volumes accessed only via HTTP routes don't need it. The plugin docs have the latest resource-requirement behavior. ```yaml +# app.yaml env: - name: DATABRICKS_VOLUME_UPLOADS valueFrom: uploads-volume @@ -258,11 +311,13 @@ env: ## Troubleshooting -| Error | Cause | Solution | -| --- | --- | --- | -| `Unknown volume key "X"` | Volume env var not set or misspelled | Check `DATABRICKS_VOLUME_X` is set in `app.yaml` or `.env` | -| 413 on upload | File exceeds `maxUploadSize` | Increase `maxUploadSize` in plugin config or per-volume config | -| `read()` rejects large file | File > 10 MB default limit | Use `download()` for large files or pass `{ maxSize: }` | -| Blocked content type on `/raw` | Dangerous MIME type (html, js, svg) | Use `/download` instead — these types are forced to attachment | -| Service principal access blocked | Called volume method without `.asUser(req)` | Always use `appkit.files("key").asUser(req).method()` | -| `path traversal` error | Path contains `../` | Use relative paths from volume root or absolute `/Volumes/...` paths | +| Error | Cause | Solution | +| ------------------------------------------ | ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | +| `Unknown volume "X"` | Volume env var not set or misspelled | Check `DATABRICKS_VOLUME_X` is set in `app.yaml` or `.env` | +| 413 on upload | File exceeds `maxUploadSize` | Increase `maxUploadSize` in plugin config or per-volume config | +| `read()` rejects large file | File > 10 MB default limit | Use `download()` for large files or pass `{ maxSize: }` | +| Blocked content type on `/raw` | Dangerous MIME type (html, js, svg) | Use `/download` instead — these types are forced to attachment | +| 403 on HTTP route | Volume's policy denied the action for the requesting user | Inspect `policy` config; user id comes from the `x-forwarded-user` header | +| Writes return 403 unexpectedly | Volume has no `policy` configured → defaults to `publicRead()` which denies writes | Set explicit `policy: files.policy.allowAll()` (or stricter) on volumes that accept writes | +| `PolicyDeniedError` from programmatic call | Volume's policy denied the action — SP identity used if `asUser(req)` was omitted | Call `.asUser(req)` for user-driven calls; gate trusted SP code with `policy.allowAll()` | +| Invalid path error | Path contains `../`, null bytes, or exceeds 4096 chars | Use relative paths from the volume root, or absolute `/Volumes/...` paths |