Skip to content

Commit 17b32bb

Browse files
authored
Merge pull request #449 from weaviate/docs/backup-export-skip-access-check
Document BACKUP_SKIP_ACCESS_CHECK and EXPORT_SKIP_ACCESS_CHECK env vars
2 parents 440461d + 54d2322 commit 17b32bb

3 files changed

Lines changed: 14 additions & 2 deletions

File tree

docs/deploy/configuration/backups.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ To use any provider:
122122
- Option 1: Set the necessary environment variables
123123
- Option 2 (Kubernetes): Configure the [Helm chart values](#kubernetes-configuration)
124124

125-
Note multiple providers can be enabled simultaneously
125+
Note multiple providers can be enabled simultaneously.
126126

127127
### S3 (AWS or S3-compatible)
128128

@@ -778,6 +778,16 @@ The backup API is built in a way that no long-running network requests are requi
778778

779779
If you would like your application to wait for the background backup process to complete, you can use the "wait for completion" feature that is present in all language clients. The clients will poll the status endpoint in the background and block until the status is either `SUCCESS` or `FAILED`. This makes it easy to write simple synchronous backup scripts, even with the async nature of the API.
780780

781+
### Skip the storage access check
782+
783+
When a cloud backup backend (`backup-s3`, `backup-gcs`, or `backup-azure`) initializes, Weaviate verifies that the configured credentials can write to and delete from the target bucket. It does this by writing a temporary `access-check` object and then removing it. This probe fails on immutable (write-once / WORM) buckets, or with least-privilege credentials that are not permitted to delete objects.
784+
785+
Set `BACKUP_SKIP_ACCESS_CHECK=true` to skip this probe. The variable applies to all cloud backup backends, defaults to `false`, and is applied at startup (a restart is required to change it).
786+
787+
| Environment variable | Required | Description |
788+
| --- | --- | --- |
789+
| `BACKUP_SKIP_ACCESS_CHECK` | no | Skip the write-and-delete access check performed when a backup backend initializes. Set to `true` for immutable (write-once / WORM) buckets or least-privilege credentials that cannot delete objects. Defaults to `false`.<br/><br/>Added in `v1.37.8`. |
790+
781791
## Other Use cases
782792

783793
### Migrating to another environment

docs/deploy/configuration/env-vars/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ import APITable from '@site/src/components/APITable';
5050
| `EXPORT_DEFAULT_PATH` | Optional base path prefix for exported files within the bucket for [collection exports](/docs/deploy/configuration/export.md). Defaults to `""` (no prefix). _Changed in `v1.37.1`: previously required to be explicitly set._<br/>Added in `v1.37` | `string` | `exports/my-cluster` |
5151
| `EXPORT_ENABLED` | Enable the [collection export](/docs/deploy/configuration/export.md) API. Default: `false`<br/>Added in `v1.37` | `boolean` | `true` |
5252
| `EXPORT_PARALLELISM` | Number of concurrent scan workers for [collection exports](/docs/deploy/configuration/export.md). `0` uses GOMAXPROCS. Default: `0`<br/>Added in `v1.37` | `string - number` | `4` |
53+
| `EXPORT_SKIP_ACCESS_CHECK` | Skip the write-and-delete access check that runs when the [collection export](/docs/deploy/configuration/export.md) backend initializes. Set to `true` for immutable (write-once / WORM) buckets or least-privilege credentials that cannot delete objects. Applied at startup. Default: `false`<br/>Added in `v1.37.8` | `boolean` | `true` |
5354
| `GODEBUG` | Controls debugging variables within the runtime. [See official Go docs](https://pkg.go.dev/runtime). | `string - comma-separated list of name=val pairs` | `gctrace=1` |
5455
| `GOMAXPROCS` | Set the maximum number of threads that can be executing simultaneously. If this value is set, it be respected by `LIMIT_RESOURCES`. | `string - number` | `NUMBER_OF_CPU_CORES` |
5556
| `GOMEMLIMIT` | Set the memory limit for the Go runtime. A suggested value is between 80-90% of your total memory for Weaviate. The Go runtime tries to make sure that long-lived and temporary memory allocations do not exceed this value by making the garbage collector more aggressive as the memory usage approaches the limit. [Learn more about GOMEMLIMIT](https://weaviate.io/blog/gomemlimit-a-game-changer-for-high-memory-applications). | `string - memory limit in SI units` | `4096MiB` |

docs/deploy/configuration/export.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,9 @@ Set these [environment variables](/docs/deploy/configuration/env-vars/index.md)
3232
| `EXPORT_DEFAULT_BUCKET` | (empty) | Storage bucket name. Required for S3, GCS, and Azure backends. |
3333
| `EXPORT_DEFAULT_PATH` | `""` | Optional base path prefix for exported files within the bucket. Defaults to an empty string (no prefix). _Changed in `v1.37.1`: previously required to be explicitly set._ |
3434
| `EXPORT_PARALLELISM` | `0` (GOMAXPROCS) | Number of concurrent scan workers. |
35+
| `EXPORT_SKIP_ACCESS_CHECK` | `false` | Skip the write-and-delete access check that runs when the export backend initializes. Set to `true` for immutable (write-once / WORM) buckets or least-privilege credentials that cannot delete objects. _Added in `v1.37.8`._ |
3536

36-
All four variables are [runtime-configurable](/docs/deploy/configuration/env-vars/runtime-config.md) and can be changed without restarting Weaviate.
37+
`EXPORT_ENABLED`, `EXPORT_DEFAULT_BUCKET`, `EXPORT_DEFAULT_PATH`, and `EXPORT_PARALLELISM` are [runtime-configurable](/docs/deploy/configuration/env-vars/runtime-config.md) and can be changed without restarting Weaviate. `EXPORT_SKIP_ACCESS_CHECK` is applied at startup and requires a restart to change.
3738

3839
:::note Weaviate Cloud
3940

0 commit comments

Comments
 (0)