Skip to content

Commit 4f073b3

Browse files
authored
Split out minio docs and document network isolation (#33)
1 parent a785b0c commit 4f073b3

2 files changed

Lines changed: 167 additions & 140 deletions

File tree

README.md

Lines changed: 19 additions & 140 deletions
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,21 @@ The operator introduces two Custom Resource Definitions (CRDs): `RestateCluster`
5050

5151
The `RestateCluster` CRD defines a Restate cluster. The operator watches for these objects and creates the necessary Kubernetes resources, such as `StatefulSet`, `Service`, and `NetworkPolicy` objects in a new namespace that matches the `RestateCluster` name.
5252

53+
**By default, the operator enforces network isolation on the cluster, allowing only the following**:
54+
1. Peer to peer traffic between Restate pods
55+
2. Traffic from the operator to Restate pods
56+
3. Egress traffic to the public internet
57+
4. Egress traffic to coredns
58+
4. Egress traffic to pods in any namespace labelled with `allow.restate.dev/<cluster-name>`
59+
60+
**All other traffic is denied by default.**
61+
62+
The default behaviour can be disabled with `spec.security.disableNetworkPolicies: true`.
63+
Alternatively, you can add new allowed inbound callers of the Restate ports with `spec.security.networkPeers.{ingress,admin,metrics}`, which are arrays of [`NetworkPolicyPeer`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#networkpolicypeer-v1-networking-k8s-io).
64+
You can allow new outbound destinations by adding to the `spec.security.networkEgressRules` list, which is an array of [`NetworkPolicyEgressRule`](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#networkpolicyegressrule-v1-networking-k8s-io).
65+
5366
**NOTE**: Each cluster is created in its own namespace (enforced by the operator). Do not create the namespace manually,
54-
and do not use the same namespace as the operator is in.
67+
and do not use the same namespace that the operator is in.
5568

5669
#### Minimal Example
5770

@@ -69,6 +82,8 @@ spec:
6982
storageRequestBytes: 2147483648 # 2 GiB
7083
```
7184
85+
For the full schema as a [Pkl](https://pkl-lang.org/) template see [`crd/RestateCluster.pkl`](./crd/RestateCluster.pkl).
86+
7287
More examples are available just below the spec that follows.
7388

7489
#### Spec Fields
@@ -269,146 +284,10 @@ spec:
269284
# aws-allow-http = true
270285
```
271286

272-
#### MinIO Configuration Example
273-
274-
To configure a `RestateCluster` with a self-hosted S3-compatible object store like [MinIO](https://min.io/), you can point the server to your MinIO instance. For security, it's best to create a dedicated service account with credentials scoped only to the buckets Restate needs.
275-
276-
##### 1. Create a Scoped Access Key & Buckets
277-
278-
First, we'll define a policy, create the buckets, and then create a service account with a new access key that is restricted by that policy.
279-
280-
The following commands use the `mc` client to set up your MinIO instance. They assume you have port-forwarded your MinIO service.
281-
282-
```bash
283-
# Forward the MinIO service to your local machine
284-
kubectl port-forward --namespace storage svc/minio 9000:443
285-
286-
# Alias your MinIO deployment for easier access
287-
# Use https:// and --insecure if connecting to a port-forwarded TLS port (like 443)
288-
mc alias set local-minio https://localhost:9000 YOUR_ADMIN_ACCESS_KEY YOUR_ADMIN_SECRET_KEY --insecure
289-
290-
# Create the buckets
291-
mc mb --insecure local-minio/restate-metadata
292-
mc mb --insecure local-minio/restate-snapshots
293-
294-
# Add the new policy to MinIO
295-
296-
cat <<EOF | mc admin policy add local-minio restate-s3-policy
297-
{
298-
"Version": "2012-10-17",
299-
"Statement": [
300-
{
301-
"Effect": "Allow",
302-
"Action": [
303-
"s3:GetBucketLocation",
304-
"s3:ListBucket"
305-
],
306-
"Resource": [
307-
"arn:aws:s3:::restate-metadata",
308-
"arn:aws:s3:::restate-snapshots"
309-
]
310-
},
311-
{
312-
"Effect": "Allow",
313-
"Action": [
314-
"s3:PutObject",
315-
"s3:GetObject",
316-
"s3:DeleteObject",
317-
"s3:ListMultipartUploadParts",
318-
"s3:AbortMultipartUpload"
319-
],
320-
"Resource": [
321-
"arn:aws:s3:::restate-metadata/*",
322-
"arn:aws:s3:::restate-snapshots/*"
323-
]
324-
}
325-
]
326-
}
327-
EOF
328-
329-
# Create a new service account for the Restate application
330-
# This command will output a new AccessKey and SecretKey.
331-
mc admin service-account add local-minio restate
332-
333-
# Attach the policy to the new service account
334-
mc admin policy set local-minio restate-s3-policy service-account=restate
335-
```
336-
337-
When you run `mc admin service-account add`, it will output a new `AccessKey` and `SecretKey`. **Save these securely**, as you will use them in the next step.
338-
339-
##### 2. Create a Kubernetes Secret
340-
341-
Next, create a Kubernetes `Secret` containing the new, **scoped credentials** you just generated for the `restate` service account.
342-
343-
**Important**: This secret must be created in the namespace where the cluster will run, which is the same as the `metadata.name` of your `RestateCluster`. For this example, the namespace is `restate-with-minio`.
344-
345-
```yaml
346-
apiVersion: v1
347-
kind: Secret
348-
metadata:
349-
name: minio-credentials
350-
namespace: restate-with-minio
351-
type: Opaque
352-
stringData:
353-
# Use the keys generated from the 'mc admin service-account add' command
354-
AWS_ACCESS_KEY_ID: YOUR_NEW_RESTATE_ACCESS_KEY
355-
AWS_SECRET_ACCESS_KEY: YOUR_NEW_RESTATE_SECRET_KEY
356-
```
357-
358-
##### 3. Configure the `RestateCluster`
359-
360-
Finally, define your `RestateCluster` resource. This manifest is the same as before, but it will now use the Kubernetes secret containing the limited-permission keys.
361-
362-
```yaml
363-
apiVersion: restate.dev/v1
364-
kind: RestateCluster
365-
metadata:
366-
name: restate-minio-test
367-
spec:
368-
compute:
369-
replicas: 3
370-
image: restatedev/restate:1.4
371-
env:
372-
- name: AWS_ACCESS_KEY_ID
373-
valueFrom:
374-
secretKeyRef:
375-
name: minio-credentials
376-
key: AWS_ACCESS_KEY_ID
377-
- name: AWS_SECRET_ACCESS_KEY
378-
valueFrom:
379-
secretKeyRef:
380-
name: minio-credentials
381-
key: AWS_SECRET_ACCESS_KEY
382-
storage:
383-
storageRequestBytes: 2147483648 # 2 GiB
384-
config: |
385-
roles = [ "worker", "admin", "log-server", "http-ingress" ]
386-
auto-provision = true
387-
default-num-partitions = 128
388-
default-replication = 2
389-
390-
[metadata-client]
391-
type = "object-store"
392-
path = "s3://restate-metadata/metadata"
393-
aws-endpoint-url = "http://minio.minio-namespace.svc.cluster.local:9000"
394-
aws-allow-http = true
395-
aws-region = "local"
396-
397-
[bifrost]
398-
default-provider = "replicated"
399-
400-
[worker.snapshots]
401-
destination = "s3://restate-snapshots/snapshots"
402-
snapshot-interval-num-records = 10000
403-
aws-endpoint-url = "http://minio.minio-namespace.svc.cluster.local:9000"
404-
aws-allow-http = true
405-
aws-region = "local"
406-
```
407-
408-
> **A Note on AWS Credentials**: You might notice we are using standard AWS environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) directly, rather than the Restate-specific format like `RESTATE_WORKER__SNAPSHOTS_AWS_ACCESS_KEY_ID`. This is because Restate uses the underlying AWS SDK, which automatically and conventionally discovers credentials from these standard environment variables. However, if you are using Minio for snapshots and also need to use AWS Lambda services from your Restate cluster, then you may need different AWS credentials for different components.
409-
410-
For the full schema as a [Pkl](https://pkl-lang.org/) template see [`crd/RestateCluster.pkl`](./crd/RestateCluster.pkl).
287+
Note that Restate needs `s3:ListBucket` on the bucket, and `s3:GetObject`/`s3:PutObject` on the bucket contents.
411288

289+
#### MinIO example
290+
See [docs/minio.md](./docs/minio.md)
412291

413292
### `RestateDeployment`
414293

docs/minio.md

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# MinIO Configuration Example
2+
3+
To configure a `RestateCluster` with a self-hosted S3-compatible object store like [MinIO](https://min.io/), you can point the server to your MinIO instance. For security, it's best to create a dedicated service account with credentials scoped only to the buckets Restate needs.
4+
5+
## 1. Create a Scoped Access Key & Buckets
6+
7+
First, we'll define a policy, create the buckets, and then create a service account with a new access key that is restricted by that policy.
8+
9+
The following commands use the `mc` client to set up your MinIO instance. They assume you have port-forwarded your MinIO service.
10+
11+
```bash
12+
# Forward the MinIO service to your local machine
13+
kubectl port-forward --namespace storage svc/minio 9000:443
14+
15+
# Alias your MinIO deployment for easier access
16+
# Use https:// and --insecure if connecting to a port-forwarded TLS port (like 443)
17+
mc alias set local-minio https://localhost:9000 YOUR_ADMIN_ACCESS_KEY YOUR_ADMIN_SECRET_KEY --insecure
18+
19+
# Create the buckets
20+
mc mb --insecure local-minio/restate-metadata
21+
mc mb --insecure local-minio/restate-snapshots
22+
23+
# Add the new policy to MinIO
24+
25+
cat <<EOF | mc admin policy add local-minio restate-s3-policy
26+
{
27+
"Version": "2012-10-17",
28+
"Statement": [
29+
{
30+
"Effect": "Allow",
31+
"Action": [
32+
"s3:ListBucket"
33+
],
34+
"Resource": [
35+
"arn:aws:s3:::restate-metadata",
36+
"arn:aws:s3:::restate-snapshots"
37+
]
38+
},
39+
{
40+
"Effect": "Allow",
41+
"Action": [
42+
"s3:PutObject",
43+
"s3:GetObject",
44+
],
45+
"Resource": [
46+
"arn:aws:s3:::restate-metadata/*",
47+
"arn:aws:s3:::restate-snapshots/*"
48+
]
49+
}
50+
]
51+
}
52+
EOF
53+
54+
# Create a new service account for the Restate application
55+
# This command will output a new AccessKey and SecretKey.
56+
mc admin service-account add local-minio restate
57+
58+
# Attach the policy to the new service account
59+
mc admin policy set local-minio restate-s3-policy service-account=restate
60+
```
61+
62+
When you run `mc admin service-account add`, it will output a new `AccessKey` and `SecretKey`. **Save these securely**, as you will use them in the next step.
63+
64+
### 2. Create a Kubernetes Secret
65+
66+
Next, create a Kubernetes `Secret` containing the new, **scoped credentials** you just generated for the `restate` service account.
67+
68+
**Important**: This secret must be created in the namespace where the cluster will run, which is the same as the `metadata.name` of your `RestateCluster`. For this example, the namespace is `restate-with-minio`.
69+
70+
```yaml
71+
apiVersion: v1
72+
kind: Secret
73+
metadata:
74+
name: minio-credentials
75+
namespace: restate-with-minio
76+
type: Opaque
77+
stringData:
78+
# Use the keys generated from the 'mc admin service-account add' command
79+
access-key: YOUR_NEW_RESTATE_ACCESS_KEY
80+
secret-key: YOUR_NEW_RESTATE_SECRET_KEY
81+
```
82+
83+
##### 3. Configure the `RestateCluster`
84+
85+
Finally, define your `RestateCluster` resource. This manifest is the same as before, but it will now use the Kubernetes secret containing the limited-permission keys.
86+
Additionally, we add the MinIO namespace to the allowed list of egress peers.
87+
88+
```yaml
89+
apiVersion: restate.dev/v1
90+
kind: RestateCluster
91+
metadata:
92+
name: restate-minio-test
93+
spec:
94+
compute:
95+
replicas: 3
96+
image: restatedev/restate:1.4
97+
env:
98+
- name: RESTATE_METADATA_CLIENT__AWS_ACCESS_KEY_ID
99+
valueFrom:
100+
secretKeyRef:
101+
name: minio-credentials
102+
key: access-key
103+
- name: RESTATE_METADATA_CLIENT__AWS_SECRET_ACCESS_KEY
104+
valueFrom:
105+
secretKeyRef:
106+
name: minio-credentials
107+
key: secret-key
108+
- name: RESTATE_WORKER__SNAPSHOTS__AWS_ACCESS_KEY_ID
109+
valueFrom:
110+
secretKeyRef:
111+
name: minio-credentials
112+
key: access-key
113+
- name: RESTATE_WORKER__SNAPSHOTS__AWS_SECRET_ACCESS_KEY
114+
valueFrom:
115+
secretKeyRef:
116+
name: minio-credentials
117+
key: secret-key
118+
security:
119+
networkEgressRules:
120+
- to:
121+
- namespaceSelector:
122+
matchLabels:
123+
kubernetes.io/metadata.name: minio-namespace
124+
storage:
125+
storageRequestBytes: 2147483648 # 2 GiB
126+
config: |
127+
roles = [ "worker", "admin", "log-server", "http-ingress" ]
128+
auto-provision = true
129+
default-num-partitions = 128
130+
default-replication = 2
131+
132+
[metadata-client]
133+
type = "object-store"
134+
path = "s3://restate-metadata/metadata"
135+
aws-endpoint-url = "http://minio.minio-namespace.svc.cluster.local:9000"
136+
aws-allow-http = true
137+
aws-region = "local"
138+
139+
[bifrost]
140+
default-provider = "replicated"
141+
142+
[worker.snapshots]
143+
destination = "s3://restate-snapshots/snapshots"
144+
snapshot-interval-num-records = 10000
145+
aws-endpoint-url = "http://minio.minio-namespace.svc.cluster.local:9000"
146+
aws-allow-http = true
147+
aws-region = "local"
148+
```

0 commit comments

Comments
 (0)