Skip to content

Commit 5d3ed5e

Browse files
xrstfxrstf
committed
add docs for the isolation feature, fix hard-to-reach usage page
On-behalf-of: @SAP christoph.mewes@sap.com
1 parent 0d60961 commit 5d3ed5e

3 files changed

Lines changed: 179 additions & 1 deletion

File tree

‎docs/content/usage/.pages‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
11
nav:
2-
- api-concepts.md
2+
- index.md
3+
- api-concepts.md
4+
- synchronization.md

‎docs/content/usage/synchronization.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
---
2+
title: Resource Synchronization
3+
description: |
4+
Deep dive into kube-bind's core API types, their relationships, and how they work together to enable cross-cluster service binding.
5+
weight: 210
6+
---
7+
8+
# Resource Synchronization
9+
10+
This document describes the way kube-bind synchronizes Kubernetes resources across clusters.
11+
12+
## Overview
13+
14+
kube-bind synchronizes objects between two kinds of clusters:
15+
16+
* The **provider cluster** is run by a service provider and hosts a service, operator, or any other kind of Kubernetes API. This is also where there kube-bind **backend** is running in order to offer these APIs to consumers.
17+
* The **consumer clusters** are where endusers consume services/APIs offered by providers.
18+
19+
There exists a 1:n relationship, where one provider cluster can be connected to from many different consumer clusters.
20+
21+
### Cluster Namespaces
22+
23+
In kube-bind, each successful `bind` operation yields a new, so-called "cluster namespace" on the provider cluster. This namespace contains all kube-bind-related resources, like `APIServiceExports` or `BoundSchemas` and is communicated to the consumer by being included in the provided kubeconfig.
24+
25+
By default the cluster namespaces are named `kube-bind-[random string]`, like `kube-bind-hd73s`. Their name also serves as a unique identifier for this "contract" between consumer and provider and is used in other places, for example as a prefix in the `prefixed` cluster isolation mode.
26+
27+
### Sync Direction
28+
29+
In the kube-bind architecture, the consumer clusters represent the source of truth (the actual desired state by the enduser) and the provider clusters merely contain copies of those objects.
30+
31+
During the synchronization,
32+
33+
* the **spec** (desired state) of an object is copied from the consumer cluster to the provider and
34+
* the **status** (if any) is copied in the opposite direction, to the consumer.
35+
36+
kube-bind will continuously watch the object and its copy on both clusters and update the other side as needed.
37+
38+
### Connectivity
39+
40+
The object synchronization logic lives in the kube-bind **konnector**, a Kubernetes agent that runs on each *consumer cluster*. It reads a local Secret that contains a kubeconfig pointing to **a specific namespace** on the *provider cluster*. In this namespace the konnector will find all resources describing the resources to sync (chiefly `APIServiceExports`, `APIServiceNamespaces` and `BoundSchemas`).
41+
42+
!!! note
43+
This kubeconfig is automatically generated as part of the `kubectl bind` handshake with a service provider.
44+
45+
This design allows consumer clusters to be mostly firewalled off, but requires provider clusters to not only be reachable from all consumers, but also to ensure multiple consumers do not conflict with each other. This is achieved by a combination of RBAC and isolation modes, which are described further down in this document.
46+
47+
### RBAC
48+
49+
Great care must be taken to ensure multiple consumers do not collide with each other on a single provider cluster. To achieve this, kube-bind offers two different informer scoping options:
50+
51+
* `namespaced` will make the konnector watch and inform on each relevant namespace on the provider cluster individually.
52+
* `cluster` will instead make the konnector use a cluster-scoped (global) informer. This scoping method requires the konnector to have much wider permissions, but is more performant.
53+
54+
Regardless of scoping mode, the konnector itself will take care to not overwrite/touch other consumers' objects. This however does not mean that an attacker with access to the konnector kubeconfig could not exploit an RBAC policy that is too wide.
55+
56+
## Cluster Isolation
57+
58+
This section outlines how kube-bind deals with differently scoped Kubernetes objects.
59+
60+
In Kubernetes, an object can be either namespaced or cluster-scoped. This scope greatly affects how kube-bind processes the object.
61+
62+
### Namespaced
63+
64+
For namespaced objects (like a `Deployment`), kube-bind will map the consumer-side namespace to a unique, but random provider-side namespace: first the konnector will create an `APIServiceNamespace` object on the provider cluster (inside the cluster namespace), with the name of the consumer-side namespace (so a namespace `app1` will lead to an `APIServiceNamespace` `app1` in the cluster namespace). After this, the konnector waits until the backend has assigned this `APIServiceNamespace` a provider-side namespace to use. Once that namespace is present in the `APIServiceNamespace`'s status, the konnector will use it for all objects originating in the same consumer-side namespace.
65+
66+
In YAML, this means
67+
68+
```yaml
69+
# consumer-side object
70+
71+
apiVersion: provider.example.com/v1
72+
kind: MangoDB
73+
metadata:
74+
name: my-first-db
75+
namespace: team1
76+
spec:
77+
size: large
78+
```
79+
80+
will lead to
81+
82+
```yaml
83+
# provider-side objects
84+
85+
apiVersion: kube-bind.io/v1alpha2
86+
kind: APIServiceNamespace
87+
metadata:
88+
name: team1 # name of the namespace from the consumer
89+
namespace: kube-bind-hd73d # cluster namespace
90+
spec: {}
91+
status:
92+
# a common implementation in the backend is to construct the provider-side
93+
# namespace by just concatenating the two values, like so:
94+
namespace: kube-bind-hd73d-team1
95+
96+
---
97+
apiVersion: v1
98+
kind: Namespace
99+
metadata:
100+
name: kube-bind-hd73d-team1
101+
102+
---
103+
apiVersion: provider.example.com/v1
104+
kind: MangoDB
105+
metadata:
106+
name: my-first-db
107+
namespace: kube-bind-hd73d-team1
108+
spec:
109+
size: large
110+
```
111+
112+
!!! note
113+
For natively namespaced resources (those that are namespaced in both the provider and consumer cluster), this is always the strategy being used by the konnector. The other isolation modes only influence how the konnector deals with cluster-scoped objects in the consumer cluster.
114+
115+
### Cluster-Scoped
116+
117+
Cluster-scoped objects require a different approach to isolation than namespaced objects. kube-bind offers three different so-called isolation strategies to deal with them. The strategy to use is configured globally via the `--cluster-scoped-isolation` CLI flag on the kube-bind backend and will from there affect all services offered by that backend.
118+
119+
#### None Strategy
120+
121+
The `none` strategy providers no consumer-separation at all. Any cluster-scoped object on the consumer side is copied 1:1 to the provider side.
122+
123+
!!! warning
124+
Due to the obvious downsides of this approach, `none` should be used only in special circumstances.
125+
126+
#### Prefixed Strategy
127+
128+
The `prefixed` strategy is kube-bind's default strategy and will use the name of the cluster namespace as a prefix for object names.
129+
130+
In YAML, this means
131+
132+
```yaml
133+
# consumer-side
134+
apiVersion: provider.example.com/v1
135+
kind: MangoDB
136+
metadata:
137+
name: my-first-db
138+
spec:
139+
size: large
140+
```
141+
142+
will lead to
143+
144+
```yaml
145+
# provider-side
146+
apiVersion: provider.example.com/v1
147+
kind: MangoDB
148+
metadata:
149+
name: kube-bind-hd73d-my-first-db
150+
spec:
151+
size: large
152+
```
153+
154+
This strategy works well for separating objects, but
155+
156+
* requires a broad RBAC policy, which will always grant too many permissions, and
157+
* would technically break for Kubernetes objects with names close to the maximum allowed length (253 characters).
158+
159+
#### Namespaced Strategy
160+
161+
!!! note
162+
Not to be confused with the strategy used for natively namespaced resources, described earlier.
163+
164+
The `namespaced` strategy will convert cluster-scoped objects into namespaced ones.
165+
166+
For this to work, the original CRD on the provider cluster has to be **namespaced**. The backend will turn it into a cluster-scoped CRD (stored in the `BoundSchema`), so on the consumer cluster, all objects are cluster-scoped.
167+
168+
During synchronization, the konnector will then place each cluster-scoped object into the cluster namespace (`kube-bind-hd73d`) on the provider side.
169+
170+
This strategy provides excellent isolation between consumers, but requires that the original CRD from the provider still makes sense to the consumer when it's suddenly cluster-scoped. For example, if references were to be used, especially to Secrets in the same namespace, this concept would get mangled to some degree during the synchronization.
171+
172+
!!! warning
173+
At the moment, when the backend is started with `--cluster-scoped-isolation=namespaced`, it will convert **all namespaced CRDs** in all ServiceExports to become cluster-scoped on the consumer side, even if you intended for a namespaced CRD to stay namespaced on both sides of the sync.

‎docs/requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,6 @@ mkdocs-macros-plugin==1.0.5
55
mkdocs-material==9.5.49
66
mkdocs-material-extensions==1.3.1
77
mkdocs-static-i18n==1.2.2
8+
9+
# https://github.com/mkdocs/mkdocs/issues/4032
10+
click<=8.2.1

0 commit comments

Comments
 (0)