|
| 1 | +--- |
| 2 | +title: Resource Synchronization |
| 3 | +description: | |
| 4 | + Overview over the general resource synchronization logic and different isolation modes. |
| 5 | +weight: 220 |
| 6 | +--- |
| 7 | + |
| 8 | +# Resource Synchronization |
| 9 | + |
| 10 | +This document describes the way kube-bind synchronizes Kubernetes resources across clusters. |
| 11 | + |
| 12 | +## Overview |
| 13 | + |
| 14 | +kube-bind synchronizes objects between two kinds of clusters: |
| 15 | + |
| 16 | +* The **provider cluster** is run by a service provider and hosts a service, operator, or any other kind of Kubernetes API. This is also where there kube-bind **backend** is running in order to offer these APIs to consumers. |
| 17 | +* The **consumer clusters** are where endusers consume services/APIs offered by providers. |
| 18 | + |
| 19 | +There exists a 1:n relationship, where one provider cluster can be connected to from many different consumer clusters. |
| 20 | + |
| 21 | +### Cluster Namespaces |
| 22 | + |
| 23 | +In kube-bind, each successful `bind` operation yields a new, so-called "cluster namespace" on the provider cluster. This namespace contains all kube-bind-related resources, like `APIServiceExports` or `BoundSchemas` and is communicated to the consumer by being included in the provided kubeconfig. |
| 24 | + |
| 25 | +By default the cluster namespaces are named `kube-bind-[random string]`, like `kube-bind-hd73s`. Their name also serves as a unique identifier for this "contract" between consumer and provider and is used in other places, for example as a prefix in the `prefixed` cluster isolation mode. |
| 26 | + |
| 27 | +### Sync Direction |
| 28 | + |
| 29 | +In the kube-bind architecture, the consumer clusters represent the source of truth (the actual desired state by the enduser) and the provider clusters merely contain copies of those objects. |
| 30 | + |
| 31 | +During the synchronization, |
| 32 | + |
| 33 | +* the **spec** (desired state) of an object is copied from the consumer cluster to the provider and |
| 34 | +* the **status** (if any) is copied in the opposite direction, to the consumer. |
| 35 | + |
| 36 | +kube-bind will continuously watch the object and its copy on both clusters and update the other side as needed. |
| 37 | + |
| 38 | +### Connectivity |
| 39 | + |
| 40 | +The object synchronization logic lives in the kube-bind **konnector**, a Kubernetes agent that runs on each *consumer cluster*. It reads a local Secret that contains a kubeconfig pointing to **a specific namespace** on the *provider cluster*. In this namespace the konnector will find all resources describing the resources to sync (chiefly `APIServiceExports`, `APIServiceNamespaces` and `BoundSchemas`). |
| 41 | + |
| 42 | +!!! note |
| 43 | + This kubeconfig is automatically generated as part of the `kubectl bind` handshake with a service provider. |
| 44 | + |
| 45 | +This design allows consumer clusters to be mostly firewalled off, but requires provider clusters to not only be reachable from all consumers, but also to ensure multiple consumers do not conflict with each other. This is achieved by a combination of RBAC and isolation modes, which are described further down in this document. |
| 46 | + |
| 47 | +### RBAC |
| 48 | + |
| 49 | +Great care must be taken to ensure multiple consumers do not collide with each other on a single provider cluster. To achieve this, kube-bind offers two different informer scoping options: |
| 50 | + |
| 51 | +* `namespaced` will make the konnector watch and inform on each relevant namespace on the provider cluster individually. |
| 52 | +* `cluster` will instead make the konnector use a cluster-scoped (global) informer. This scoping method requires the konnector to have much wider permissions, but is more performant. |
| 53 | + |
| 54 | +Regardless of scoping mode, the konnector itself will take care to not overwrite/touch other consumers' objects. This however does not mean that an attacker with access to the konnector kubeconfig could not exploit an RBAC policy that is too wide. |
| 55 | + |
| 56 | +## Cluster Isolation |
| 57 | + |
| 58 | +This section outlines how kube-bind deals with differently scoped Kubernetes objects. |
| 59 | + |
| 60 | +In Kubernetes, an object can be either namespaced or cluster-scoped. This scope greatly affects how kube-bind processes the object. |
| 61 | + |
| 62 | +### Namespaced |
| 63 | + |
| 64 | +For namespaced objects (like a `Deployment`), kube-bind will map the consumer-side namespace to a unique, but random provider-side namespace: first the konnector will create an `APIServiceNamespace` object on the provider cluster (inside the cluster namespace), with the name of the consumer-side namespace (so a namespace `app1` will lead to an `APIServiceNamespace` `app1` in the cluster namespace). After this, the konnector waits until the backend has assigned this `APIServiceNamespace` a provider-side namespace to use. Once that namespace is present in the `APIServiceNamespace`'s status, the konnector will use it for all objects originating in the same consumer-side namespace. |
| 65 | + |
| 66 | +In YAML, this means |
| 67 | + |
| 68 | +```yaml |
| 69 | +# consumer-side object |
| 70 | + |
| 71 | +apiVersion: provider.example.com/v1 |
| 72 | +kind: MangoDB |
| 73 | +metadata: |
| 74 | + name: my-first-db |
| 75 | + namespace: team1 |
| 76 | +spec: |
| 77 | + size: large |
| 78 | +``` |
| 79 | +
|
| 80 | +will lead to |
| 81 | +
|
| 82 | +```yaml |
| 83 | +# provider-side objects |
| 84 | + |
| 85 | +apiVersion: kube-bind.io/v1alpha2 |
| 86 | +kind: APIServiceNamespace |
| 87 | +metadata: |
| 88 | + name: team1 # name of the namespace from the consumer |
| 89 | + namespace: kube-bind-hd73d # cluster namespace |
| 90 | +spec: {} |
| 91 | +status: |
| 92 | + # a common implementation in the backend is to construct the provider-side |
| 93 | + # namespace by just concatenating the two values, like so: |
| 94 | + namespace: kube-bind-hd73d-team1 |
| 95 | + |
| 96 | +--- |
| 97 | +apiVersion: v1 |
| 98 | +kind: Namespace |
| 99 | +metadata: |
| 100 | + name: kube-bind-hd73d-team1 |
| 101 | + |
| 102 | +--- |
| 103 | +apiVersion: provider.example.com/v1 |
| 104 | +kind: MangoDB |
| 105 | +metadata: |
| 106 | + name: my-first-db |
| 107 | + namespace: kube-bind-hd73d-team1 |
| 108 | +spec: |
| 109 | + size: large |
| 110 | +``` |
| 111 | +
|
| 112 | +!!! note |
| 113 | + For natively namespaced resources (those that are namespaced in both the provider and consumer cluster), this is always the strategy being used by the konnector. The other isolation modes only influence how the konnector deals with cluster-scoped objects in the consumer cluster. |
| 114 | +
|
| 115 | +### Cluster-Scoped |
| 116 | +
|
| 117 | +Cluster-scoped objects require a different approach to isolation than namespaced objects. kube-bind offers three different so-called isolation strategies to deal with them. The strategy to use is configured globally via the `--cluster-scoped-isolation` CLI flag on the kube-bind backend and will from there affect all services offered by that backend. |
| 118 | + |
| 119 | +#### None Strategy |
| 120 | + |
| 121 | +The `none` strategy providers no consumer-separation at all. Any cluster-scoped object on the consumer side is copied 1:1 to the provider side. |
| 122 | + |
| 123 | +!!! warning |
| 124 | + Due to the obvious downsides of this approach, `none` should be used only in special circumstances. |
| 125 | + |
| 126 | +#### Prefixed Strategy |
| 127 | + |
| 128 | +The `prefixed` strategy is kube-bind's default strategy and will use the name of the cluster namespace as a prefix for object names. |
| 129 | + |
| 130 | +In YAML, this means |
| 131 | + |
| 132 | +```yaml |
| 133 | +# consumer-side |
| 134 | +apiVersion: provider.example.com/v1 |
| 135 | +kind: MangoDB |
| 136 | +metadata: |
| 137 | + name: my-first-db |
| 138 | +spec: |
| 139 | + size: large |
| 140 | +``` |
| 141 | + |
| 142 | +will lead to |
| 143 | + |
| 144 | +```yaml |
| 145 | +# provider-side |
| 146 | +apiVersion: provider.example.com/v1 |
| 147 | +kind: MangoDB |
| 148 | +metadata: |
| 149 | + name: kube-bind-hd73d-my-first-db |
| 150 | +spec: |
| 151 | + size: large |
| 152 | +``` |
| 153 | + |
| 154 | +This strategy works well for separating objects, but |
| 155 | + |
| 156 | +* requires a broad RBAC policy, which will always grant too many permissions, and |
| 157 | +* would technically break for Kubernetes objects with names close to the maximum allowed length (253 characters). |
| 158 | + |
| 159 | +#### Namespaced Strategy |
| 160 | + |
| 161 | +!!! note |
| 162 | + Not to be confused with the strategy used for natively namespaced resources, described earlier. |
| 163 | + |
| 164 | +The `namespaced` strategy will convert cluster-scoped objects into namespaced ones. |
| 165 | + |
| 166 | +For this to work, the original CRD on the provider cluster has to be **namespaced**. The backend will turn it into a cluster-scoped CRD (stored in the `BoundSchema`), so on the consumer cluster, all objects are cluster-scoped. |
| 167 | + |
| 168 | +During synchronization, the konnector will then place each cluster-scoped object into the cluster namespace (`kube-bind-hd73d`) on the provider side. |
| 169 | + |
| 170 | +This strategy provides excellent isolation between consumers, but requires that the original CRD from the provider still makes sense to the consumer when it's suddenly cluster-scoped. For example, if references were to be used, especially to Secrets in the same namespace, this concept would get mangled to some degree during the synchronization. |
| 171 | + |
| 172 | +!!! warning |
| 173 | + At the moment, when the backend is started with `--cluster-scoped-isolation=namespaced`, it will convert **all namespaced CRDs** in all ServiceExports to become cluster-scoped on the consumer side, even if you intended for a namespaced CRD to stay namespaced on both sides of the sync. |
0 commit comments