You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/endpointslice-controller/README.md
+40-51Lines changed: 40 additions & 51 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,25 +4,25 @@ The EndpointSlice operator is a "hybrid" Kubernetes controller that demonstrates
4
4
5
5
## Description
6
6
7
-
Certain use cases cannot be fully implemented in a purely declarative style, for instance because the Kubernetes operator needs to manipulate an imperative API. Such is the case if the task is to implement an **endpoint-discovery service**to collect the endpoints for a Kubernetes service and program an underlying system, say, a service mesh proxy, with the discovered endpoints. Δ-controller can come in handy in such cases as well, by letting the difficult part of the operator, the endpoint-discovery pipeline, to be implemented in a declarative form, leaving only the reconciliation logic, which updates the imperative API based on the endpoints discovered by the declarative controller, to be written in imperative Go.
7
+
Certain use cases cannot be fully implemented in a purely declarative style, for instance because a Kubernetes operator needs to manipulate an imperative API. Such is the case if the task is to implement an **endpoint-discovery service**in order to program an underlying system (say, a service mesh proxy) with the endpoints for a Kubernetes service (e.g., for load-balancing). Δ-controller can come in handy in such cases as well, by letting the difficult part of the operator, the endpoint-discovery pipeline, to be implemented in a declarative form, leaving only the reconciliation logic, which updates the imperative API based on the endpoints discovered by the declarative controller, to be written in imperative Go.
8
8
9
9
This example demonstrates the use of Δ-controller in such a use case. The example code comprises two parts: an imperative **endpoint-discovery operator** that is written in Go using the Δ-controller API, and a declarative **controller pipeline** that automates the difficult part: generating the up-to-date list of endpoints for a Kubernetes Service based on the Kubernetes resources obtained from the API server.
10
10
11
11
### The controller pipeline
12
12
13
13
The declarative controller pipeline spec is read from a YAML manifest. There are two versions:
14
-
-`endpointslice-controller-spec.yaml`: this is the default spec, that will generate a separate view object per each (service, service-port, endpoint-address) combination. This is the one we discuss below.
15
-
-`endpointslice-controller-gather-spec.yaml`: the alternative spec gathers all separate endpoint addresses into view object that will hold a list of all the endpoints addresses for a (service, service-port) combination. This is mostly the same as the default spec, but it contains a final "gather" aggregation stage that will collapse the endpoint addresses into a list. See the YAML spec for the details.
14
+
-`endpointslice-controller-spec.yaml`: this is the default spec, which will generate a separate view object per each (service, service-port, endpoint-address) combination. This is the one we discuss below.
15
+
-`endpointslice-controller-gather-spec.yaml`: the alternative spec gathers all endpoint addresses into a single view object per (service, service-port) combination. This pipeline is mostly the same as the default spec but it contains a final "gather" aggregation stage that will collapse the endpoint addresses into a list. See the YAML for the details.
16
16
17
-
The default declarative pipeline to controllers:
18
-
- the `service-controller` will watch the Kubernetes core.v1 Service API, generate a separate out object per each service-port, and load the resultant objects into a view called ServiceView.
19
-
- the `endpointslice-controller` watches the objects from the ServiceView and the EndpointSlice objects from the Kubernetes discovery.v1 API, pair service view objects with the corresponding EndpointSlices to expand it with the endpoint-addresses, filter addresses with `ready` status, demultiplex the endpoint addresses into a separate object and converts the shape of the resultant objects into a simpler form, and then load the results into a view called EndpointView.
17
+
The default declarative pipeline defines to controllers:
18
+
- the `service-controller` will watch the Kubernetes core.v1 Service API, generate a separate object per each service-port, and load the resultant objects into an internal view called ServiceView.
19
+
- the `endpointslice-controller` watches the objects from the ServiceView and the EndpointSlice objects from the Kubernetes discovery.v1 API, pairs service view objects with the corresponding EndpointSlices to match it with the endpoint-addresses, filters addresses with `ready` status, demultiplexes the endpoint addresses into separate objects, converts the resultant objects into a simpler form, and then load the results into a view called EndpointView.
20
20
21
-
The idea is that the imperative controller will watch this EndpointView, which already contains the objects demultplexed and converted into the shape we want, instead of us having to write the imperative code to watch the Kubernetes API and perform the conversions ourselves. This helps cutting down development costs.
21
+
The idea is that the imperative controller will watch this EndpointView to learn all the (service, service-port, endpoint-address) combinations in the form of a view object that is converted into a convenient shape that can be used just like any regular Kubernetes object. This is much simpler than writing the entire logic to watch the Kubernetes API and perform all the joins and conversions in go.
22
22
23
23
#### The Service controller
24
24
25
-
The task of this service is to generate a single object per each service-port in the watched Services. To simplify the task we will process only the Services annotated with `dcontroller.io/endpointslice-controller-enabled`.
25
+
The first controller will generate an object per each service-port per the watched Services to fill the ServiceView. To simplify the task we will process only the Services annotated with `dcontroller.io/endpointslice-controller-enabled`.
26
26
27
27
The pipeline is as follows.
28
28
@@ -36,11 +36,9 @@ The pipeline is as follows.
36
36
kind: Service
37
37
```
38
38
39
-
3. Create the **aggregation** pipeline.
40
-
41
-
This contains a single aggregation with 3 stages:
42
-
- filter the Services annotated with `dcontroller.io/endpointslice-controller-enabled`. Note that we use the long-form JSONpath expression `$["metadata"][..."]` because the annotation contains a `/` that is incompatible with the simpler short-form,
43
-
- remove some useless fields and converts the shape of the resultant objects,
39
+
3. Create the **aggregation** pipeline. The aggregation consists of 3 stages:
40
+
- filter the Services annotated with `dcontroller.io/endpointslice-controller-enabled`. Note that we use the long-form JSONpath expression format `$["metadata"][..."]` because the annotation contains a `/` that is incompatible with the simpler short-form,
41
+
- remove some useless fields and convert the shape of the resultant objects,
44
42
- demultiplex the result into multiple objects by blowing up the `$.spec.ports` list.
45
43
46
44
The pipeline is as follows:
@@ -98,14 +96,12 @@ The pipeline is as follows.
98
96
- $.EndpointSlice.metadata.namespace
99
97
```
100
98
101
-
4. Create the **aggregation** pipeline to convert the shape of the resultant, somewhat convoluted objects.
102
-
103
-
The aggregation pipeline again has multiple stages:
104
-
- set up the metadata, copy the ServiceView spec and the endpoints from the EndpointSlice, and create an `id` that will be used later to generate a unique stable name for the resultant objects,
99
+
4. Create the **aggregation** pipeline to convert the shape of the resultant, somewhat convoluted objects. The aggregation again has multiple stages:
100
+
- set up the metadata, copy the ServiceView spec and the endpoints from the EndpointSlice,
105
101
- demultiplex on the `$.endpoint` list,
106
102
- filter ready addresses,
107
103
- demultiplex again, now on the `$.endpoint.addresses` field of the original EndpointSlice object, and
108
-
- finally again convert the object shape: set a unique name by concatenating Service name with the hash of the object id (this name will be different per each (service, service-port, endpoint-address)) and copy the relevant fields of the spec.
104
+
- finally convert the object into a simple shape and set a unique stable object name by concatenating Service name with the hash of the object spec.
109
105
110
106
```yaml
111
107
"@aggregate":
@@ -115,32 +111,28 @@ The pipeline is as follows.
115
111
namespace: $.ServiceView.metadata.namespace
116
112
spec: $.ServiceView.spec
117
113
endpoints: $.EndpointSlice.endpoints
118
-
id:
119
-
name: $.ServiceView.spec.serviceName
120
-
namespace: $.ServiceView.metadata.namespace
121
-
type: $.ServiceView.spec.type
122
-
protocol: $.ServiceView.spec.ports.protocol
123
-
port: $.ServiceView.spec.ports.port
124
-
targetPort: $.ServiceView.spec.ports.targetPort
125
114
- "@unwind": $.endpoints
126
115
- "@select":
127
116
"@eq": ["$.endpoints.conditions.ready", true]
128
117
- "@unwind": $.endpoints.addresses
129
118
- "@project":
130
119
metadata:
131
-
name:
132
-
"@concat":
133
-
- $.metadata.name
134
-
- "-"
135
-
- { "@hash": $.id }
136
120
namespace: $.metadata.namespace
137
121
spec:
138
122
serviceName: $.spec.serviceName
139
123
type: $.spec.type
140
-
port: $.id.port
141
-
targetPort: $.id.targetPort
142
-
protocol: $.id.protocol
124
+
port: $.spec.ports.port
125
+
targetPort: $.spec.ports.targetPort
126
+
protocol: $.spec.ports.protocol
143
127
address: $.endpoints.addresses
128
+
- "@project":
129
+
"@merge":
130
+
metadata:
131
+
name:
132
+
"@concat":
133
+
- $.spec.serviceName
134
+
- "-"
135
+
- { "@hash": $.spec }
144
136
```
145
137
146
138
5. Set up the **target** to update the EndpointView view with the results.
@@ -154,11 +146,11 @@ The pipeline is as follows.
154
146
155
147
### The endoint-discovery operator
156
148
157
-
The operator will be written in Go. This will watch the EndpointView view as generated by the declarative pipeline for updates and implement the reconciliation logic. Again, the idea is that we don't have to write the tedious join+aggregation pipeline in imperative Go, instead we can implement this logic in a purely declarative style.
149
+
The endoint-discovery operator will process the events on the EndpointView view. Since the reconciliation logic often needs to interact with an imperative API (say, to program a service-mesh proxy), this part will be written in Go. Recall, the idea is that we don't want to write the tedious join+aggregation pipeline in imperative Go; rather we implement just a minimal part in Go while the complex data manipulation logic will be handled in a purely declarative style (see above).
158
150
159
-
The Go code itself will differ too much from a standard [Kubernetes operator](https://book.kubebuilder.io/), just with the common packages taken from Δ-controller instead of the usual [Kubernetes controller runtime](https://pkg.go.dev/sigs.k8s.io/controller-runtime).
151
+
The Go code itself will not differ too much from a standard [Kubernetes operator](https://book.kubebuilder.io/), just with the common packages taken from Δ-controller instead of the usual [Kubernetes controller runtime](https://pkg.go.dev/sigs.k8s.io/controller-runtime).
160
152
161
-
1. Define the usual Kubernetes operator boilerplate by importing packages, defining constants, parsing the command line arguments, and setting up a logger.
153
+
1. Define the usual boilerplate: import packages, define constants, parse command line arguments, and set up a logger.
162
154
163
155
2. Create a Δ-controller manager:
164
156
@@ -169,15 +161,15 @@ The Go code itself will differ too much from a standard [Kubernetes operator](ht
169
161
if err != nil { ... }
170
162
```
171
163
172
-
3. Load the declarative operator pipeline from a file held in `specFile` (see later):
164
+
3. Load the declarative controllers we have implemented above:
@@ -224,18 +216,13 @@ The constructor will be called `NewEndpointSliceController`:
224
216
if err != nil { ... }
225
217
```
226
218
227
-
4. And finally set up a watch that will bind our controller to the above source so that every time
228
-
there is an update on the `EndpointView` our `Reconcile(...)` function will be called with
229
-
the event to process it.
219
+
4. And finally set up a watch that will bind our controller to the above source so that every time there is an update on the `EndpointView` our `Reconcile(...)` function will be called with the update event.
230
220
231
221
```go
232
222
if err := c.Watch(src); err != nil { ... }
233
223
```
234
224
235
-
And finally the most important part, the `Reconcile(...)` function. Normally, this would be the
236
-
function that implements the business logic of our operator, say, by controlling a proxy with the
237
-
endpoints discovered by our operator. Here for simplicity we will just log the events and return a
238
-
successful reconcile result.
225
+
And finally the most important part, the `Reconcile(...)` function. Normally, this would be the function that implements the business logic of our operator, say, by programming a proxy with the endpoints discovered by our operator. Here for simplicity we will just log the events and return a successful reconcile result.
And that's all: the operative part is only some 200 lines of code, at least an order of magnitude less than if we had to implement the entire operator logic in an imperative style.
252
+
264
253
## Testing
265
254
266
255
Take off from an empty cluster and start the EndpointSlice controller:
@@ -270,7 +259,7 @@ cd <project-root>
270
259
go run examples/endpointslice-controller/main.go -zap-log-level info -disable-endpoint-pooling
The controller will emit 4 `Add` events for each object generated by the EndpointSlice controller, one per each (service, service-port, endpoint-address) combination:
291
+
The controller now will emit 4 `Add` events for each object generated by the EndpointSlice controller, one per each (service, service-port, endpoint-address) combination:
0 commit comments