Skip to content

Commit 636a630

Browse files
c-piusnataliasitko
andauthored
docs: Align 'Configuring Runtime Watcher' setup guide with implementation (#712)
* docs: Align 'Configuring Runtime Watcher' setup guide with implementation * de-duplicate watcher-setup-guide and api docs * Add TW feedback Co-authored-by: Natalia Sitko <80401180+nataliasitko@users.noreply.github.com> --------- Co-authored-by: Natalia Sitko <80401180+nataliasitko@users.noreply.github.com>
1 parent de2f534 commit 636a630

3 files changed

Lines changed: 82 additions & 221 deletions

File tree

docs/api.md

Lines changed: 38 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -1,83 +1,62 @@
11
# Watcher CR
22

3-
The [Watcher CR](https://github.com/kyma-project/lifecycle-manager/blob/main/api/v1beta2/watcher_types.go#L121) configures the Kyma Control Plane (KCP) setup and Runtime Watcher on Kyma runtimes. Read the following document to see the the parameters the CR consists of.
4-
5-
## Runtime-Watcher configuration
6-
7-
### **spec.resourceToWatch**
8-
9-
Defines which resources the Runtime Watcher watches. By default, the **version** value is a wildcard (`*`), so an update is not required after the API version changes.
3+
The [Watcher CR](https://github.com/kyma-project/lifecycle-manager/blob/main/api/v1beta2/watcher_types.go) configures the Kyma Control Plane (KCP) setup and Runtime Watcher on Kyma runtimes. Read the following document to see the parameters that the CR consists of.
104

115
```yaml
6+
apiVersion: operator.kyma-project.io/v1beta2
7+
kind: Watcher
8+
metadata:
9+
name: <name>
10+
namespace: kcp-system
1211
spec:
1312
resourceToWatch:
14-
group: operator.kyma-project.io
15-
version: "*"
16-
resource: kymas
17-
```
18-
19-
### **spec.labelsToWatch**
20-
21-
> **NOTE:** The resources you're watching must have the `operator.kyma-project.io/watched-by` label.
22-
23-
With the **spec.labelsToWatch** attribute, you can filter the specified Group/Version/Kind (GVK) in **spec.resourceToWatch**. For example, if the specified GVK is `secrets`, then it is useful to filter them by a specific label, otherwise, the Runtime Watcher would send an event to the KCP for every Create/Read/Update/Delete (CRUD) event of any Secret in the Kyma cluster.
24-
25-
```yaml
26-
spec:
13+
group: <api-group>
14+
version: <version>
15+
resource: <kind>
2716
labelsToWatch:
28-
"operator.kyma-project.io/watched-by": "kyma"
29-
"example.label.to.watch": "true"
30-
resourceToWatch:
31-
group: ""
32-
version: "v1"
33-
resource: secrets
17+
"<some>": "<label>"
18+
field: <"spec" or "status">
19+
manager: <manager-name>
20+
serviceInfo:
21+
name: <service-name>
22+
port: <port>
23+
namespace: <namespace>
24+
gateway: # don't change
25+
selector:
26+
matchLabels:
27+
"operator.kyma-project.io/watcher-gateway": "default"
3428
```
3529
30+
### Resources to Watch
3631
37-
### **spec.field**
32+
The **spec.resourceToWatch** field specifies the GVK of the resources Runtime Watcher watches. Note that **spec.resourceToWatch.resource** must be the API resource name, not the kind of the resource. For example, it must be "configmaps" instead of "ConfigMap". It is possible to specify the wildcard `*` for **spec.resourceToWatch.version**.
3833

39-
It uses either the `spec` or `status` value to define if Runtime Watcher sends an event to KCP if the spec of the specified GVK or the status changes.
34+
### Labels to Watch
4035

41-
### **spec.manager**
36+
Optionally, the **spec.labelsToWatch** field allows to filter the resources by a specific label value.
4237

43-
Specifies the component responsible for handling webhook requests from Runtime Watcher. This field determines the routing path for webhook validation requests to KCP (`/validate/<manager-value>`).
38+
> [!NOTE]
39+
> Runtime Watcher does not provide a mechanism to add this label to the resources. You must ensure that the resources you want to watch have this label.
4440

45-
```yaml
46-
spec:
47-
manager: "lifecycle-manager"
48-
```
41+
### Field
4942

50-
> ### Note:
51-
> **Backward Compatibility:** If you don't set **spec.manager**, the system falls back to the `operator.kyma-project.io/managed-by` label for backward compatibility. However, this label-based approach is deprecated and will be removed in a future version. It is recommended to migrate to using the **spec.manager** field.
43+
The **spec.field** field specifies what parts of the watched object trigger events. Allowed values are `spec` and `status`.
5244

53-
## KCP configuration
45+
If `status` is specified, watch events are only emitted if the `.status` subresource of the watched object changes. The ValidatingWebhookConfiguration is configured to watch only the status subresource. For example, "pods/status" instead of "pods".
5446

55-
### **spec.serviceInfo**
47+
If `spec` is specified, watch events are only emitted if the `.spec` field of the watched object changes. If the object doesn't contain a `.spec` field, it falls back to emit a watch event on **any** change to the object, including changes to metadata or status.
5648

57-
Specifies to which name, Namespace, and port the incoming events are routed.
49+
### Manager
5850

59-
```yaml
60-
spec:
61-
name: klm-event-service
62-
namespace: kcp-system
63-
port: 8082
64-
```
51+
The **spec.manager** field defines the URL path to which the Runtime Watcher sends events. The entire path follows the format `/v2/<spec.manager>/event`. Accordingly, a VirtualService is created that matches the prefix `/v2/<spec.manager>/event` and routes received requests to the Service defined in **spec.serviceInfo**.
6552

66-
### **spec.gateway**
53+
> [!NOTE]
54+
> In Kyma runtime, this setting configures the ValidatingWebhookConfiguration to call `/validate/<spec.manager>` of the Runtime Watcher deployment.
6755

68-
Defines to which label selector of a Gateway the VirtualService binds.
56+
### Service Info
6957

70-
```yaml
71-
spec:
72-
gateway:
73-
selector:
74-
matchLabels:
75-
"operator.kyma-project.io/watcher-gateway": "default"
76-
```
58+
The **spec.serviceInfo** specifies the name, namespace, and port to which events received from the Runtime Watcher are routed.
7759

78-
## Labels
60+
### Gateway
7961

80-
- **`operator.kyma-project.io/managed-by`:** **[DEPRECATED]** This label previously specified the module that manages and listens to the Watcher CR's corresponding webhook. The value was used to generate the path in KCP for webhook requests (`/validate/<label-value>`).
81-
82-
> ### Caution
83-
> The `operator.kyma-project.io/managed-by` label is deprecated. Use the **spec.manager** field instead. The label-based fallback will be removed in a future API version. For more information, see the [**spec.manager**](#specmanager) section.
62+
The **spec.gateway** field defines the label selector of the Istio Gateway in KCP. Don't change the default value.

docs/watcher-setup-guide.md

Lines changed: 34 additions & 162 deletions
Original file line numberDiff line numberDiff line change
@@ -8,196 +8,68 @@ The Watcher mechanism is deployed to the SKR as `ValidatingWebhookConfiguration`
88

99
To set up a watch on a resource, you must define and apply a Watcher CR for it. The Watcher CR defines which resources Runtime Watcher notifies changes for and where to forward the events in KCP.
1010

11-
Here is the definition of the Watcher CR. The detailed filed descriptions are provided below in the [Resources to Watch](#resources-to-watch) and [Events to Consume](#events-to-consume) sections.
11+
Here is an example of the Watcher CR. The detailed field descriptions are provided in the [Watcher API definition](./api.md).
1212

1313
```yaml
1414
apiVersion: operator.kyma-project.io/v1beta2
1515
kind: Watcher
1616
metadata:
1717
name: <name>
1818
namespace: kcp-system
19-
labels:
20-
"operator.kyma-project.io/managed-by": "<operator-name>"
2119
spec:
2220
resourceToWatch:
2321
group: <api-group>
24-
version: <version> # wildcard "*" is allowed
22+
version: <version>
2523
resource: <kind>
2624
labelsToWatch:
27-
"operator.kyma-project.io/watched-by": "<label>" # needs to be on the resource to watch
28-
field: "" # possible values: "spec", "status"
25+
"<some>": "<label>"
26+
field: <"spec" or "status">
27+
manager: <manager-name>
2928
serviceInfo:
3029
name: <service-name>
3130
port: <port>
3231
namespace: <namespace>
33-
gateway:
32+
gateway: # don't change
3433
selector:
3534
matchLabels:
36-
"operator.kyma-project.io/watcher-gateway": "default" # don't change
35+
"operator.kyma-project.io/watcher-gateway": "default"
3736
```
3837
39-
For more information, see the [Watcher API definition](./api.md).
38+
## Consuming Events
4039
41-
### Resources to Watch
42-
43-
The **spec.resourceToWatch** field specifies the GVK of the resources Runtime Watcher watches.
44-
45-
These resources must have the `operator.kyma-project.io/watched-by` label. The **spec.labelsToWatch** field allows you to filter the resources by a specific label value.
46-
47-
> [!NOTE]
48-
> Runtime Watcher does not provide a mechanism to add this label to the resources. You must ensure that the resources you want to watch have this label.
49-
50-
Using the **spec.field** field, you can choose between values `spec` or `status`, to set either the `status` subresource as a notification trigger or the `spec` field.
51-
52-
See an example of a Watcher CR that watches changes on Secrets' spec:
53-
54-
```yaml
55-
spec:
56-
resourceToWatch:
57-
group: ""
58-
version: v1
59-
resource: secrets
60-
labelsToWatch:
61-
"operator.kyma-project.io/watched-by": "my-operator"
62-
field: "spec"
63-
```
64-
65-
### Events to Consume
66-
67-
The **spec.serviceInfo** field specifies the name, namespace, and port to which the events are routed.
68-
69-
The **spec.gateway** field defines the label selector of the Istio Gateway in KCP. Don't change the default value.
70-
71-
See an example of a Watcher CR that forwards events to the `my-operator-service` service in the `my-system` namespace on port `8080`:
72-
73-
```yaml
74-
spec:
75-
serviceInfo:
76-
name: my-operator-service
77-
namespace: my-system
78-
port: 8080
79-
```
80-
81-
The service receiving the events can be any arbitrary service that is listening on the specified port, or it can be a k8s controller using the [Listener package](./listener.md).
82-
83-
This is the request body of the event that is sent to the service:
40+
The service receiving the events can be any arbitrary service that is listening on the specified port. Behind the service, there must be a consumer expecting POST requests on `/v2/<spec.manager>/event` with the following content:
8441

8542
```json
8643
{
87-
"owner": {
88-
"name": "my-kyma",
89-
"namespace": "kcp-system"
90-
},
91-
"watched": {
92-
"name": "my-secret",
93-
"namespace": "kyma-system"
94-
},
95-
"watchedGvk": {
96-
"group": "",
97-
"version": "v1",
98-
"kind": "secrets"
99-
}
44+
"watched": { "Namespace": "<watched object's namespace>", "Name": "<watched object's name>" },
45+
"watchedGvk": { "group": "<watched object's group>", "version": "<watched object's version>", "kind": "<watched object's kind>" }
10046
}
10147
```
10248

103-
The **owner** field contains the namespaced name of the resource that owns the watched resource. It is the reference to the resource in KCP that should be reconciled when the event is received. It is parsed from the `operator.kyma-project.io/owned-by` label on the watched resource in the `<namespace>/<name>` format.
104-
105-
## Examples
106-
107-
### Arbitrary Service
108-
109-
It is possible to consume events in any arbitrary service that listens on the specified port. It receives the Events as POST requests on the route `/v1/<operator-name>/event`. This <operator-name> is the value of the `operator.kyma-project.io/managed-by` label in the Watcher CR.
110-
111-
See a Golang example of a server that listens on port `8080` and prints the received events:
112-
113-
```go
114-
package main
115-
import (
116-
"fmt"
117-
"io"
118-
"log"
119-
"net/http"
120-
)
121-
func main() {
122-
http.HandleFunc("/v1/my-operator/event", func(w http.ResponseWriter, r *http.Request) {
123-
body, err := io.ReadAll(r.Body)
124-
if err != nil {
125-
http.Error(w, "can't read body", http.StatusBadRequest)
126-
return
127-
}
128-
fmt.Println(string(body))
129-
})
130-
log.Fatal(http.ListenAndServe(":8080", nil))
49+
To identify the Kyma runtime from which the received event originates, the Runtime Id can be extracted from the Common Name of the certificate attached to the request. The certificate attached to the request is available as an HTTP header, and the `listener` package provides the [`GetCertificateFromHeader()`](https://github.com/kyma-project/runtime-watcher/blob/de2f534ce7c0c73da817505c9aad0db12f966b27/listener/pkg/v2/certificate/parse_certificate.go#L26-L65) helper function to extract it. It can be used as follows:
50+
51+
```Go
52+
func getRuntimeIdFromRequest(req *http.Request) (string, *UnmarshalError) {
53+
clientCertificate, err := certificate.GetCertificateFromHeader(req)
54+
if err != nil {
55+
return "", &UnmarshalError{
56+
fmt.Sprintf("could not get client certificate from request: %v", err),
57+
http.StatusUnauthorized,
58+
}
59+
}
60+
61+
if clientCertificate.Subject.CommonName == "" {
62+
return "", &UnmarshalError{
63+
"client certificate common name is empty",
64+
http.StatusBadRequest,
65+
}
66+
}
67+
68+
return clientCertificate.Subject.CommonName, nil
13169
}
13270
```
13371

134-
### Listener Package
135-
136-
The listener package simplifies setting up an endpoint for an operator residing in KCP, which receives events sent by the Watcher to KCP. It provides a simple API to register a handler for the event type and start the server in a controller.
137-
138-
See an example of setting up a listener in a controller and reconciling the owner resource:
139-
140-
```go
141-
import (
142-
"context"
143-
"net/http"
144-
ctrl "sigs.k8s.io/controller-runtime"
145-
"sigs.k8s.io/controller-runtime/pkg/client"
146-
"sigs.k8s.io/controller-runtime/pkg/source"
147-
"sigs.k8s.io/controller-runtime/pkg/event"
148-
"sigs.k8s.io/controller-runtime/pkg/handler"
149-
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
150-
"k8s.io/client-go/util/workqueue"
151-
watcherevent "github.com/kyma-project/runtime-watcher/listener/pkg/event"
152-
"github.com/kyma-project/runtime-watcher/listener/pkg/types"
153-
)
154-
func (r *Reconciler) SetupWithManager(mgr ctrl.Manager, ...) error {
155-
verifyFunc = func(r *http.Request, watcherEvtObject *types.WatchEvent) error {
156-
return nil // If needed, implement your verification logic here
157-
}
158-
runnableListener := watcherevent.NewSKREventListener(
159-
":8080", // The port on which the listener listens
160-
"operator-name", // The value of the `operator.kyma-project.io/managed-by` label in the Watcher CR
161-
verifyFunc,
162-
)
163-
if err := mgr.Add(runnableListener); err != nil {
164-
// Handle error
165-
}
166-
...
167-
if err := ctrl.NewControllerManagedBy(mgr).For(...).
168-
... // Add your controller setup here
169-
WatchesRawSource(source.Channel(runnableListener.ReceivedEvents, r.skrEventHandler())).
170-
Complete(r); err != nil {
171-
// Handle error
172-
}
173-
}
174-
175-
func (r *Reconciler) skrEventHandler() *handler.Funcs {
176-
return &handler.Funcs{
177-
GenericFunc: func(ctx context.Context, evnt event.GenericEvent,
178-
queue workqueue.TypedRateLimitingInterface[ctrl.Request],
179-
) {
180-
unstructWatcherEvt, conversionOk := evnt.Object.(*unstructured.Unstructured)
181-
if !conversionOk {
182-
// Handle error
183-
}
184-
185-
unstructuredOwner, ok := unstructWatcherEvt.Object["owner"]
186-
if !ok {
187-
// Handle error
188-
}
189-
190-
ownerObjectKey, conversionOk := unstructuredOwner.(client.ObjectKey)
191-
if !conversionOk {
192-
// Handle error
193-
}
194-
195-
queue.Add(ctrl.Request{
196-
NamespacedName: ownerObjectKey,
197-
})
198-
},
199-
}
200-
}
201-
```
72+
For further convenience, the `listener` package also provides a [`SKREventListener`](../listener/pkg/v2/event/skr_events_listener.go#L32-L43) that handles the requests and exposes a channel via [`ReceivedEvents()`](../listener/pkg/v2/event/skr_events_listener.go#L46-L54) providing an unstructured object for every received event. For an example in Lifecycle Manager, see:
20273

203-
The package provides a `SKREventListener` struct that implements the `Runnable` interface. The `SKREventListener` struct listens on the specified port and verifies the incoming events. The `ReceivedEvents` channel is used to receive the events. The `SKREventListener` struct is added to the manager, and the controller watches the `ReceivedEvents` channel to reconcile the owner resource through WatcherRawSource.
74+
- https://github.com/kyma-project/lifecycle-manager/blob/d76d77a2c636b26084a0233b876c41189c556d77/internal/controller/kyma/setup.go#L30-L37
75+
- https://github.com/kyma-project/lifecycle-manager/blob/d76d77a2c636b26084a0233b876c41189c556d77/internal/controller/kyma/setup.go#L50-L51

listener/pkg/v2/event/skr_events_listener.go

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,10 @@ type SKREventListener struct {
2929
events chan types.GenericEvent
3030
}
3131

32+
// NewSKREventListener creates a new instance of SKREventListener.
33+
// addr specifies the TCP address for the server to listen on in the form "host:port".
34+
// componentName is used to construct the API path for receiving events.
35+
// The API path will be in the format "/v2/{componentName}/event".
3236
func NewSKREventListener(addr, componentName string,
3337
) *SKREventListener {
3438
unbufferedEventsChan := make(chan types.GenericEvent)
@@ -39,6 +43,12 @@ func NewSKREventListener(addr, componentName string,
3943
}
4044
}
4145

46+
// ReceivedEvents returns a read-only channel that emits GenericEvent objects received by the listener.
47+
// GenericEvent is a wrapper around a Object of type *unstructured.Unstructured.
48+
// The data contains:
49+
// - watched: NamespacedName
50+
// - watched-gvk: GroupVersionKind
51+
// - runtime-id: string
4252
func (l *SKREventListener) ReceivedEvents() <-chan types.GenericEvent {
4353
return l.events
4454
}

0 commit comments

Comments
 (0)