Skip to content

Commit f50b74a

Browse files
docs(eventing): refresh page content and add event source hierarchy (#3369)
Signed-off-by: Dennis-Mircea Ciupitu <dennis.mircea.ciupitu@gmail.com>
1 parent 50bd70b commit f50b74a

1 file changed

Lines changed: 104 additions & 59 deletions

File tree

docs/content/en/docs/documentation/eventing.md

Lines changed: 104 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -9,29 +9,30 @@ See also
99
this [blog post](https://csviri.medium.com/java-operator-sdk-introduction-to-event-sources-a1aab5af4b7b)
1010
.
1111

12-
Event sources are a relatively simple yet powerful and extensible concept to trigger controller
13-
executions, usually based on changes to managed resources. You typically need an event source
14-
when you want your `Reconciler` to be triggered when something occurs to secondary resources
15-
that might affect the state of your primary resource. This is needed because a given
16-
`Reconciler` will only listen by default to events affecting the primary resource type it is
17-
configured for. Event sources act as listen to events affecting these secondary resources so
18-
that a reconciliation of the associated primary resource can be triggered when needed. Note that
19-
these secondary resources need not be Kubernetes resources. Typically, when dealing with
20-
non-Kubernetes objects or services, we can extend our operator to handle webhooks or websockets
21-
or to react to any event coming from a service we interact with. This allows for very efficient
22-
controller implementations because reconciliations are then only triggered when something occurs
23-
on resources affecting our primary resources thus doing away with the need to periodically
24-
reschedule reconciliations.
12+
Event sources are a simple yet powerful and extensible mechanism for triggering controller
13+
executions, usually in response to changes to managed resources. You need an event source
14+
whenever your `Reconciler` must react to something happening on a *secondary* resource that
15+
affects your primary resource's state. By default, a `Reconciler` only listens to events on the
16+
primary resource type it is configured for.
17+
18+
An event source listens for events on these secondary resources and, when one fires, triggers a
19+
reconciliation of the associated primary resource. Secondary resources do not have to be
20+
Kubernetes resources: when dealing with non-Kubernetes objects or external services, an event
21+
source can wrap webhooks, websockets, or any other notification mechanism the service provides.
22+
23+
This event-driven model makes for efficient controllers, since reconciliations only run when
24+
something actually changes on a resource that matters, removing the need to periodically poll or
25+
reschedule.
2526

2627
```mermaid
2728
graph LR
29+
CES["Controller Event Source"]:::primary -- Event --> EP["Event Processor"]:::handler
30+
TES["Timer Event Source"]:::primary -- Event --> EP
2831
SR1ES["Secondary Resource 1
29-
Event Source"]:::eventsource -- Event --> EH["Event Handler"]:::handler
30-
CRES["Custom Resource
31-
Event Source"]:::primary -- Event --> EH
32+
Event Source"]:::eventsource -- Event --> EP
3233
SR2ES["Secondary Resource 2
33-
Event Source"]:::eventsource -- Event --> EH
34-
EH --> C["Controller"]:::controller
34+
Event Source"]:::eventsource -- Event --> EP
35+
EP --> C["Controller"]:::controller
3536
C --> SR1["Secondary Resource 1"]:::secondary
3637
C --> SR2["Secondary Resource 2"]:::secondary
3738
@@ -42,15 +43,25 @@ graph LR
4243
classDef secondary fill:#3AAFA9,stroke:#2B807B,color:#fff
4344
```
4445

45-
There are few interesting points here:
46-
47-
The `CustomResourceEventSource` event source is a special one, responsible for handling events
48-
pertaining to changes affecting our primary resources. This `EventSource` is always registered
49-
for every controller automatically by the SDK. It is important to note that events always relate
50-
to a given primary resource. Concurrency is still handled for you, even in the presence of
51-
`EventSource` implementations, and the SDK still guarantees that there is no concurrent execution of
52-
the controller for any given primary resource (though, of course, concurrent/parallel executions
53-
of events pertaining to other primary resources still occur as expected).
46+
A few things worth highlighting about the diagram above.
47+
- The `ControllerEventSource` is a special, internal event source responsible for handling events
48+
pertaining to changes affecting the primary resource. The SDK registers it automatically for every
49+
controller and you never instantiate it yourself.
50+
- Every controller also gets a dedicated `TimerEventSource` (named
51+
`RetryAndRescheduleTimerEventSource`) that the SDK uses to drive retry attempts after a failed
52+
reconciliation, `UpdateControl.rescheduleAfter(...)` requests, and the periodic max-interval
53+
failsafe trigger. The `EventProcessor` is the sole caller into this timer, scheduling delayed
54+
events back to itself via `scheduleOnce(...)`. Like the controller event source, this one is
55+
wired internally and is not something you register or interact with directly.
56+
- Once an event reaches the `EventProcessor`, dispatch is delegated to the
57+
`ReconciliationDispatcher`, which prepares the execution context, handles finalizers and other
58+
framework concerns, and ultimately invokes `reconcile(...)` on the internal `Controller` wrapper,
59+
which in turn calls the user-implemented `Reconciler`.
60+
61+
Events always relate to a given primary resource, and the SDK guarantees that there is no
62+
concurrent execution of the reconciler for any given primary resource, even in the presence of
63+
additional `EventSource` implementations. Events pertaining to other primary resources are still
64+
processed in parallel as expected.
5465

5566
### Caching and Event Sources
5667

@@ -69,34 +80,30 @@ is processed for the `InformerEventSource` implementation. However, this does no
6980
hold true for all event source implementations (`PerResourceEventSource` for example). The SDK
7081
provides methods to handle this situation elegantly, allowing you to check if an object is
7182
cached, retrieving it from a provided supplier if not. See
72-
related [method](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java#L146)
83+
related [method](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java#L160-L180)
7384
.
7485

7586
### Registering Event Sources
7687

77-
To register event sources, your `Reconciler` has to override the `prepareEventSources` and return
78-
list of event sources to register. One way to see this in action is
79-
to look at the
88+
To register event sources, your `Reconciler` overrides the `prepareEventSources` default method
89+
and returns the list of event sources to register. One way to see this in action is to look at the
8090
[WebPage example](https://github.com/operator-framework/java-operator-sdk/blob/main/sample-operators/webpage/src/main/java/io/javaoperatorsdk/operator/sample/WebPageReconciler.java)
8191
(irrelevant details omitted):
8292

8393
```java
84-
85-
import java.util.List;
86-
8794
@ControllerConfiguration
88-
public class WebappReconciler
89-
implements Reconciler<Webapp>, Cleaner<Webapp>, EventSourceInitializer<Webapp> {
90-
// ommitted code
95+
public class WebPageReconciler implements Reconciler<WebPage> {
9196

9297
@Override
93-
public List<EventSource<?, Webapp>> prepareEventSources(EventSourceContext<Webapp> context) {
94-
InformerEventSourceConfiguration<Webapp> configuration =
95-
InformerEventSourceConfiguration.from(Deployment.class, Webapp.class)
98+
public List<EventSource<?, WebPage>> prepareEventSources(EventSourceContext<WebPage> context) {
99+
var configuration =
100+
InformerEventSourceConfiguration.from(Deployment.class, WebPage.class)
96101
.withLabelSelector(SELECTOR)
97102
.build();
98103
return List.of(new InformerEventSource<>(configuration, context));
99104
}
105+
106+
// omitted code
100107
}
101108
```
102109

@@ -126,7 +133,7 @@ resources live outside of a cluster.
126133
This is why JOSDK provides the `SecondaryToPrimaryMapper` interface so that you can provide
127134
alternative ways for the SDK to identify which primary resource needs to be reconciled when
128135
something occurs to your secondary resources. We even provide some of these alternatives in the
129-
[Mappers](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/Mappers.java)
136+
[Mappers](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/Mappers.java)
130137
class.
131138

132139
Note that, while a set of `ResourceID` is returned, this set usually consists only of one
@@ -150,11 +157,36 @@ integration test for a sample.
150157

151158
### Built-in EventSources
152159

153-
There are multiple event-sources provided out of the box, the following are some more central ones:
160+
There are multiple event-sources provided out of the box, the following are some more central ones.
161+
162+
All of them implement the `EventSource` interface. JOSDK provides an abstract base,
163+
`AbstractEventSource`, together with two abstract intermediate classes that capture the two most
164+
common patterns: caching of externally-fetched resources (`ExternalResourceCachingEventSource`)
165+
and informer-based watching of Kubernetes resources (`ManagedInformerEventSource`). Concrete
166+
event sources extend the most appropriate base for their use case.
167+
168+
```mermaid
169+
graph TD
170+
ES["EventSource<br/>«interface»"]:::iface --> AES["AbstractEventSource<br/>«abstract»"]:::abs
171+
AES --> TES["TimerEventSource"]:::builtin
172+
AES --> SIES["SimpleInboundEventSource"]:::concrete
173+
AES --> ERCES["ExternalResourceCachingEventSource<br/>«abstract»"]:::abs
174+
AES --> MIES["ManagedInformerEventSource<br/>«abstract»"]:::abs
175+
ERCES --> PES["PollingEventSource"]:::concrete
176+
ERCES --> PRPES["PerResourcePollingEventSource"]:::concrete
177+
ERCES --> CIES["CachingInboundEventSource"]:::concrete
178+
MIES --> IES["InformerEventSource"]:::concrete
179+
MIES --> CES["ControllerEventSource"]:::builtin
180+
181+
classDef iface fill:#3FAA5F,stroke:#2A8045,color:#fff
182+
classDef abs fill:#326CE5,stroke:#1A4AAF,color:#fff
183+
classDef concrete fill:#3AAFA9,stroke:#2B807B,color:#fff
184+
classDef builtin fill:#C0527A,stroke:#8C3057,color:#fff
185+
```
154186

155187
#### `InformerEventSource`
156188

157-
[InformerEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java)
189+
[InformerEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java)
158190
is probably the most important `EventSource` implementation to know about. When you create an
159191
`InformerEventSource`, JOSDK will automatically create and register a `SharedIndexInformer`, a
160192
fabric8 Kubernetes client class, that will listen for events associated with the resource type
@@ -170,39 +202,52 @@ reconciliations from being triggered when not needed and allowing efficient oper
170202

171203
#### `PerResourcePollingEventSource`
172204

173-
[PerResourcePollingEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java)
205+
[PerResourcePollingEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PerResourcePollingEventSource.java)
174206
is used to poll external APIs, which don't support webhooks or other event notifications. It
175207
extends the abstract
176-
[ExternalResourceCachingEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ExternalResourceCachingEventSource.java)
208+
[ExternalResourceCachingEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/ExternalResourceCachingEventSource.java)
177209
to support caching.
178-
See [MySQL Schema sample](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/MySQLSchemaReconciler.java)
210+
See [MySQL Schema sample](https://github.com/operator-framework/java-operator-sdk/blob/main/sample-operators/mysql-schema/src/main/java/io/javaoperatorsdk/operator/sample/MySQLSchemaReconciler.java)
179211
for usage.
180212

181213
#### `PollingEventSource`
182214

183-
[PollingEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PollingEventSource.java)
184-
is similar to `PerResourceCachingEventSource` except that, contrary to that event source, it
215+
[PollingEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/polling/PollingEventSource.java)
216+
is similar to `PerResourcePollingEventSource` except that, contrary to that event source, it
185217
doesn't poll a specific API separately per resource, but periodically and independently of
186218
actually observed primary resources.
187219

188220
#### Inbound event sources
189221

190-
[SimpleInboundEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/inbound/SimpleInboundEventSource.java)
222+
[SimpleInboundEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/inbound/SimpleInboundEventSource.java)
191223
and
192-
[CachingInboundEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/inbound/CachingInboundEventSource.java)
224+
[CachingInboundEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/inbound/CachingInboundEventSource.java)
193225
are used to handle incoming events from webhooks and messaging systems.
194226

195-
#### `ControllerResourceEventSource`
227+
#### `ControllerEventSource`
196228

197-
[ControllerResourceEventSource](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerResourceEventSource.java)
229+
[ControllerEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/controller/ControllerEventSource.java)
198230
is a special `EventSource` implementation that you will never have to deal with directly. It is,
199-
however, at the core of the SDK is automatically added for you: this is the main event source
231+
however, at the core of the SDK and is automatically added for you: this is the main event source
200232
that listens for changes to your primary resources and triggers your `Reconciler` when needed.
201233
It features smart caching and is really optimized to minimize Kubernetes API accesses and avoid
202-
triggering unduly your `Reconciler`.
234+
triggering your `Reconciler` unnecessarily.
235+
236+
#### `TimerEventSource`
237+
238+
[TimerEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/timer/TimerEventSource.java)
239+
is an internal `EventSource` that the SDK automatically registers for every controller (under
240+
the name `RetryAndRescheduleTimerEventSource`). It is used internally to schedule delayed events
241+
back to the `EventProcessor`, namely retry attempts after a failed reconciliation, explicit
242+
rescheduling requests via `UpdateControl.rescheduleAfter(...)`, and the periodic failsafe trigger
243+
governed by `maxReconciliationInterval`. As with `ControllerEventSource`, this is not something
244+
you instantiate, configure, or interact with directly. If you need periodic or delayed
245+
reconciliation in your `Reconciler`, use `UpdateControl.rescheduleAfter(...)` from inside
246+
`reconcile(...)` or configure `maxReconciliationInterval` on your controller; do not depend on
247+
`TimerEventSource` directly.
203248

204249
More on the philosophy of the non Kubernetes API related event source see in
205-
issue [#729](https://github.com/java-operator-sdk/java-operator-sdk/issues/729).
250+
issue [#729](https://github.com/operator-framework/java-operator-sdk/issues/729).
206251

207252

208253
## InformerEventSource Multi-Cluster Support
@@ -244,7 +289,7 @@ To turn off this feature, set `generationAwareEventProcessing` to `false` for th
244289

245290
When informers / event sources are properly set up, and the `Reconciler` implementation is
246291
correct, no additional reconciliation triggers should be needed. However, it's
247-
a [common practice](https://github.com/java-operator-sdk/java-operator-sdk/issues/848#issuecomment-1016419966)
292+
a [common practice](https://github.com/operator-framework/java-operator-sdk/issues/848#issuecomment-1016419966)
248293
to have a failsafe periodic trigger in place, just to make sure resources are nevertheless
249294
reconciled after a certain amount of time. This functionality is in place by default, with a
250295
rather high time interval (currently 10 hours) after which a reconciliation will be
@@ -262,7 +307,7 @@ The event is not propagated at a fixed rate, rather it's scheduled after each re
262307
next reconciliation will occur at most within the specified interval after the last reconciliation.
263308

264309
This feature can be turned off by setting `maxReconciliationInterval`
265-
to [`Constants.NO_MAX_RECONCILIATION_INTERVAL`](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Constants.java#L20-L20)
310+
to [`Constants.NO_MAX_RECONCILIATION_INTERVAL`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/Constants.java#L20-L20)
266311
or any non-positive number.
267312

268313
The automatic retries are not affected by this feature so a reconciliation will be re-triggered
@@ -292,8 +337,8 @@ evict cold resources than try to limit cache sizes.
292337

293338
See usage of the related implementation using [Caffeine](https://github.com/ben-manes/caffeine) cache in integration
294339
tests
295-
for [primary resources](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/caffeine-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/sample/AbstractTestReconciler.java).
340+
for [primary resources](https://github.com/operator-framework/java-operator-sdk/blob/main/caffeine-bounded-cache-support/src/test/java/io/javaoperatorsdk/operator/processing/event/source/cache/sample/AbstractTestReconciler.java).
296341

297342
See
298-
also [CaffeineBoundedItemStores](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java)
343+
also [CaffeineBoundedItemStores](https://github.com/operator-framework/java-operator-sdk/blob/main/caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java)
299344
for more details.

0 commit comments

Comments
 (0)