Skip to content

Commit 31a590c

Browse files
committed
improved
Signed-off-by: Attila Mészáros <a_meszaros@apple.com>
1 parent fefb3d7 commit 31a590c

File tree

1 file changed

+70
-73
lines changed

1 file changed

+70
-73
lines changed

docs/content/en/docs/documentation/access-resources.md

Lines changed: 70 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -3,69 +3,67 @@ title: Accessing resources in caches
33
weight: 48
44
---
55

6-
As described in [Event sources and related topics](eventing.md) event sources are the backbone
7-
for caching resources and triggering the reconciliation for primary resources that are related
6+
As described in [Event sources and related topics](eventing.md), event sources serve as the backbone
7+
for caching resources and triggering reconciliation for primary resources that are related
88
to cached resources.
99

10-
In Kubernetes world, the component that does this is called Informer. Without going into
11-
the details (there are plenty of good documents online regarding informers), its responsibility
12-
is to watch resources, cache them, and emit an event if the resource changed.
10+
In the Kubernetes ecosystem, the component responsible for this is called an Informer. Without delving into
11+
the details (there are plenty of excellent resources online about informers), its responsibility
12+
is to watch resources, cache them, and emit events when resources change.
1313

14-
EventSource is a generalized concept of Informer to non-Kubernetes resources. Thus,
15-
to cache external resources, and trigger reconciliation if those change.
14+
EventSource is a generalized concept that extends the Informer pattern to non-Kubernetes resources,
15+
allowing you to cache external resources and trigger reconciliation when those resources change.
1616

1717
## The InformerEventSource
1818

19-
The underlying informer implementation comes from the fabric8 client, called [DefaultSharedIndexInformer](https://github.com/fabric8io/kubernetes-client/blob/main/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/informers/impl/DefaultSharedIndexInformer.java).
20-
[InformerEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java)
21-
in Java Operator SDK wraps informers from fabric8 client.
22-
The purpose of such wrapping is to add additional capabilities required for controllers.
23-
(In general, Informers are not used only for implementing controllers).
19+
The underlying informer implementation comes from the fabric8 client, specifically the [DefaultSharedIndexInformer](https://github.com/fabric8io/kubernetes-client/blob/main/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/informers/impl/DefaultSharedIndexInformer.java).
20+
[InformerEventSource](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/InformerEventSource.java)
21+
in Java Operator SDK wraps the fabric8 client informers.
22+
This wrapper adds additional capabilities specifically required for controllers
23+
(note that informers have broader applications beyond just implementing controllers).
2424

25-
Such capabilities are:
26-
- maintaining an index to which primary resources the secondary resources in informer cache are related to.
27-
- setting up multiple informers for the same type if needed. You need informer per namespace if the informer
28-
is not watching the whole cluster.
29-
- Dynamically adding/removing watched namespaces.
30-
- Some others, which are out of the scope of this document.
25+
These additional capabilities include:
26+
- Maintaining an index that maps secondary resources in the informer cache to their related primary resources
27+
- Setting up multiple informers for the same resource type when needed (for example, you need one informer per namespace if the informer is not watching the entire cluster)
28+
- Dynamically adding and removing watched namespaces
29+
- Other capabilities that are beyond the scope of this document
3130

3231
### Associating Secondary Resources to Primary Resource
3332

34-
The question is, how to trigger reconciliation of a primary resource (your custom resource),
35-
when Informer receives a new resource.
36-
For this purpose the framework uses [`SecondaryToPrimaryMapper`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java)
37-
that tells (usually) based on the resource which primary resource reconciliation to trigger.
38-
The mapping is usually done based on the owner reference or annotation on the secondary resource.
39-
(But not always, as we will see)
33+
The key question is: how do you trigger reconciliation of a primary resource (your custom resource)
34+
when an Informer receives a new resource?
35+
The framework uses [`SecondaryToPrimaryMapper`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/SecondaryToPrimaryMapper.java)
36+
to determine which primary resource reconciliation to trigger based on the received resource.
37+
This mapping is typically done using owner references or annotations on the secondary resource,
38+
though other mapping strategies are possible (as we'll see later).
4039

41-
It is important to realize that if a resource triggers the reconciliation of a primary resource, that
42-
resource naturally will be used during reconciliation. So the reconciler will need to access it.
43-
Therefore, InformerEventSource maintains a reverse index [PrimaryToSecondaryIndex](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/DefaultPrimaryToSecondaryIndex.java),
44-
based on the result of the `SecondaryToPrimaryMapper`.
40+
It's important to understand that when a resource triggers reconciliation of a primary resource,
41+
that resource will naturally be needed during the reconciliation process, so the reconciler must be able to access it.
42+
Therefore, InformerEventSource maintains a reverse index called [PrimaryToSecondaryIndex](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/DefaultPrimaryToSecondaryIndex.java),
43+
which is built based on the results of the `SecondaryToPrimaryMapper`.
4544

4645
## Unified API for Related Resources
4746

48-
To access all related resources for a primary resource, the framework provides an API to access the related
49-
secondary resources using:
47+
To access all related resources for a primary resource, the framework provides a unified API:
5048

5149
```java
5250
Context.getSecondaryResources(Class<R> expectedType);
5351
```
5452

55-
That will list all the related resources of a certain type, based on the `InformerEventSource`'s `PrimaryToSecondaryIndex`.
56-
Based on that index, it reads the resources from the Informers cache. Note that since all those steps work
57-
on top of indexes, those operations are very fast, usually O(1).
53+
This method lists all related resources of a specific type, based on the `InformerEventSource`'s `PrimaryToSecondaryIndex`.
54+
It reads the resources from the Informer's cache using this index. Since these operations work
55+
directly with indexes, they are very fast—typically O(1) performance.
5856

59-
We mostly talk about InformerEventSource, but this works in similar ways for the generalized EventSources concept, since
60-
the [`EventSource`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/EventSource.java#L93)
61-
actually implements the `Set<R> getSecondaryResources(P primary);` method. That is just called from the context.
57+
While we've focused on InformerEventSource, this pattern works similarly for the broader EventSource concept.
58+
The [`EventSource`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/EventSource.java#L93)
59+
interface actually implements the `Set<R> getSecondaryResources(P primary);` method, which is called by the context.
6260

63-
It is a bit more complex than that, since there can be multiple event sources for the same type, in that case
64-
the union of the results is returned.
61+
The implementation is slightly more complex when multiple event sources exist for the same resource type—
62+
in such cases, the union of all results is returned.
6563

6664
## Getting Resources Directly from Event Sources
6765

68-
Note that nothing stops you to directly access the resources in the cache (so not just through `getSecondaryResources(...)`):
66+
You can also directly access resources in the cache (not just through `getSecondaryResources(...)`):
6967

7068
```java
7169
public class WebPageReconciler implements Reconciler<WebPage> {
@@ -94,27 +92,26 @@ public class WebPageReconciler implements Reconciler<WebPage> {
9492

9593
## The Use Case for PrimaryToSecondaryMapper
9694

97-
TL;DR: `PrimaryToSecondaryMapper` is used to access secondary resources in `InformerEventSource` instead
98-
of the PrimaryToSecondaryIndex, thus `InfomerEventSource.getSecondaryResources(..)` will call this mapper
99-
to get the target secondary resources. This is usually required in cases when the `SecondaryToPrimaryMapper`
100-
is using the informer caches to list the target resources.
95+
**TL;DR**: `PrimaryToSecondaryMapper` allows `InformerEventSource` to access secondary resources directly
96+
instead of using the PrimaryToSecondaryIndex. When this mapper is configured, `InformerEventSource.getSecondaryResources(..)`
97+
will call the mapper to retrieve the target secondary resources. This is typically required when the `SecondaryToPrimaryMapper`
98+
uses informer caches to list the target resources.
10199

102-
As we discussed, we provide a unified API to access related resources using `Context.getSecondaryResources(...)`.
103-
The name `Secondary` refers to resources that a reconciler needs to take into account to properly reconcile a primary
104-
resource. These resources cover more than only `child` resources as resources created by a reconciler are sometimes
105-
called and which usually have an owner reference pointing to the primary (and, typically, custom) resource. These also
106-
cover `related` resources (which might or might not be managed by Kubernetes) that serve as input for reconciliations.
100+
As discussed, we provide a unified API to access related resources using `Context.getSecondaryResources(...)`.
101+
The term "Secondary" refers to resources that a reconciler needs to consider when properly reconciling a primary
102+
resource. These resources encompass more than just "child" resources (resources created by a reconciler that
103+
typically have an owner reference pointing to the primary custom resource). They also include
104+
"related" resources (which may or may not be managed by Kubernetes) that serve as input for reconciliations.
107105

108-
There are cases where the SDK needs more information than what is readily available, in particular when some of these
109-
secondary resources do not have owner references or any direct link to the primary resource they are associated
110-
with.
106+
In some cases, the SDK needs additional information beyond what's readily available, particularly when
107+
secondary resources lack owner references or any direct link to their associated primary resource.
111108

112-
As an example we provide, consider a `Job` primary resource which can be assigned to run on a cluster, represented by a
109+
Consider this example: a `Job` primary resource can be assigned to run on a cluster, represented by a
113110
`Cluster` resource.
114-
Multiple jobs can run on a given cluster so multiple `Job` resources can reference the same `Cluster` resource. However,
115-
a `Cluster` resource should not know about `Job` resources as this information is not part of what a cluster *is*.
116-
However, when a cluster changes, we might want to redirect the associated jobs to other clusters. Our reconciler
117-
therefore needs to figure out which `Job` (primary) resources are associated with the changed `Cluster` (secondary)
111+
Multiple jobs can run on the same cluster, so multiple `Job` resources can reference the same `Cluster` resource. However,
112+
a `Cluster` resource shouldn't know about `Job` resources, as this information isn't part of what defines a cluster.
113+
When a cluster changes, we might want to redirect associated jobs to other clusters. Our reconciler
114+
therefore needs to determine which `Job` (primary) resources are associated with the changed `Cluster` (secondary)
118115
resource.
119116
See full
120117
sample [here](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework/src/test/java/io/javaoperatorsdk/operator/baseapi/primarytosecondary).
@@ -128,15 +125,15 @@ InformerEventSourceConfiguration
128125
.collect(Collectors.toSet()))
129126
```
130127

131-
This will trigger all the related `Jobs` if the related cluster changes. Also, it maintains the `PrimaryToSecondaryIndex`.
132-
So we can use the `getSecondaryResources` in the `Job` reconciler to access the cluster.
133-
However, there is an issue: what if a new `Job` is created? The new job does not propagate
134-
automatically to `PrimaryToSecondaryIndex` in the `InformerEventSource` of the `Cluster`. That re-indexing
135-
happens when there is an event received for the `Cluster` and triggers all the `Jobs` again.
136-
Until that happens again you could not use `getSecondaryResources` for the new `Job`, since the new
137-
job won't be present in the reverse index.
128+
This configuration will trigger all related `Jobs` when the associated cluster changes and maintains the `PrimaryToSecondaryIndex`,
129+
allowing us to use `getSecondaryResources` in the `Job` reconciler to access the cluster.
130+
However, there's a potential issue: when a new `Job` is created, it doesn't automatically propagate
131+
to the `PrimaryToSecondaryIndex` in the `Cluster`'s `InformerEventSource`. Re-indexing only occurs
132+
when a `Cluster` event is received, which triggers all related `Jobs` again.
133+
Until this re-indexing happens, you cannot use `getSecondaryResources` for the new `Job`, since it
134+
won't be present in the reverse index.
138135

139-
You could access the Cluster directly from cache though in the reconciler:
136+
You can work around this by accessing the Cluster directly from the cache in the reconciler:
140137

141138
```java
142139

@@ -149,29 +146,29 @@ public UpdateControl<Job> reconcile(Job resource, Context<Job> context) {
149146
}
150147
```
151148

152-
But if you still want to use the unified API (thus `context.getSecondaryResources()`), we have to add
153-
`PrimaryToSecondaryMapper`:
149+
However, if you prefer to use the unified API (`context.getSecondaryResources()`), you need to add
150+
a `PrimaryToSecondaryMapper`:
154151

155152
```java
156153
clusterInformer.withPrimaryToSecondaryMapper( job ->
157154
Set.of(new ResourceID(job.getSpec().getClusterName(), job.getMetadata().getNamespace())));
158155
```
159156

160-
Using `PrimaryToSecondaryMapper` the InformerEventSource won't use the `PrimaryToSecondaryIndex`
161-
to get the resources, instead will call this mapper and will get the resources based on its result.
162-
In fact if this mapper is set the `PrimaryToSecondaryIndex` is not even initialized.
157+
When using `PrimaryToSecondaryMapper`, the InformerEventSource bypasses the `PrimaryToSecondaryIndex`
158+
and instead calls the mapper to retrieve resources based on its results.
159+
In fact, when this mapper is configured, the `PrimaryToSecondaryIndex` isn't even initialized.
163160

164161
### Using Informer Indexes to Improve Performance
165162

166-
In the `SecondaryToPrimaryMapper` above we are looping through all the resources in the cache:
163+
In the `SecondaryToPrimaryMapper` example above, we iterate through all resources in the cache:
167164

168165
```java
169166
context.getPrimaryCache()
170167
.list().filter(job -> job.getSpec().getClusterName().equals(cluster.getMetadata().getName()))
171168
```
172169

173-
This can be inefficient in case there is a large number of primary (Job) resources. To make it more efficient, we can
174-
create an index in the underlying Informer that indexes the target jobs for a cluster:
170+
This approach can be inefficient when dealing with a large number of primary (Job) resources. To improve performance,
171+
you can create an index in the underlying Informer that indexes the target jobs for each cluster:
175172

176173
```java
177174

@@ -194,7 +191,7 @@ private String indexKey(String clusterName, String namespace) {
194191
}
195192
```
196193

197-
From this point, we can use the index to get the target resources very efficiently:
194+
With this index in place, you can retrieve the target resources very efficiently:
198195

199196
```java
200197

0 commit comments

Comments
 (0)