Skip to content

Commit ed2592b

Browse files
theakshaypantclaude
authored and
zakisk
committed
docs(informer): add cache optimization documentation
Document the informer cache TransformFunc settings for Repository and PipelineRun objects, following the tektoncd/pipeline#9316 as reference. Includes field-by-field stripping tables, benchmark results, graceful degradation details, and developer guidance. Signed-off-by: Akshay Pant <akpant@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8892691 commit ed2592b

2 files changed

Lines changed: 142 additions & 0 deletions

File tree

‎docs/content/docs/dev/architecture.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -543,6 +543,9 @@ data:
543543
- Configure PipelineRun retention (`max-keep-runs`)
544544
- Enable remote task caching
545545
- Use volume workspaces instead of PVCs for better performance
546+
- Informer cache transforms automatically strip unused fields from cached
547+
objects, reducing watcher memory usage by up to 94% per object — see
548+
[Informer Cache Optimization]({{< relref "/docs/operations/informer-cache.md" >}})
546549

547550
## Security Architecture
548551

‎docs/content/docs/operations/informer-cache.md‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
---
2+
title: Informer Cache Optimization
3+
weight: 9
4+
---
5+
6+
This page describes the informer cache transform functions that
7+
Pipelines-as-Code uses to reduce the memory footprint of its watcher
8+
controller. These transforms are applied automatically and require no
9+
configuration.
10+
11+
## Background
12+
13+
The PAC watcher controller maintains in-memory caches (via Kubernetes
14+
[informers](https://pkg.go.dev/k8s.io/client-go/tools/cache#SharedInformer))
15+
for **Repository** and **PipelineRun** objects. In clusters with many
16+
repositories or long-running PipelineRuns, these caches can consume a
17+
significant amount of memory because each cached object carries fields that
18+
the watcher never reads — `managedFields`, `last-applied-configuration`
19+
annotations, embedded specs, and status details.
20+
21+
Pipelines-as-Code registers
22+
[TransformFunc](https://pkg.go.dev/k8s.io/client-go/tools/cache#TransformFunc)
23+
callbacks on each informer. A TransformFunc is called on every object
24+
**before** it is stored in the cache, allowing unnecessary fields to be
25+
stripped while the object is still in use.
26+
27+
This is the same approach used by the
28+
[Tekton Pipelines controller](https://github.com/tektoncd/pipeline/pull/9316)
29+
to reduce its own cache memory usage.
30+
31+
## What Gets Stripped
32+
33+
### Repository Objects
34+
35+
| Field | Why it is safe to strip |
36+
| --- | --- |
37+
| `metadata.managedFields` | Written by the API server for server-side apply tracking; not read by any reconciler logic |
38+
| `metadata.annotations` | No reconciler logic reads Repository annotations from the lister; the largest annotation (`kubectl.kubernetes.io/last-applied-configuration`) can be 500-2000 bytes alone |
39+
| `status` | The reconciler always fetches `Repository.Status` via a direct API call before updating it; it is never read from the lister |
40+
41+
**Benchmark result:** ~89% JSON size reduction per Repository object.
42+
43+
### PipelineRun Objects
44+
45+
#### Fields preserved (required for reconciliation)
46+
47+
| Field | Used for |
48+
| --- | --- |
49+
| `metadata.name`, `metadata.namespace` | Object identity |
50+
| `metadata.labels` | Repository lookup, pipeline identification |
51+
| `metadata.annotations` | PAC state (`pipelinesascode.tekton.dev/state`), repository keys |
52+
| `metadata.finalizers`, `metadata.deletionTimestamp` | Finalizer-based cleanup |
53+
| `spec.status` | Detecting `PipelineRunSpecStatusPending` |
54+
| `status.conditions` | Completion state and reason |
55+
| `status.startTime`, `status.completionTime` | Metrics recording |
56+
57+
#### Fields stripped
58+
59+
| Field | Why it is safe to strip |
60+
| --- | --- |
61+
| `metadata.managedFields` | API server bookkeeping, not used in reconciliation |
62+
| `spec.pipelineRef` | Not read from the cache by the watcher |
63+
| `spec.pipelineSpec` | Embedded pipeline definition; can be very large (~20KB in production) |
64+
| `spec.params` | Not read from the cache |
65+
| `spec.workspaces` | Not read from the cache |
66+
| `spec.taskRunSpecs` | Not read from the cache |
67+
| `spec.taskRunTemplate` | Not read from the cache |
68+
| `spec.timeouts` | Not read from the cache |
69+
| `status.pipelineSpec` | Snapshot of the executed pipeline spec; the largest status field (~20KB) |
70+
| `status.childReferences` | References to child TaskRuns |
71+
| `status.provenance` | Build provenance metadata |
72+
| `status.spanContext` | Tracing span context |
73+
74+
When the reconciler needs the full PipelineRun (for example, during
75+
`postFinalStatus` or `GetStatusFromTaskStatusOrFromAsking`), it fetches the
76+
complete object directly from the API server.
77+
78+
**Benchmark result:** ~94% JSON size reduction per PipelineRun object.
79+
80+
## Memory Impact
81+
82+
Benchmarks using realistic object sizes from production clusters show the
83+
following per-object savings:
84+
85+
| Object | Original size | After transform | Reduction |
86+
| --- | --- | --- | --- |
87+
| Repository (5 status entries) | ~5.7 KB | ~0.6 KB | ~89% |
88+
| PipelineRun (15-task pipeline) | ~10.7 KB | ~0.7 KB | ~94% |
89+
90+
For a cluster with 1000 Repositories and 700 PipelineRuns, the estimated
91+
watcher cache reduction is approximately **12 MB**.
92+
93+
## Graceful Degradation
94+
95+
The transform functions are designed to degrade gracefully:
96+
97+
- If an object is wrapped in a
98+
[`DeletedFinalStateUnknown`](https://pkg.go.dev/k8s.io/client-go/tools/cache#DeletedFinalStateUnknown)
99+
tombstone (which happens when the watcher misses a delete event), the
100+
transform unwraps it, strips the inner object, and re-wraps it.
101+
- If the transform receives an unexpected type, it returns the object
102+
unmodified rather than returning an error.
103+
- If any error occurs during transformation, the original object is returned
104+
unchanged so that the informer cache continues to function.
105+
106+
## Developer Notes
107+
108+
If you add new reconciliation logic that reads a field from cached objects
109+
(via listers), you **must** verify that the field is not stripped by these
110+
transforms. Fields stripped from cached objects will be `nil` or empty even
111+
though they exist in etcd.
112+
113+
If you need a stripped field, fetch the full object via the API client
114+
instead of the lister:
115+
116+
```go
117+
// Don't do this — spec.params is stripped from the cache:
118+
pr, _ := pipelineRunLister.PipelineRuns(ns).Get(name)
119+
params := pr.Spec.Params // always nil!
120+
121+
// Do this instead — fetch from the API server:
122+
pr, _ := tektonClient.TektonV1().PipelineRuns(ns).Get(ctx, name, metav1.GetOptions{})
123+
params := pr.Spec.Params // full object from etcd
124+
```
125+
126+
The transform functions and their benchmarks live in
127+
[`pkg/informer/transform/`](https://github.com/tektoncd/pipelines-as-code/tree/main/pkg/informer/transform).
128+
129+
To run the benchmarks yourself:
130+
131+
```bash
132+
go test -bench=. -benchmem -v ./pkg/informer/transform/
133+
```
134+
135+
To see the size reduction report:
136+
137+
```bash
138+
go test -v -run 'TestMeasure.*TransformSavings' ./pkg/informer/transform/
139+
```

0 commit comments

Comments
 (0)