kustomize: support merging the input OpenAPI schemas with the built-in ones

[matheuscscp]

xref: fluxcd/flux2#5972 (Avoiding Closes to keep the issue open until a patch is released)

This PR uses envtest to fully reproduce and verify the fix for the issue above:

1. A DaemonSet is used.
2. A strategic merge patch is applied on it.
3. A dry-run is performed with the result.

With merge, dry-run succeeds. Without merge, the exact dry-run error from fluxcd/flux2#5972 is asserted.

Note: We preserve Kustomize's winning rule for multiple openapi.path occurrences in a given build: overlay wins, otherwise first base wins.

[matheuscscp]

We need to clone the built-in OpenAPI schema for each openapi.path in the tree because we mutate this clone to merge it with the contents of that openapi.path file.

This effectively results in, per openapi.path, roughly 11.4 MiB of JSON encode/decode plus writing a ~3.8 MiB .flux-openapi-merged.json to disk.

[stefanprodan]

All of this happens under the mutex lock?

[matheuscscp]

Yes, like everything else in a kustomize build:

  kustomizeBuildMutex.Lock()
  defer kustomizeBuildMutex.Unlock()

  openapi.ResetOpenAPI()

  if buildOpts.mergeOpenAPIPathWithBuiltins { // defaults to true
      mergeOpenAPIPathWithBuiltins(fs, dirPath)
  }

  k.Run(fs, dirPath)

[stefanprodan]

I wander what's the real memory impact, these clones will accumulate till GC decides to wipe them. For users that don't set openapi.path there is still a price to pay since we have to walk the file tree and parse all kustomization.yaml along the way.

[matheuscscp]

For users that don't set openapi.path there is still a price to pay since we have to walk the file tree and parse all kustomization.yaml along the way.

This is negligible compared to the existing parsing of the full builtin schema when a strategic merge patch is defined. Cloning, merging and writing the full builtin schema once per openapi.path in the tree is the real concern. Users can mitigate that by setting openapi.path only once in the overlay. Don't set in the bases...

[stefanprodan]

LGTM

Thanks @matheuscscp 🏅

[fluxcdbot]

Successfully created backport PR for flux/v2.9.x:

• [flux/v2.9.x] kustomize: support merging the input OpenAPI schemas with the built-in ones #1259