Fix various sources

[sebhoss]

The koperator repository had a wrong version prefix and we were not able to detect any versions.

The following repos all used searchPaths instead of kustomizationPaths which caused the inclusion of CRDs with an empty group and kind. Take for example https://github.com/aws/amazon-network-policy-controller-k8s/tree/main/config/crd which contains both CRDs and patches for those CRDs. The usage of searchPaths will now cause those patches to overwrite the actual CRD. The end result is that we get a file like https://github.com/CustomResourceDefinition/catalog/blob/main/definitions/.yaml in the catalog:

• aws-amazon-network-policy-controller-k8s
• loft-sh-cluster-api-provider-vcluster
• oracle-cluster-api-provider-oci
• tinkerbell-cluster-api-provider-tinkerbell
• vmware-tanzu-cluster-api-provider-bringyourownhost

The repository https://github.com/upbound/provider-terraform was removed and therefore deleted from the configuration file as well.

The clusternet repo was missing some helm values to render properly.

[sebhoss]

BTW .. after opening this PR I thought that maybe I should not have updated the definitions/schemas locally and instead only push the changes to the configuration file? Not sure what your preferred workflow here is?

Likewise, for local development I think it would be great if there is an option to run the update command with an additional CLI flag to filter the repository you want to update? Using this flag, you would be able to edit the configuration file, run the updater to check the result and commit (only the config change?). Would a CLI flag like that be something that you'd consider? I could open a PR for that!

[CodeReaper]

BTW .. after opening this PR I thought that maybe I should not have updated the definitions/schemas locally and instead only push the changes to the configuration file? Not sure what your preferred workflow here is?

I prefer only changing the configuration file and allowing the workflow to actually make the updates, mainly to ensure changes are consistent (and not accidentally affected by local changes).

Likewise, for local development ...

I have long been wanting to make a guide for running local testing of configuration changes, but I have not gotten around to making one.

Could you describe how you tested locally?

The short version of how I am testing locally is:

• Update test/configuration.yaml with (only) repositories to test
• make shell
• build/bin/catalog update --configuration test/configuration.yaml --output build/ephemeral/schema --definitions build/ephemeral/schema

To debug in Go, there is this unit test.

[sebhoss]

Ok ok will adapt the commits so that only the config file is modified.

I did not see test/configuration.yaml and did not realize that build/ephemeral/schema exists, therefore I tested against the root configuration file and modified it so that it only contains the repositories I wanted to test. Since I had to revert this change after testing, I thought that a CLI flag to only select certain repositories would make sense.

[sebhoss]

Oh I should probably keep the deletion of https://github.com/CustomResourceDefinition/catalog/blob/main/definitions/.yaml since there is nothing that cleans this up, right?

[CodeReaper]

I did not see test/configuration.yaml and did not realize that build/ephemeral/schema exists, therefore I tested against the root configuration file and modified it so that it only contains the repositories I wanted to test.

Yes, this is why some sort of guide is needed :)

Oh I should probably keep the deletion of https://github.com/CustomResourceDefinition/catalog/blob/main/definitions/.yaml since there is nothing that cleans this up, right?

Yes definitions/.yaml will need to be deleted manually.