|
1 | 1 | Default Catalog Consistency |
2 | 2 | ======================== |
3 | 3 |
|
4 | | -Those tests are used to check the consistency of the default catalogs in OpenShift. |
| 4 | +These tests verify the consistency of the default catalogs shipped with OpenShift. |
| 5 | +Default catalogs provide the operator content that OLM manages on every cluster. |
| 6 | +If a catalog is broken, operators cannot be installed or upgraded, which directly |
| 7 | +affects supportability. |
5 | 8 |
|
6 | | -### Running Tests |
| 9 | +## What it checks |
| 10 | + |
| 11 | +The test suite reads catalog image references from the ClusterCatalog manifests |
| 12 | +(at `catalogd/manifests.yaml`), pulls each image, and runs four layers of checks: |
| 13 | + |
| 14 | +| Layer | What it validates | Examples | |
| 15 | +|-------|-------------------|----------| |
| 16 | +| **Image** | The container image is well-formed | Valid OCI manifest, has required labels (e.g. `operators.operatorframework.io.index.configs.v1=/configs`) | |
| 17 | +| **Multi-arch** | The image supports all required platforms | linux/amd64, linux/arm64, linux/ppc64le, linux/s390x | |
| 18 | +| **Filesystem** | Expected files and directories exist inside the image | `bin/opm` is a valid Go binary, `configs/` and `tmp/cache/` directories exist | |
| 19 | +| **Catalog (FBC)** | The File-Based Catalog content is valid | No bundles use the deprecated `olm.bundle.object` property; all channel heads have `olm.csv.metadata` | |
| 20 | + |
| 21 | +The check implementations live in `pkg/check/` and the test entry point is |
| 22 | +`test/validate/suite_test.go`. |
| 23 | + |
| 24 | +## CI configuration |
| 25 | + |
| 26 | +CI and periodic jobs are configured in the |
| 27 | +[openshift/release](https://github.com/openshift/release) repository under |
| 28 | +[ci-operator/config/openshift/operator-framework-operator-controller/](https://github.com/openshift/release/tree/master/ci-operator/config/openshift/operator-framework-operator-controller). |
| 29 | + |
| 30 | +### Presubmit |
| 31 | + |
| 32 | +A presubmit job runs on every PR that touches catalog-related files. |
| 33 | +See the [main branch config](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/operator-framework-operator-controller/openshift-operator-framework-operator-controller-main.yaml) |
| 34 | +(search for `default-catalog-consistency`). |
| 35 | + |
| 36 | +### Periodic |
| 37 | + |
| 38 | +Each release branch has a periodic job that runs on a schedule and reports results |
| 39 | +to [Sippy](https://sippy.dptools.openshift.org/sippy-ng/). Look for files ending |
| 40 | +in `__periodics.yaml`, for example: |
| 41 | + |
| 42 | +- [release-4.20 periodics](https://github.com/openshift/release/blob/main/ci-operator/config/openshift/operator-framework-operator-controller/openshift-operator-framework-operator-controller-release-4.20__periodics.yaml) |
| 43 | + |
| 44 | +You can view results per release in Sippy. For example: |
| 45 | +[4.22 default-catalog jobs](https://sippy.dptools.openshift.org/sippy-ng/jobs/4.22?filters=%257B%2522items%2522%253A%255B%257B%2522columnField%2522%253A%2522variants%2522%252C%2522operatorValue%2522%253A%2522has%2520entry%2522%252C%2522value%2522%253A%2522never-stable%2522%252C%2522not%2522%253Atrue%257D%252C%257B%2522id%2522%253A99%252C%2522columnField%2522%253A%2522name%2522%252C%2522operatorValue%2522%253A%2522contains%2522%252C%2522value%2522%253A%2522default-catalog%2522%257D%255D%257D&sort=asc&sortField=net_improvement). |
| 46 | + |
| 47 | +## Monitoring and alerts |
| 48 | + |
| 49 | +When periodic jobs fail, alerts are sent to the `#forum-ocp-catalogs-program` Slack channel. |
| 50 | +[Example alert](https://redhat-internal.slack.com/archives/C08D3G4EMRA/p1777089324894579). |
| 51 | + |
| 52 | +> **Note:** Currently, thresholds are high and many alerts are caused by flakes or |
| 53 | +> infrastructure issues (like the example above). We must still monitor these alerts |
| 54 | +> because a real failure means the default catalogs are broken -- operators cannot be |
| 55 | +> installed or upgraded, and this is critical for the content managed by OLM. |
| 56 | +
|
| 57 | +## Running tests locally |
7 | 58 |
|
8 | 59 | ```bash |
9 | 60 | make test-catalog |
10 | 61 | ``` |
11 | 62 |
|
12 | | -### Before Pushing Changes |
| 63 | +## Before pushing changes |
13 | 64 |
|
14 | | -Before pushing changes to the repository, you should verify that the tests pass. |
15 | | -This can be done by running: |
| 65 | +Run the following to verify that tests pass and code is properly formatted: |
16 | 66 |
|
17 | 67 | ```bash |
18 | 68 | make verify |
19 | | -``` |
| 69 | +``` |
0 commit comments