feat: e2e test as single task cmd (#81)

Provide e2e test execution for extensions as a single Taskfile command
rather than having to chain multiple steps externally.

Closes: #80
---------

Signed-off-by: Gabriele Fedi <gabriele.fedi@enterprisedb.com>
Signed-off-by: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>
Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Co-authored-by: Gabriele Bartolini <gabriele.bartolini@enterprisedb.com>
Co-authored-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Co-authored-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>

Author: 4 people

BUILD.md

@@ -94,96 +94,89 @@ cluster with CloudNativePG pre-installed.
 > **Pre-submission requirement:** You must successfully run local tests before
 > submitting a Pull Request for any extension.
 
+### The Fast Path (Automated Testing)
+
+End-to-end (E2E) tests are powered by [Chainsaw](https://github.com/kyverno/chainsaw).
+To simplify the workflow, use the `e2e:test:full` task.
+This single command automates environment setup, image building, and test
+execution:
+
+```bash
+# Replace <extension> with the name of the extension (e.g., pgvector)
+task e2e:test:full TARGET="<extension>"
+```
+
+Example for testing the `pgvector` extension:
+
+```bash
+task e2e:test:full TARGET="pgvector"
+```
+
+**When to use the step-by-step guide:** If the automated test fails during
+environment setup, image building, or test execution, the step-by-step guide
+below provides granular control for debugging each phase independently.
+
+---
+
+## E2E step-by-step Guide
+
 ### Initialize the environment
 
-The `e2e:setup-env` utility automates the heavy lifting. It creates a Kind
-cluster, attaches a local Docker registry (available at `localhost:5000`), and
-installs the CloudNativePG operator.
+The `e2e:setup-env` utility creates a Kind cluster and attaches a local Docker
+registry (available at `localhost:5000`).
 
 ```bash
 task e2e:setup-env
 ```
 
 ### Get access to the cluster
 
-Even though the cluster is running, your local `kubectl` doesn't know how to
-talk to it yet. You need to "export" the credentials (the Kubeconfig).
-
-If you want to run `kubectl get pods` from your own laptop's terminal, use the
-standard export:
+To interact with the cluster via `kubectl` from your local terminal:
 
 ```bash
 task e2e:export-kubeconfig KUBECONFIG_PATH=./kubeconfig
 export KUBECONFIG=$PWD/kubeconfig
 ```
 
-If you are running a test script that is also running inside a Docker container
-on the same network, like in the case of Kind, it needs the "internal" address
-to find the API server:
+To allow the test suite (running within the Docker network) to reach the API
+server:
 
 ```bash
 task e2e:export-kubeconfig KUBECONFIG_PATH=./kubeconfig INTERNAL=true
 ```
 
 ### Build and push the extension (`bake`)
 
-Before the cluster can use your extension, you must build the image and push it
-to the local registry (see ["Push images for a specific project" above](#6-push-images-for-a-specific-project)):
+Build the image and push it to the local registry. This command tags the image
+for `localhost:5000` automatically.
 
 ```bash
 task bake TARGET="<extension>" PUSH=true
 ```
 
-This command tags the image for `localhost:5000` and pushes it automatically.
-
-> [!TIP]
-> You can change the default registry through the `registry` environment variable
-> (defined in the `docker/bake.hcl` file).
+> [!NOTE]
+> The destination registry is controlled by the `registry` variable defined within the `docker/bake.hcl` file.
 
 ### Prepare testing values
 
-We use [Chainsaw](https://github.com/kyverno/chainsaw) for declarative
-end-to-end testing. Before running tests, you must generate specific
-configuration values for your extension image.
-
-Run the following command to export these values into your extension's
-directory:
+Generate configuration values so Chainsaw knows which local image to target for
+the E2E tests:
 
 ```bash
 task e2e:generate-values TARGET="<extension>" EXTENSION_IMAGE="<my-local-image>"
 ```
 
-For example, to generate the values for the local test of the local image, you could run something similar to the following:
-
-```bash
-# The actual name of the image might be different on your system
-task e2e:generate-values TARGET=pgvector EXTENSION_IMAGE="localhost:5000/pgvector-testing:0.8.1-18-trixie"
-```
-
 ### Execute End-to-End tests
 
-The testing framework requires an internal Kubeconfig to communicate correctly
-within the Docker network.
-
-First, export the internal configuration as shown above:
-
-```bash
-task e2e:export-kubeconfig KUBECONFIG_PATH=./kubeconfig INTERNAL=true
-```
-
-Then, run the `e2e:test` task. This executes both the generic tests (located in
-the global `/test` folder) and any extension-specific tests (located in the
-target's `/test` folder):
+Run the test suite using the internal Kubeconfig. This executes both the
+generic tests (global `/test` folder) and extension-specific tests (target
+`/test` folder).
 
 ```bash
 task e2e:test TARGET="<extension>" KUBECONFIG_PATH="./kubeconfig"
 ```
 
-You can test the `pgvector` extension with:
-
-```bash
-task e2e:test TARGET="pgvector" KUBECONFIG_PATH="./kubeconfig"
-```
+---
 
 ### Tear down the local test environment
 

Taskfile.yml

@@ -75,16 +75,17 @@ tasks:
     vars:
       PUSH: '{{.PUSH | default "false"}}'
       DRY_RUN: '{{.DRY_RUN | default "false"}}'
+      METADATA_FILE: '{{.METADATA_FILE | default ""}}'
     preconditions:
       - sh: test -f "{{.TARGET}}/metadata.hcl"
         msg: 'Not a valid target, metadata.hcl file is missing. Target: {{.TARGET}}'
     cmds:
       - echo -e "{{.BLUE}}Baking target {{.TARGET}} (push={{.PUSH}})...{{.NC}}"
       - |
         if [ "{{.DRY_RUN}}" = "true" ]; then
-          echo -e "{{.GREEN}}[DRY RUN] docker buildx bake -f {{.TARGET}}/metadata.hcl -f docker-bake.hcl --push={{.PUSH}}{{.NC}}"
+          echo -e "{{.GREEN}}[DRY RUN] docker buildx bake -f {{.TARGET}}/metadata.hcl -f docker-bake.hcl --metadata-file="{{.METADATA_FILE}}" --push={{.PUSH}}{{.NC}}"
         else
-          docker buildx bake -f {{.TARGET}}/metadata.hcl -f docker-bake.hcl --push={{.PUSH}}
+          docker buildx bake -f {{.TARGET}}/metadata.hcl -f docker-bake.hcl --metadata-file="{{.METADATA_FILE}}" --push={{.PUSH}}
         fi
       - echo -e "{{.GREEN}}--- Successfully baked {{.TARGET}} ---{{.NC}}"
     requires:
@@ -343,6 +344,42 @@ tasks:
           TARGET: "{{ .ITEM }}"
         task: e2e:test
 
+  e2e:test:full:
+    desc: "Run full E2E tests: setup env, generate values, run tests"
+    vars:
+      METADATA_FILE: "{{ .TARGET }}/bake-metadata.json"
+      KUBECONFIG_PATH: "./kubeconfig"
+      DISTRO: '{{ .DISTRO | default "trixie" }}'
+    cmds:
+      - task: e2e:setup-env
+      - task: bake
+        vars:
+          PUSH: "true"
+          TARGET: "{{ .TARGET }}"
+          METADATA_FILE: "{{ .METADATA_FILE }}"
+      - defer: rm -f "{{ .METADATA_FILE }}"
+      - task: e2e:generate-values
+        vars:
+          TARGET: "{{ .TARGET }}"
+          EXTENSION_IMAGE:
+            sh: >
+              jq -r --arg distro "{{ .DISTRO }}"
+              'to_entries | .[] | select(.key | endswith("-" + $distro)) | .value["image.name"] | split(",")[0]'
+              "{{ .METADATA_FILE }}"
+      - defer: rm -f "{{ .TARGET }}/values.yaml"
+      - task: e2e:export-kubeconfig
+        vars:
+          KUBECONFIG_PATH: "{{ .KUBECONFIG_PATH }}"
+          INTERNAL: "true"
+      - defer: rm -f "{{ .KUBECONFIG_PATH }}"
+      - task: e2e:test
+        vars:
+          TARGET: "{{ .TARGET }}"
+          KUBECONFIG_PATH: "{{ .KUBECONFIG_PATH }}"
+    requires:
+      vars:
+        - name: TARGET
+
   e2e:cleanup:
     desc: Cleanup E2E resources
     deps: