ci: scaffold e2e testing framework infrastructure#763
Conversation
50ea8f9 to
1428a84
Compare
|
/ok-to-test |
4dccdd2 to
0f1461d
Compare
0f1461d to
4b75619
Compare
|
This pull request has been automatically marked as stale because it has not had recent activity. Members may comment |
|
not stale... outstanding items i need to add:
|
4b75619 to
cbed7b1
Compare
christian-heusel
left a comment
There was a problem hiding this comment.
Nice work @andyatmiami 🔥
I think there are a few things that can be tweaked before we can get this over the finish line, I have left a few review comments with points that I found during a quick review below 😊
A few of those remarks (regarding the .gitignore for example) are a bit nitpicky, feel free to ignore them at your liking 🤗
The deploy target's `kustomize edit set image` was never actually overriding the base kustomization's image tag. This was masked on non-release branches because the base newTag (latest) happened to match the hardcoded test image tag. On release branches, the base newTag changes to the release version, exposing the mismatch and causing e2e test pods to stay Pending (image not found in Kind). Fix by: - Copying kustomize to .output/ so edits don't modify tracked files - Matching both short name and registry-prefixed name to ensure the override works regardless of base kustomization state Extracted from kubeflow#763. Signed-off-by: Andy Stoneberg <astonebe@redhat.com>
92add3a to
4daaf34
Compare
christian-heusel
left a comment
There was a problem hiding this comment.
Nice work, I took a second look at this PR after the rebase and had a few minor things to note, but most of it are optional improvements / remarks, feel free to implement or ignore as desired 😊
4daaf34 to
d2a2057
Compare
This commit establishes the foundational infrastructure for end-to-end testing of the workspaces components. While some of the scripts appear redundant with the developing/ directory, a deliberate decision was made to keep this logic mutually exclusive to be more flexible. For instance, the testing/ directory has a need to deploy a Gateway that is not necessary for developing. Keeping everything separate for now makes it easier to evolve independently. Changes include: - Add new `testing/` directory with Makefile and setup scripts: * `setup-kind.sh`: Automated Kind cluster creation and configuration * `setup-cert-manager.sh`: Cert-manager installation (v1.12.13 LTS) * `setup-istio.sh`: Istio service mesh installation with Gateway and TLS certificate provisioning via cert-manager * `check-kind-context.sh`: Safety check to prevent accidental deployment to non-Kind clusters * `sanity-check.sh`: Post-deploy verification including rollout status, TLS handshake (webhook), HTTP health endpoints, and Istio gateway routing for backend and frontend * `gateway.yaml`: Istio Gateway (HTTP + HTTPS) for kubeflow-gateway * `gateway-cert.yaml`: Self-signed ClusterIssuer and Certificate for gateway TLS termination * Makefile targets: setup-cluster, deploy-all, sanity-check, teardown-cluster, clean, and local-e2e (placeholder) - Add GitHub Actions workflow (`.github/workflows/ws-e2e-test.yml`): * Triggers on pushes to main branches and PRs affecting workspaces * Pipeline: setup-cluster -> deploy-all -> sanity-check -> local-e2e Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com>
d2a2057 to
b5a5a5e
Compare
Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com>
b5a5a5e to
4008f6d
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com>
christian-heusel
left a comment
There was a problem hiding this comment.
Only had two minor comments, left a LGTM 👍🏻
| --timeout=120s | ||
|
|
||
| # Apply gateway resources directly (in developing/ these are managed by Tilt) | ||
| kubectl create namespace kubeflow --dry-run=client -o yaml | kubectl apply -f - |
There was a problem hiding this comment.
Creating the namespace this way will not apply needed annotations like PSS labels that the manifests use in this case, see:
apiVersion: v1
kind: Namespace
metadata:
name: kubeflow
labels:
control-plane: kubeflow
istio-injection: enabled
pod-security.kubernetes.io/enforce: restrictedThere was a problem hiding this comment.
added the pod-security.kubernetes.io/enforce: restricted in 0a49ddc
the others are not needed in these tests
|
/lgtm reviewed live with @thesuperzapper on the commits he pushed to this PR. i acknowledge and accept these changes. |
|
@andyatmiami: you cannot LGTM your own PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
So happy to have this merged, we finally have the base of our E2E tests running on every PR/commit! /lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: thesuperzapper The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
24ce51e
into
kubeflow:notebooks-v2
# This is the 1st commit message: feat: implement podtemplate details endpoint for workspace overlay Signed-off-by: Sneha Das <154408198+Snehadas2005@users.noreply.github.com> chore: update ts-jest to 29.4.9 (kubeflow#1096) Signed-off-by: Sneha Das <154408198+Snehadas2005@users.noreply.github.com> fix: add namespace to storageclasses api and check pvc create for auth (kubeflow#1097) * fix: add namespace parameter to storageclasses endpoint Regular Kubeflow users get 403 on /storageclasses because the backend issues a cluster-wide SAR (namespace="") that requires a ClusterRoleBinding, but users only have a namespace-scoped RoleBinding. - Add namespace query param to GET /storageclasses: when provided the SAR is scoped to that namespace (matching the existing workspacekinds pattern), allowing regular users through; without it the endpoint remains admin-only - Add nolint:dupl to GetStorageClassesHandler and GetWorkspaceKindsHandler Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Christian Heusel <christian@heusel.eu> * chore: disable duplication linter for handler files Add a targeted golangci.yml exclude rule for api/*_handler.go files instead of per-function //nolint:dupl comments, and remove all stale directives from pvcs, secrets, workspaces, workspacekinds, and storageclasses handlers. Link: kubeflow#1097 (comment) Signed-off-by: Christian Heusel <christian@heusel.eu> --------- Signed-off-by: Christian Heusel <christian@heusel.eu> # This is the commit message kubeflow#2: chore: note reason for the role aggregation structure (kubeflow#1100) This has caused multiple people confusion by now which means that we should most likely note it in the code itself. Additionally this way we guard ourselves against reverting / "simplifying" this on accident in the future. Signed-off-by: Christian Heusel <christian@heusel.eu> Co-authored-by: Mathew Wicks <5735406+thesuperzapper@users.noreply.github.com> # This is the commit message kubeflow#3: chore: Bump the mod-arch-packages group across 1 directory with 3 updates (kubeflow#1084) Bumps the mod-arch-packages group with 3 updates in the /workspaces/frontend directory: [mod-arch-core](https://github.com/opendatahub-io/mod-arch-library/tree/HEAD/mod-arch-core), [mod-arch-kubeflow](https://github.com/opendatahub-io/mod-arch-library/tree/HEAD/mod-arch-kubeflow) and [mod-arch-shared](https://github.com/opendatahub-io/mod-arch-library/tree/HEAD/mod-arch-shared). Updates `mod-arch-core` from 1.15.4 to 1.16.1 - [Release notes](https://github.com/opendatahub-io/mod-arch-library/releases) - [Commits](https://github.com/opendatahub-io/mod-arch-library/commits/1.16.1/mod-arch-core) Updates `mod-arch-kubeflow` from 1.15.4 to 1.16.1 - [Release notes](https://github.com/opendatahub-io/mod-arch-library/releases) - [Commits](https://github.com/opendatahub-io/mod-arch-library/commits/1.16.1/mod-arch-kubeflow) Updates `mod-arch-shared` from 1.15.4 to 1.16.1 - [Release notes](https://github.com/opendatahub-io/mod-arch-library/releases) - [Commits](https://github.com/opendatahub-io/mod-arch-library/commits/1.16.1/mod-arch-shared) --- updated-dependencies: - dependency-name: mod-arch-core dependency-version: 1.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: mod-arch-packages - dependency-name: mod-arch-kubeflow dependency-version: 1.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: mod-arch-packages - dependency-name: mod-arch-shared dependency-version: 1.16.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: mod-arch-packages ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> # This is the commit message kubeflow#4: feat: enable WorkspaceKind update in FrontEnd (kubeflow#1093) * feat: Enable WorkspaceKind Update Signed-off-by: Charles Thao <cthao@redhat.com> * Only enable Save when changes are made Signed-off-by: Charles Thao <cthao@redhat.com> * add tests Signed-off-by: Charles Thao <cthao@redhat.com> --------- Signed-off-by: Charles Thao <cthao@redhat.com> # This is the commit message kubeflow#5: feat: use listValues to render Image and Pod Config steps in Workspac… (kubeflow#1098) * feat: use listValues to render Image and Pod Config steps in Workspace Form Signed-off-by: Charles Thao <cthao@redhat.com> * fix: make error message display more explicit Signed-off-by: Charles Thao <cthao@redhat.com> --------- Signed-off-by: Charles Thao <cthao@redhat.com> # This is the commit message kubeflow#6: fix: pass namespace to list workspacekinds on workspace edit (kubeflow#1106) * fix(frontend): use useWorkspaceKinds in useWorkspaceFormData to fix 403 on edit When editing a Workspace via the UI, non-admin users received a 403 error. The root cause was that useWorkspaceFormData called listWorkspaceKinds({}) directly — without the namespaceFilter query parameter. Without namespaceFilter, the backend checks for cluster-level "list WorkspaceKinds" permission instead of the namespace-scoped "create Workspaces" permission that regular users have. The create flow worked because WorkspaceFormKindSelection fetches kinds through useWorkspaceKinds(namespace), which always includes namespaceFilter. The edit flow bypassed that code path entirely. Rather than just adding namespaceFilter to the raw API call, this fix replaces the direct listWorkspaceKinds({}) call with the shared useWorkspaceKinds(namespace) hook. Both the edit flow (useWorkspaceFormData) and the create flow (WorkspaceFormKindSelection) now go through the same code path, so they cannot diverge on authorization behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> * chore: address PR feedback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> --------- Signed-off-by: Andy Stoneberg <astonebe@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> # This is the commit message kubeflow#7: fix: pass namespace query param when listing storage classes (kubeflow#1107) PR kubeflow#1097 added a `namespace` query parameter to the backend's GET /storageclasses endpoint. When provided, the backend checks namespace-scoped "create PersistentVolumeClaims" permission instead of cluster-level "list StorageClasses" — allowing regular (non-admin) users to access storage classes within their namespace. The frontend's useStorageClasses hook was calling listStorageClasses() without the namespace parameter, causing non-admin users to receive a 403 when opening the volume creation or attachment modals in the Workspace Wizard (both Create and Edit flows). This fix mirrors the approach taken in PR kubeflow#1106 for workspace kinds: the useStorageClasses hook now accepts an optional namespace argument and passes it as a query parameter to the API. Both VolumesCreateModal and VolumesAttachModal pass selectedNamespace from the namespace selector context. swagger.version is updated from c9a8a7c to 142a30f (the PR kubeflow#1097 commit) so that the generated API client includes the new namespace query parameter on listStorageClasses. Signed-off-by: Andy Stoneberg <astonebe@redhat.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> # This is the commit message kubeflow#8: ci: automatically approve tests for org members (kubeflow#1111) The idea behind this is that they could set the 'ok-to-test' label anyways via prow and it eases the burden of the maintainers having to do this manually for org members. The code is taken from the way that `kubeflow/sdk` does it for their equivalent job. Link: https://github.com/kubeflow/sdk/blob/main/.github/workflows/gh-workflow-approve.yaml Signed-off-by: Christian Heusel <christian@heusel.eu> # This is the commit message kubeflow#9: chore: upgrade setup-go to 6.4.0 (kubeflow#1116) * chore: Bump actions/setup-go from 5.6.0 to 6.4.0 Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.6.0 to 6.4.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@40f1582...4a36011) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.4.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * fix: set GOTOOLCHAIN=local+auto in Makefiles for setup-go compatibility The setup-go GitHub Action sets GOTOOLCHAIN=local, which prevents go-install-tool from downloading tools that require a newer Go version. Override with local+auto in both backend and controller Makefiles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> # This is the commit message kubeflow#10: ci: scaffold e2e testing framework infrastructure (kubeflow#763) * ci: scaffold e2e testing framework infrastructure This commit establishes the foundational infrastructure for end-to-end testing of the workspaces components. While some of the scripts appear redundant with the developing/ directory, a deliberate decision was made to keep this logic mutually exclusive to be more flexible. For instance, the testing/ directory has a need to deploy a Gateway that is not necessary for developing. Keeping everything separate for now makes it easier to evolve independently. Changes include: - Add new `testing/` directory with Makefile and setup scripts: * `setup-kind.sh`: Automated Kind cluster creation and configuration * `setup-cert-manager.sh`: Cert-manager installation (v1.12.13 LTS) * `setup-istio.sh`: Istio service mesh installation with Gateway and TLS certificate provisioning via cert-manager * `check-kind-context.sh`: Safety check to prevent accidental deployment to non-Kind clusters * `sanity-check.sh`: Post-deploy verification including rollout status, TLS handshake (webhook), HTTP health endpoints, and Istio gateway routing for backend and frontend * `gateway.yaml`: Istio Gateway (HTTP + HTTPS) for kubeflow-gateway * `gateway-cert.yaml`: Self-signed ClusterIssuer and Certificate for gateway TLS termination * Makefile targets: setup-cluster, deploy-all, sanity-check, teardown-cluster, clean, and local-e2e (placeholder) - Add GitHub Actions workflow (`.github/workflows/ws-e2e-test.yml`): * Triggers on pushes to main branches and PRs affecting workspaces * Pipeline: setup-cluster -> deploy-all -> sanity-check -> local-e2e Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> * chore: address PR feedback Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> * chore: address PR feedback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Andy Stoneberg <astonebe@redhat.com> * mathew: 1 Signed-off-by: Mathew Wicks <5735406+thesuperzapper@users.noreply.github.com> * mathew: 2 Signed-off-by: Mathew Wicks <5735406+thesuperzapper@users.noreply.github.com> --------- Signed-off-by: Andy Stoneberg <astonebe@redhat.com> Signed-off-by: Mathew Wicks <5735406+thesuperzapper@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Mathew Wicks <5735406+thesuperzapper@users.noreply.github.com> # This is the commit message kubeflow#11: fix: format Go files with gofmt Signed-off-by: Sneha Das <sn2005eha26das@gmail.com> # This is the commit message kubeflow#12: refactor: address reviewer comments, fix inline type duplication and switch tests to gomega framework Signed-off-by: Sneha Das <154408198+Snehadas2005@users.noreply.github.com> # This is the commit message kubeflow#13: fix(test): use http.NoBody and fix import formatting for linter compliance Signed-off-by: Sneha Das <154408198+Snehadas2005@users.noreply.github.com> # This is the commit message kubeflow#14: fix(test): use http.NoBody and fix import formatting for linter compliance Signed-off-by: Sneha Das <154408198+Snehadas2005@users.noreply.github.com>
ℹ️ NO GH ISSUE
This commit establishes the foundational infrastructure for end-to-end testing of the workspaces components. This is an intermediate step that sets up the testing framework; no actual test execution is implemented yet and will be added in subsequent work.
Changes include:
Add new
testing/directory with Makefile and setup scripts:setup-kind.sh: Automated Kind cluster creation and configurationsetup-cert-manager.sh: Cert-manager installation (v1.12.13 LTS)setup-istio.sh: Istio service mesh installationAdd GitHub Actions workflow (
.github/workflows/ws-e2e-test.yml):The framework enables automated deployment of all three components (controller, backend, frontend) to a Kind cluster with cert-manager and Istio pre-configured, providing the foundation for comprehensive e2e test scenarios in future commits.
VERIFICATION
You can see what this looks like when I ran it on my fork: