Skip to content

test: add pairing logic to toolchain-e2e #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rsoaresd merged 10 commits into from
Dec 16, 2025
Merged

test: add pairing logic to toolchain-e2e #16

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
daf4b34
add pairing logic to toolchain-e2e
rsoaresd Dec 9, 2025
be26196
Merge branch 'master' into pairing_logic_to_toolchain-e2e
rsoaresd Dec 9, 2025
cedc8c8
prettier
rsoaresd Dec 9, 2025
39bf4ba
Merge branch 'master' into pairing_logic_to_toolchain-e2e
rsoaresd Dec 10, 2025
8046aad
improvements
rsoaresd Dec 11, 2025
a6c26c9
improve
rsoaresd Dec 11, 2025
db84024
improve
rsoaresd Dec 11, 2025
50bf95e
improve
rsoaresd Dec 15, 2025
c913c2f
Merge branch 'master' into pairing_logic_to_toolchain-e2e
rsoaresd Dec 16, 2025
785d468
Merge branch 'master' into pairing_logic_to_toolchain-e2e
rsoaresd Dec 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,29 @@ npx backstage-cli versions:bump --release 1.36.1
```

More details on the upgrade process are available [keeping-backstage-updated](https://backstage.io/docs/getting-started/keeping-backstage-updated/)

## Running E2E Tests

The E2E tests live in a separate repository, but you can run them directly from this repo against your current working code.

_Prerequisites_:

1. You need a OCP cluster
- ROSA cluster from ClusterBot will not work since we are not able to modify the OAuth configuration of ROSA clusters created by the ClusterBot.
Copy link
Copy Markdown
Contributor

@mfrancisc mfrancisc Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also CRC didn't worked for me - the rdhd pod was crashloopbacking and not deploying properly ( for some still unknown reason 😞 ) , this was the error in the logs:

Traceback (most recent call last):
  File "/opt/app-root/src/install-dynamic-plugins.py", line 523, in <module>
    main()
  File "/opt/app-root/src/install-dynamic-plugins.py", line 405, in main
    raise InstallException(f"Error while adding OCI plugin {package} to downloader: {e}")
InstallException: Error while adding OCI plugin oci://quay.io/fmuntean/sandbox-rhdh-plugin:27174259!red-hat-developer-hub-backstage-plugin-sandbox to downloader: Command '['/usr/bin/skopeo', 'copy', 'docker://quay.io/fmuntean/sandbox-rhdh-plugin:27174259', 'dir:/tmp/tmpp_ugusz8/150cc6c226e9b94f9ca76b7308907c1b64b83718596fb4cb230d9e94d70384f1']' died with <Signals.SIGABRT: 6>.

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's strange because I was able to pull the image 🤨 Maybe we could contact rhdh team, what do you think? TBH, I do not use CRC

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems all good:

rsoaresd-mac:devsandbox-dashboard rsoaresd$ skopeo inspect docker://quay.io/fmuntean/sandbox-rhdh-plugin:27174259

{
    "Name": "quay.io/fmuntean/sandbox-rhdh-plugin",
    "Digest": "sha256:535f668fd81618d785e2e758543afd37ebcff6982904b724f994deab1cdf4ffc",
    "RepoTags": [
        "24192715",
        "27172521",
        "27174259",
        "28094436",
        "28100141",
        "28102512",
        "latest"
    ],
    "Created": "2025-11-27T16:43:13.528528033Z",
    "DockerVersion": "",
    "Labels": {
        "io.buildah.version": "1.42.0"
    },
    "Architecture": "arm64",
    "Os": "linux",
    "Layers": [
        "sha256:b1e527af46d144ade39a07127009a2fc2778a6f1d727c874fa94c164e6be2912"
    ],
    "LayersData": [
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "Digest": "sha256:b1e527af46d144ade39a07127009a2fc2778a6f1d727c874fa94c164e6be2912",
            "Size": 4476720,
            "Annotations": null
        }
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ]
}
rsoaresd-mac:devsandbox-dashboard rsoaresd$ 
rsoaresd-mac:devsandbox-dashboard rsoaresd$ skopeo copy docker://quay.io/fmuntean/sandbox-rhdh-plugin:27174259 dir:/tmp/test-dir
Getting image source signatures
Copying blob b1e527af46d1 done   | 
Copying config 0a900427ab done   | 
Writing manifest to image destination
rsoaresd-mac:devsandbox-dashboard rsoaresd$

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P.S. Going to try with CRC

Copy link
Copy Markdown
Contributor

@mfrancisc mfrancisc Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah it just fails when the rhdh installer tries to pull it.

If I try to pull it with skopeo on my Mac it works.

2. Ensure you are using Node.js version 22
- to easily manage it, you can run `nvm use 22`
3. Ensure you have `yarn` installed
4. Make sure you can log in at <https://sso.devsandbox.dev/auth/realms/sandbox-dev/account> using your `SSO_USERNAME` and `SSO_PASSWORD`. You can contact the [Developer Sandbox team](devsandbox@redhat.com) to obtain the test user credentials.
5. It's required that you create a repository called `sandbox-rhdh-plugin` in your quay organization and make it public

The following Makefile targets are available:

- `make test-e2e` - this target clones latest changes from [toolchain-e2e](https://github.com/codeready-toolchain/toolchain-e2e) and runs e2e tests. As deployment for `devsandbox-dashboard` it uses the current code that is at HEAD.
- `make test-e2e-local` - this target doesn't clone anything, but it runs e2e tests from the directory `../toolchain-e2e`. As deployment for `devsandbox-dashboard` it uses the current code that is at HEAD.
- `make test-e2e-in-container` — runs e2e tests inside a container; clones latest changes from [toolchain-e2e](https://github.com/codeready-toolchain/toolchain-e2e).
- `make test-e2e-local-in-container` — runs e2e tests inside a container from the directory `../toolchain-e2e`.

Note that you need to set `SSO_USERNAME` and `SSO_PASSWORD`. For example:
`make test-e2e SSO_USERNAME=${SSO_USERNAME} SSO_PASSWORD=${SSO_PASSWORD}`

For more detailed information, check [toolchain-e2e documentation](https://github.com/codeready-toolchain/toolchain-e2e/blob/master/test/e2e/sandbox-ui/README.md).
74 changes: 74 additions & 0 deletions make/test.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
###########################################################
Copy link
Copy Markdown
Contributor

@mfrancisc mfrancisc Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one thing I'm missing is the IMAGE_PLATFORM which drives the build of the tests and/or the build of the deployment images.

Is that being exposed somewhere else ?

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, here: https://github.com/codeready-toolchain/devsandbox-dashboard/blob/master/make/podman.mk#L6

Copy link
Copy Markdown
Contributor

@mfrancisc mfrancisc Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! But I guess that will not be passed to the test-e2e targets ? Maybe I'm wrong.

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. test-e2e will call manage-devsandbox-dashboard script: https://github.com/codeready-toolchain/toolchain-e2e/blob/master/make/devsandbox-dashboard.mk#L29

  2. By its turn, the script will call the podman-push in https://github.com/codeready-toolchain/devsandbox-dashboard/blob/master/make/podman.mk#L6

Copy link
Copy Markdown
Contributor

@mfrancisc mfrancisc Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, thanks for explaining. I'll try it out later on 👍

#
# End-to-end Tests
#
###########################################################

E2E_REPO_PATH := ""
PUBLISH_UI ?= false
DEPLOY_UI ?= true

.PHONY: test-e2e-local
test-e2e-local:
$(MAKE) test-e2e E2E_REPO_PATH=${PWD}/../toolchain-e2e PUBLISH_UI=true DEPLOY_UI=true

.PHONY: publish-current-bundles-for-e2e
publish-current-bundles-for-e2e: get-e2e-repo
# build & publish the bundles via toolchain-e2e repo
$(MAKE) -C ${E2E_REPO_PATH} publish-current-bundles-for-e2e UI_REPO_PATH=${PWD}

.PHONY: test-e2e
test-e2e: get-e2e-repo
# run the e2e test via toolchain-e2e repo
$(MAKE) -C ${E2E_REPO_PATH} prepare-and-deploy-e2e test-devsandbox-dashboard-e2e UI_REPO_PATH=${PWD} PUBLISH_UI=${PUBLISH_UI} DEPLOY_UI=${DEPLOY_UI}
Copy link
Copy Markdown
Contributor

@MatousJobanek MatousJobanek Dec 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused, why the test-e2e-local runs test-e2e while the test-e2e runs the two targets prepare-and-deploy-e2e test-devsandbox-dashboard-e2e - shouldn't it be consistent?

Also, I'm a bit confused with the mix of the usages of the PUBLISH_UI env var - why is it in she second case set to default which is false?

I'm comparing it with the other repos where we have the same logic and it's not matching.

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused, why the test-e2e-local runs test-e2e while the test-e2e runs the two targets prepare-and-

test-e2e-local runs test-e2e that it is defined in file test.mk in this repo, not the one defined in toolchain-e2e
test-e2e runs the two targets prepare-and-deploy-e2e test-devsandbox-dashboard-e2e because I think we do not want to run the operators e2e tests in this repo, right?

Also, I'm a bit confused with the mix of the usages of the PUBLISH_UI env var - why is it in she second case set to default which is false?

I set them on that file for being clear which are the default values (if not present there, the person who is looking for the code needs to search in toolchain-e2e to figure it out the default values). But I can just mention in a comment, I will change it, thanks!

Copy link
Copy Markdown
Contributor Author

@rsoaresd rsoaresd Dec 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MatousJobanek, afterall, the vars config was needed for calling test-e2e (if not set, that vars will be empty) and the run will fail, example: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/72470/rehearse-72470-pull-ci-codeready-toolchain-devsandbox-dashboard-pairing_logic_to_toolchain-e2e-e2e/1999129264741421056

So, I will drop all the config of that vars in this repo (a6c26c9), since we can simply to do this in toolchain-e2e side: codeready-toolchain/toolchain-e2e#1233

Copy link
Copy Markdown
Contributor

@MatousJobanek MatousJobanek Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test-e2e-local runs test-e2e that it is defined in file test.mk in this repo, not the one defined in toolchain-e2e

stupid me 🤦‍♂️

test-e2e runs the two targets prepare-and-deploy-e2e test-devsandbox-dashboard-e2e because I think we do not want to run the operators e2e tests in this repo, right?

ah, right. yeah, that makes sense 👍


.PHONY: test-e2e-local-in-container
test-e2e-local-in-container:
# run the e2e test via toolchain-e2e repo
$(MAKE) test-e2e-in-container E2E_REPO_PATH=${PWD}/../toolchain-e2e

.PHONY: test-e2e-in-container
test-e2e-in-container: get-e2e-repo
# run the e2e test via toolchain-e2e repo
$(MAKE) -C ${E2E_REPO_PATH} prepare-and-deploy-e2e test-devsandbox-dashboard-in-container UI_REPO_PATH=${PWD}

.PHONY: clean-e2e
clean-e2e: get-e2e-repo
$(MAKE) -C ${E2E_REPO_PATH} clean-devsandbox-dashboard

.PHONY: get-e2e-repo
get-e2e-repo:
ifeq ($(E2E_REPO_PATH),"")
# set e2e repo path to tmp directory
$(eval E2E_REPO_PATH = /tmp/toolchain-e2e)
# delete to have clear environment
rm -rf ${E2E_REPO_PATH}
# clone
git clone https://github.com/codeready-toolchain/toolchain-e2e.git ${E2E_REPO_PATH}
ifneq ($(CI),)
ifneq ($(GITHUB_ACTIONS),)
$(eval BRANCH_NAME = ${GITHUB_HEAD_REF})
$(eval AUTHOR_LINK = https://github.com/${AUTHOR})
else
$(eval AUTHOR_LINK = $(shell jq -r '.refs[0].pulls[0].author_link' <<< $${CLONEREFS_OPTIONS} | tr -d '[:space:]'))
@echo "found author link ${AUTHOR_LINK}"
$(eval BRANCH_NAME := $(shell jq -r '.refs[0].pulls[0].head_ref' <<< $${CLONEREFS_OPTIONS} | tr -d '[:space:]'))
endif
@echo "using author link ${AUTHOR_LINK}"
@echo "detected branch ${BRANCH_NAME}"
# check if a branch with the same ref exists in the user's fork of toolchain-e2e repo
$(eval REMOTE_E2E_BRANCH := $(shell curl ${AUTHOR_LINK}/toolchain-e2e.git/info/refs?service=git-upload-pack --output - 2>/dev/null | grep -a "refs/heads/${BRANCH_NAME}$$" | awk '{print $$2}'))
@echo "branch ref of the user's fork: \"${REMOTE_E2E_BRANCH}\" - if empty then not found"
# check if the branch with the same name exists, if so then merge it with master and use the merge branch, if not then use master
if [[ -n "${REMOTE_E2E_BRANCH}" ]]; then \
git config --global user.email "devsandbox@redhat.com"; \
git config --global user.name "KubeSaw"; \
# add the user's fork as remote repo \
git --git-dir=${E2E_REPO_PATH}/.git --work-tree=${E2E_REPO_PATH} remote add external ${AUTHOR_LINK}/toolchain-e2e.git; \
# fetch the branch \
git --git-dir=${E2E_REPO_PATH}/.git --work-tree=${E2E_REPO_PATH} fetch external ${REMOTE_E2E_BRANCH}; \
# merge the branch with master \
git --git-dir=${E2E_REPO_PATH}/.git --work-tree=${E2E_REPO_PATH} merge --allow-unrelated-histories --no-commit FETCH_HEAD; \
fi;
endif
endif