Skip to content
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
1ab2f6b
chore: add resolutions to smoke tests
AlbinaBlazhko17 Apr 22, 2026
a4fe0f6
chore: add overrides
AlbinaBlazhko17 Apr 22, 2026
7396cdd
chore: add pnpm overrides
AlbinaBlazhko17 Apr 22, 2026
cb57a7c
feat: add scipt to pull current installed version of config
AlbinaBlazhko17 Apr 22, 2026
a120ae3
chore: move script into separate file
AlbinaBlazhko17 Apr 22, 2026
3e06e62
chore: remove pnpm overrides
AlbinaBlazhko17 Apr 22, 2026
e7827c6
feat: add different fix to pack config from node_modules
AlbinaBlazhko17 Apr 22, 2026
da20af1
feat: migrate from yarn add to yarn install
AlbinaBlazhko17 Apr 22, 2026
1dcd7fb
chore: fix config version script
AlbinaBlazhko17 Apr 22, 2026
9fff599
fix: failing smoke tests
AlbinaBlazhko17 Apr 22, 2026
2771801
chore: remove packing config
AlbinaBlazhko17 Apr 22, 2026
3c5609b
chore: remove --no-lockfile arg
AlbinaBlazhko17 Apr 23, 2026
daa1c70
Merge branch 'main' into chore/fix-flaky-smoke-tests
AlbinaBlazhko17 Apr 23, 2026
64da1cd
chore: remove letter
AlbinaBlazhko17 Apr 23, 2026
0e709bc
feat: add to npm tests the same retry pattern as for yarn
AlbinaBlazhko17 Apr 23, 2026
66f5c91
chore: replace with the main
AlbinaBlazhko17 Apr 23, 2026
8ff38ce
feat: add pining versions for config and ajv and use only for failed …
AlbinaBlazhko17 Apr 23, 2026
54aa739
chore: remove letter
AlbinaBlazhko17 Apr 23, 2026
df6eff9
feat: add per job invocation
AlbinaBlazhko17 Apr 23, 2026
af87090
feat: add missing pin
AlbinaBlazhko17 Apr 23, 2026
b523925
feat: add pining to failed smoke
AlbinaBlazhko17 Apr 23, 2026
41e177e
feat: add to failed smoke test pinning
AlbinaBlazhko17 Apr 23, 2026
f7c6af9
Merge branch 'main' into chore/fix-flaky-smoke-tests
AlbinaBlazhko17 Apr 23, 2026
388d798
feat: add pining to the failed smoke test
AlbinaBlazhko17 Apr 23, 2026
bde52ed
feat: remove workaround to test if it works
AlbinaBlazhko17 Apr 23, 2026
801cb4a
feat: add to missing smoke test retry pattern
AlbinaBlazhko17 Apr 23, 2026
82104c8
chore: remove unused overrides
AlbinaBlazhko17 Apr 23, 2026
9fa6b3c
feat: add script to add redoc in packages and use for specific steps
AlbinaBlazhko17 Apr 24, 2026
d39f863
chore: remove eval
AlbinaBlazhko17 Apr 24, 2026
92985f2
Merge branch 'main' into chore/fix-flaky-smoke-tests
AlbinaBlazhko17 Apr 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions .github/workflows/smoke.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 24
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install" "npm run"

run-smoke--npm--node-24--redoc:
needs: prepare-smoke
Expand All @@ -53,7 +53,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 24
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redoc redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install && npm i redoc" "npm run"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't make any difference but the intention becomes less clear.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to use yarn install instead of yarn add to fix the issue, so i added into package.json dependency, which points to the tgz cli file. I changed all tests to use just install and be consistent across the tests.


run-smoke--yarn--node-24:
needs: prepare-smoke
Expand All @@ -70,7 +70,7 @@ jobs:
run: yarn cache clean
- run: |
for i in {1..2}; do # workaround for yarn cache issue
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn add ./redocly-cli.tgz" "yarn" && break
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn install" "yarn" && break
Comment thread
cursor[bot] marked this conversation as resolved.
done

run-smoke--yarn--node-24--redoc:
Expand All @@ -88,7 +88,7 @@ jobs:
run: yarn cache clean
- run: |
for i in {1..2}; do # workaround for yarn cache issue
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn add redoc ./redocly-cli.tgz" "yarn" && break
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn install && yarn add redoc" "yarn" && break
done
run-smoke--pnpm--node-24:
needs: prepare-smoke
Expand All @@ -105,7 +105,7 @@ jobs:
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10
- run: bash ./tests/smoke/basic/run-smoke.sh "pnpm add ./redocly-cli.tgz" "pnpm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "pnpm install" "pnpm run"

# Skipped due to a known Windows + NodeJS v24 issue: https://github.com/nodejs/node/issues/56645
# run-smoke--npm--node-24--windows:
Expand Down Expand Up @@ -158,7 +158,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install" "npm run"

run-smoke--npm--node-22--redoc:
needs: prepare-smoke
Expand All @@ -171,7 +171,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redoc redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install && npm i redoc" "npm run"

run-smoke--yarn--node-22:
needs: prepare-smoke
Expand All @@ -188,7 +188,7 @@ jobs:
run: yarn cache clean
- run: |
for i in {1..2}; do # workaround for yarn cache issue
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn add ./redocly-cli.tgz" "yarn" && break
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn installe" "yarn" && break
Comment thread
cursor[bot] marked this conversation as resolved.
Outdated
done

run-smoke--yarn--node-22--redoc:
Expand All @@ -206,7 +206,7 @@ jobs:
run: yarn cache clean
- run: |
for i in {1..2}; do # workaround for yarn cache issue
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn add redoc ./redocly-cli.tgz" "yarn" && break
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn install && yarn add redoc" "yarn" && break
Comment thread
cursor[bot] marked this conversation as resolved.
Outdated
done
run-smoke--pnpm--node-22:
needs: prepare-smoke
Expand All @@ -223,7 +223,7 @@ jobs:
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
with:
version: 10
- run: bash ./tests/smoke/basic/run-smoke.sh "pnpm add ./redocly-cli.tgz" "pnpm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "pnpm install" "pnpm run"

run-smoke--npm--node-22--windows:
needs: prepare-smoke
Expand All @@ -237,7 +237,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install" "npm run"

run-smoke--yarn--node-22--windows:
needs: prepare-smoke
Expand All @@ -260,7 +260,7 @@ jobs:
Start-Sleep -Seconds 20 # workaround for yarn cache issue
for ($i = 1; $i -le 3; $i++) {
try {
bash ./tests/smoke/basic/run-smoke.sh "yarn add ./redocly-cli.tgz --no-lockfile --network-timeout 100000" "yarn"
bash ./tests/smoke/basic/run-smoke.sh "yarn install --network-timeout 100000" "yarn"
if ($LASTEXITCODE -eq 0) { break }
} catch {
if ($i -eq 3) { throw }
Expand All @@ -279,7 +279,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 20
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install" "npm run"

run-smoke--npm--node-20--redoc:
needs: prepare-smoke
Expand All @@ -292,7 +292,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 20
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redoc redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install && npm i redoc" "npm run"

run-smoke--yarn--node-20:
needs: prepare-smoke
Expand All @@ -309,7 +309,7 @@ jobs:
run: yarn cache clean
- run: |
for i in {1..2}; do # workaround for yarn cache issue
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn add ./redocly-cli.tgz" "yarn" && break
sleep 5 && bash ./tests/smoke/basic/run-smoke.sh "yarn install" "yarn" && break
done

run-smoke--npm--node-20--windows:
Expand All @@ -324,7 +324,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 20
- run: bash ./tests/smoke/basic/run-smoke.sh "npm i redocly-cli.tgz" "npm run"
- run: bash ./tests/smoke/basic/run-smoke.sh "npm install" "npm run"

run-smoke--yarn--node-20--windows:
needs: prepare-smoke
Expand All @@ -347,7 +347,7 @@ jobs:
Start-Sleep -Seconds 20 # workaround for yarn cache issue
for ($i = 1; $i -le 3; $i++) {
try {
bash ./tests/smoke/basic/run-smoke.sh "yarn add ./redocly-cli.tgz --no-lockfile --network-timeout 100000" "yarn"
bash ./tests/smoke/basic/run-smoke.sh "yarn install --network-timeout 100000" "yarn"
if ($LASTEXITCODE -eq 0) { break }
} catch {
if ($i -eq 3) { throw }
Expand Down
5 changes: 4 additions & 1 deletion tests/smoke/basic/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,8 @@
"redocly-respect": "redocly respect museum-tickets.arazzo.yaml"
},
"author": "redocly",
"license": "ISC"
"license": "ISC",
"dependencies": {
"@redocly/cli": "file:./redocly-cli.tgz"
}
}
16 changes: 16 additions & 0 deletions tests/smoke/basic/pin-config-version.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

REDOCLY_CONFIG_VERSION=$(node -e "const fs = require('fs'); const p = JSON.parse(fs.readFileSync('./packages/core/package.json', 'utf8')); console.log(p.dependencies['@redocly/config'].replace(/\^/, ''));")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're emulating something that will not be the case in our users' setup, so the smokes become irrelevant. Let's try keeping them as much close to the reality as possible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bash script read the version of @redocly/config from openapi-core and write this version to package.json of smoke tests to override this dep to be sure, that all tests will resolve proper version of @redocly/config. Users will resolve the same version of @redocly/config. My first try to fix this issue is manually adding this properties to the package.json, but we can forget to change it there, so i decided to create a script.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss this offine.


echo "Pinning @redocly/config to $REDOCLY_CONFIG_VERSION in smoke test overrides"

node -e "
const fs = require('fs');
const path = './tests/smoke/basic/package.json';
const pkg = JSON.parse(fs.readFileSync(path, 'utf8'));
(pkg.resolutions ??= {})['@redocly/config'] = '$REDOCLY_CONFIG_VERSION';
(pkg.overrides ??= {})['@redocly/config'] = '$REDOCLY_CONFIG_VERSION';
((pkg.pnpm ??= {}).overrides ??= {})['@redocly/config'] = '$REDOCLY_CONFIG_VERSION';

fs.writeFileSync(path, JSON.stringify(pkg, null, 2) + '\n');
"
2 changes: 2 additions & 0 deletions tests/smoke/basic/prepare-smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ npm run pack:prepare

cp ./redocly-cli.tgz ./openapi-core.tgz ./respect-core.tgz resources/pets.yaml resources/museum.yaml resources/museum-tickets.arazzo.yaml ./tests/smoke/basic/

bash ./tests/smoke/basic/pin-config-version.sh

echo "Current directory:"
pwd
echo
Expand Down
4 changes: 2 additions & 2 deletions tests/smoke/basic/run-smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ echo "Directory content:"
ls -a
echo

# Executing the command provided as the first argument
$1
# Executing the command provided as the first argument
eval "$1"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this change, smoke tests became unstable, because we are using && operator to install redoc dep: yarn install && yarn add redoc. Using eval i can support this operator

Comment thread
AlbinaBlazhko17 marked this conversation as resolved.
Outdated

# Actual smoke test - executing the command provided as the second argument
$2 redocly-version
Expand Down
Loading