Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 70 additions & 17 deletions .github/workflows/coverage_runner.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
strategy:
matrix:
os: [ ubuntu-latest, windows-latest ]

hz_version: [ "5.6.0" ]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[] redundant.
Also this upgrades the version from 5.4 -> 5.6 which is good but probably not required for this change.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remove the [] I get a lint error. The matrix schema seems to require it.
We'll need 5.6.0 for the release of the client.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remove the [] I get a lint error.

Ah - I think that's because it's a matrix.
My preference is that:

hz_version:
   - 5.6.0

Is more readable in this case, but that's just a nit.

We'll need 5.6.0 for the release of the client.

It would be nice if that was in the PR body or a seperate PR just in case someones tries to figure out why the behaviour changed in this PR and it appears to be just RCD.

steps:
- name: Checkout code for PR
if: github.event_name == 'pull_request_target'
Expand Down Expand Up @@ -74,27 +74,89 @@ jobs:
with:
node-version: ${{ needs.node-versions.outputs.default-version }}

- name: Checkout to test artifacts
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v5
with:
role-to-assume: ${{ secrets.AWS_HAZELCAST_OIDC_GITHUB_ACTIONS_ROLE_ARN }}
aws-region: 'us-east-1'

- name: Get Secrets
uses: aws-actions/aws-secretsmanager-get-secrets@v2
with:
secret-ids: |
HAZELCAST_ENTERPRISE_KEY,CN/HZ_LICENSE_KEY

- name: Checkout to certificates
uses: actions/checkout@v4
with:
repository: hazelcast/private-test-artifacts
path: certs
ref: data
token: ${{ secrets.GH_TOKEN }}
repository: hazelcast/private-test-artifacts
path: certs
ref: data
token: ${{ secrets.GH_TOKEN }}

- name: Copy certificates JAR to destination with the appropriate name
run: |
cp ${{ github.workspace }}/certs/certs.jar ${{ github.workspace }}/certs.jar
unzip -p ${{ github.workspace }}/certs.jar com/hazelcast/nio/ssl/letsencrypt.jks > test/integration/backward_compatible/parallel/ssl/keystore.jks

- name: Create the test jar with certificates (Linux)
if: matrix.os == 'ubuntu-latest'
working-directory: certs
run: |
zip -r -j certs.jar $GITHUB_WORKSPACE/test/integration/backward_compatible/parallel/ssl/*.pem
cp certs.jar ../hazelcast-enterprise-${{ matrix.hz_version }}-tests.jar

- name: Create the test jar with certificates (Windows)
if: matrix.os == 'windows-latest'
working-directory: certs
run: |
$compress = @{
Path = "../test/integration/backward_compatible/parallel/ssl/*.pem"
CompressionLevel = "Fastest"
DestinationPath = "certs.jar"
}
Compress-Archive -Update @compress
cp certs.jar ../hazelcast-enterprise-${{ matrix.hz_version }}-tests.jar

- name: Download RCD (Linux)
if: matrix.os == 'ubuntu-latest'
shell: bash
run: |
wget -q https://client-rcd-download.s3.us-east-2.amazonaws.com/rcd-ubuntu-latest

- name: Download RCD (Windows)
if: matrix.os == 'windows-latest'
run: |
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest https://client-rcd-download.s3.us-east-2.amazonaws.com/rcd-windows-latest.exe -OutFile rcd-windows-latest.exe

- name: Install dependencies and compile client
run: |
npm install
npm run compile

- name: Run all tests
- name: Run all tests (Linux)
if: matrix.os == 'ubuntu-latest'
env:
HZ_VERSION: ${{ matrix.hz_version }}
run: |
chmod +x rcd-ubuntu-latest
./rcd-ubuntu-latest -version $HZ_VERSION -no-simple-server &
# wait for a bit for RCD to download artifacts
sleep 10
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a reliable check? some downloads last longer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

npm run coverage

- name: Run all tests (Windows)
if: matrix.os == 'windows-latest'
env:
HAZELCAST_ENTERPRISE_KEY: ${{ secrets.HAZELCAST_ENTERPRISE_KEY }}
HZ_VERSION: ${{ matrix.hz_version }}
run: |
Start-Process -FilePath .\rcd-windows-latest -ArgumentList '-version', $Env:HZ_VERSION, '-no-simple-server' -RedirectStandardOutput rcd-stdout.log -RedirectStandardError rcd-stderr.log
# wait for a bit for RCD to download artifacts
sleep 10
echo "RCD Log:"
cat rcd-stdout.log
cat rcd-stderr.log
npm run coverage

- name: Publish results to Codecov for PR coming from hazelcast organization
Expand Down Expand Up @@ -122,12 +184,3 @@ jobs:
files: coverage/lcov.info
override_pr: ${{ github.event.inputs.pr_number }}
fail_ci_if_error: true

- name: Upload remote controller logs if test run fails
uses: actions/upload-artifact@v4
if: failure()
with:
name: rc-logs-${{ matrix.os }}
path: |
rc_stderr.log
rc_stdout.log
1 change: 0 additions & 1 deletion .husky/.gitignore

This file was deleted.

9 changes: 0 additions & 9 deletions .husky/pre-commit

This file was deleted.

82 changes: 0 additions & 82 deletions scripts/download-rc.js

This file was deleted.

108 changes: 2 additions & 106 deletions scripts/test-runner.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,9 @@
'use strict';
const fs = require('fs');
const os = require('os');
const net = require('net');
const {spawnSync, spawn} = require('child_process');
const codeSampleChecker = require('./code-sample-checker');

const {
HAZELCAST_RC_VERSION,
HAZELCAST_TEST_VERSION,
HAZELCAST_ENTERPRISE_VERSION,
HAZELCAST_VERSION,
downloadRC
} = require('./download-rc.js');

const DEV_CLUSTER_CONFIG = `
<hazelcast xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
Expand All @@ -23,8 +14,6 @@ const DEV_CLUSTER_CONFIG = `
</hazelcast>
`;
const ON_WINDOWS = os.platform() === 'win32';
const HAZELCAST_ENTERPRISE_KEY = process.env.HAZELCAST_ENTERPRISE_KEY ? process.env.HAZELCAST_ENTERPRISE_KEY : '';
const PATH_SEPARATOR = ON_WINDOWS ? ';' : ':';

let cluster; // We create a cluster for checking code samples

Expand All @@ -33,97 +22,12 @@ let testType;
let rcProcess;
let testProcess;
let runTests = true;
let CLASSPATH = `hazelcast-remote-controller-${HAZELCAST_RC_VERSION}.jar${PATH_SEPARATOR}`
+ `hazelcast-${HAZELCAST_TEST_VERSION}-tests.jar${PATH_SEPARATOR}`
+ `hazelcast-sql-${HAZELCAST_VERSION}.jar${PATH_SEPARATOR}`;

if (HAZELCAST_ENTERPRISE_KEY) {
CLASSPATH = `hazelcast-enterprise-${HAZELCAST_ENTERPRISE_VERSION}.jar${PATH_SEPARATOR}`
+ `certs.jar${PATH_SEPARATOR}`
+ CLASSPATH;
} else {
CLASSPATH = `hazelcast-${HAZELCAST_VERSION}.jar${PATH_SEPARATOR}${CLASSPATH}`;
}

const isAddressReachable = (host, port, timeoutMs) => {
return new Promise((resolve) => {
const socket = new net.Socket();
socket.setTimeout(timeoutMs);
const onError = () => {
socket.destroy();
resolve(false);
};
socket.once('error', onError);
socket.once('timeout', onError);
socket.connect(port, host, () => {
socket.end();
resolve(true);
});
});
};

// Import lazily to defer side affect of the import (connection attempt to 9701)
const getRC = () => {
return require('../test/integration/RC');
};

const startRC = async () => {
console.log('Starting Hazelcast Remote Controller ...');
if (ON_WINDOWS) {
const outFD = fs.openSync('rc_stdout.log', 'w');
const errFD = fs.openSync('rc_stderr.log', 'w');
rcProcess = spawn('java', [
`-Dhazelcast.enterprise.license.key=${HAZELCAST_ENTERPRISE_KEY}`,
'-cp',
CLASSPATH,
'com.hazelcast.remotecontroller.Main'
], {
stdio: [
'ignore',
outFD,
errFD
]
});
rcProcess.on('close', () => {
fs.closeSync(outFD);
fs.closeSync(errFD);
});
} else {
const outFD = fs.openSync('rc_stdout.log', 'w');
const errFD = fs.openSync('rc_stderr.log', 'w');
rcProcess = spawn('java', [
`-Dhazelcast.enterprise.license.key=${HAZELCAST_ENTERPRISE_KEY}`,
'-cp',
CLASSPATH,
'com.hazelcast.remotecontroller.Main'
], {
stdio: [
'ignore',
outFD,
errFD
]
});
rcProcess.on('close', () => {
fs.closeSync(outFD);
fs.closeSync(errFD);
});
}

console.log('Please wait for Hazelcast Remote Controller to start ...');

const retryCount = 100;

for (let i = 0; i < retryCount; i++) {
console.log('Trying to connect to Hazelcast Remote Controller (127.0.0.1:9701)...');
const addressReachable = await isAddressReachable('127.0.0.1', 9701, 5000);
if (addressReachable) {
return;
}
await new Promise(r => setTimeout(r, 1000));
}
throw `Could not reach to Hazelcast Remote Controller (127.0.0.1:9701) after trying ${retryCount} times.`;
};

const shutdown = async () => {
if (testProcess && testProcess.exitCode === null) {
stopTestProcess();
Expand Down Expand Up @@ -229,19 +133,11 @@ if (testType === 'unit') {
process.exit(subprocess.status);
}

// For other tests, download rc files if needed.
try {
downloadRC();
} catch (err) {
console.log('An error occurred downloading remote controller:');
throw err;
}

process.on('SIGINT', shutdown);
process.on('SIGTERM', shutdown);
process.on('SIGHUP', shutdown);

startRC().then(async () => {
(async () => {
console.log('Hazelcast Remote Controller is started!');
if (runTests) {
console.log(`Running ${testType}, Command: ${testCommand}`);
Expand All @@ -265,7 +161,7 @@ startRC().then(async () => {
process.exit(exitCode);
}
}
}).catch(err => {
})().catch(err => {
console.log('Could not start Hazelcast Remote Controller due to an error:');
throw err;
});
Binary file modified test/integration/backward_compatible/parallel/ssl/keystore.jks
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this file be included? I thought it's resolved from the certs JAR?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's for CI. People cloning the repo should still be able to run the tests.

Binary file not shown.
Loading