Skip to content

Commit 9b8b437

Browse files
dtfranzdtfranz
authored
🌱 E2E Cleanup (#2704)
* E2E Cleanup - Fix undefined steps causing HTTPProxy tests to be skipped - Set "Strict" flag for godog by default to avoid undefined steps slipping by in the future - Fix concurrent test run requiring two ctrl-c's to terminate - Use parentheses instead of brackets in go test calls for E2E_TIMEOUT for consistency/correctness Signed-off-by: Daniel Franz <dfranz@redhat.com> * Fix ambiguous step definitions Signed-off-by: Daniel Franz <dfranz@redhat.com> --------- Signed-off-by: Daniel Franz <dfranz@redhat.com>
1 parent f120442 commit 9b8b437

8 files changed

Lines changed: 24 additions & 14 deletions

File tree

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,11 @@ cover.out
2626
/install.sh
2727
/install-experimental.sh
2828

29+
# upgrade-e2e artifacts
30+
/experimental-install.sh
31+
/operator-controller-standard.yaml
32+
/standard-install.sh
33+
2934
# vendored files
3035
vendor/
3136

‎Makefile‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -259,10 +259,11 @@ e2e: E2E_TIMEOUT ?= 20m
259259
e2e: GODOG_ARGS ?=
260260
e2e: #EXHELP Run the e2e tests.
261261
ifeq ($(strip $(GODOG_ARGS)),)
262+
trap 'exit 130' INT; \
262263
set +e; \
263-
go test -count=1 -v ./test/e2e/features_test.go -timeout=${E2E_TIMEOUT} -args --godog.tags="~@Serial" --godog.concurrency=100; \
264+
go test -count=1 -v ./test/e2e/features_test.go -timeout $(E2E_TIMEOUT) -args --godog.tags="~@Serial" --godog.concurrency=100; \
264265
parallelExit=$$?; \
265-
go test -count=1 -v ./test/e2e/features_test.go -timeout=${E2E_TIMEOUT} -args --godog.tags="@Serial" --godog.concurrency=1; \
266+
go test -count=1 -v ./test/e2e/features_test.go -timeout $(E2E_TIMEOUT) -args --godog.tags="@Serial" --godog.concurrency=1; \
266267
serialExit=$$?; \
267268
if [[ $$parallelExit -ne 0 ]] || [[ $$serialExit -ne 0 ]]; then \
268269
echo "e2e tests failed: parallel=$$parallelExit serial=$$serialExit"; \

‎test/e2e/features/proxy.feature‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,9 @@ Feature: HTTPS proxy support for outbound catalog requests
77

88
Background:
99
Given OLM is available
10-
And ClusterCatalog "test" serves bundles
10+
And a catalog "test" with packages:
11+
| package | version | channel | replaces | contents |
12+
| test | 1.2.0 | beta | | CRD, Deployment, ConfigMap |
1113
And ServiceAccount "olm-sa" with needed permissions is available in test namespace
1214

1315
@HTTPProxy
@@ -26,10 +28,10 @@ Feature: HTTPS proxy support for outbound catalog requests
2628
source:
2729
sourceType: Catalog
2830
catalog:
29-
packageName: test
31+
packageName: ${PACKAGE:test}
3032
selector:
3133
matchLabels:
32-
"olm.operatorframework.io/metadata.name": test-catalog
34+
"olm.operatorframework.io/metadata.name": ${CATALOG:test}
3335
"""
3436
Then ClusterExtension reports Progressing as True with Reason Retrying and Message includes:
3537
"""
@@ -57,9 +59,9 @@ Feature: HTTPS proxy support for outbound catalog requests
5759
source:
5860
sourceType: Catalog
5961
catalog:
60-
packageName: test
62+
packageName: ${PACKAGE:test}
6163
selector:
6264
matchLabels:
63-
"olm.operatorframework.io/metadata.name": test-catalog
65+
"olm.operatorframework.io/metadata.name": ${CATALOG:test}
6466
"""
6567
Then the recording proxy received a CONNECT request for the catalogd service

‎test/e2e/features/update.feature‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Feature: Update ClusterExtension
3838
"""
3939
And ClusterExtension is rolled out
4040
And ClusterExtension is available
41-
When ClusterExtension is updated to version "1.0.1"
41+
When ClusterExtension version is updated to "1.0.1"
4242
Then ClusterExtension is rolled out
4343
And ClusterExtension is available
4444
And bundle "${PACKAGE:test}.1.0.1" is installed in version "1.0.1"
@@ -219,7 +219,7 @@ Feature: Update ClusterExtension
219219
And ClusterExtension is rolled out
220220
And ClusterExtension is available
221221
And bundle "${PACKAGE:test}.1.0.0" is installed in version "1.0.0"
222-
When ClusterExtension is updated to version "1.0.4"
222+
When ClusterExtension version is updated to "1.0.4"
223223
Then ClusterExtension is rolled out
224224
And ClusterExtension is available
225225
And bundle "${PACKAGE:test}.1.0.4" is installed in version "1.0.4"
@@ -247,7 +247,7 @@ Feature: Update ClusterExtension
247247
"""
248248
And ClusterExtension is rolled out
249249
And ClusterExtension is available
250-
When ClusterExtension is updated to version "1.0.1"
250+
When ClusterExtension version is updated to "1.0.1"
251251
Then ClusterExtension is rolled out
252252
And ClusterExtension is available
253253
And bundle "${PACKAGE:test}.1.0.1" is installed in version "1.0.1"
@@ -300,7 +300,7 @@ Feature: Update ClusterExtension
300300
"""
301301
And ClusterExtension is rolled out
302302
And ClusterExtension is available
303-
When ClusterExtension is updated to version "1.2.0"
303+
When ClusterExtension version is updated to "1.2.0"
304304
Then bundle "${PACKAGE:test}.1.2.0" is installed in version "1.2.0"
305305
And ClusterExtension is rolled out
306306
And ClusterExtension is available
@@ -334,7 +334,7 @@ Feature: Update ClusterExtension
334334
"""
335335
And ClusterExtension is rolled out
336336
And ClusterExtension is available
337-
When ClusterExtension is updated to version "1.0.2"
337+
When ClusterExtension version is updated to "1.0.2"
338338
Then ClusterExtension reports "${NAME}-1, ${NAME}-2" as active revisions
339339
And ClusterObjectSet "${NAME}-2" reports Progressing as True with Reason RollingOut
340340
And ClusterObjectSet "${NAME}-2" reports Available as False with Reason ProbeFailure

‎test/e2e/features_test.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ var opts = godog.Options{
2020
Output: colors.Colored(os.Stdout),
2121
Concurrency: 1,
2222
NoColors: true,
23+
Strict: true,
2324
}
2425

2526
func init() {

‎test/e2e/steps/steps.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,7 @@ func RegisterSteps(sc *godog.ScenarioContext) {
9999
sc.Step(`^(?i)bundle "([^"]+)" is installed in version "([^"]+)"$`, BundleInstalled)
100100

101101
sc.Step(`^(?i)ClusterExtension is applied(?:\s+.*)?$`, ResourceIsApplied)
102-
sc.Step(`^(?i)ClusterExtension is updated to version "([^"]+)"$`, ClusterExtensionVersionUpdate)
102+
sc.Step(`^(?i)ClusterExtension version is updated to "([^"]+)"$`, ClusterExtensionVersionUpdate)
103103
sc.Step(`^(?i)ClusterExtension is updated(?:\s+.*)?$`, ResourceIsApplied)
104104
sc.Step(`^(?i)ClusterObjectSet "([^"]+)" lifecycle is set to "([^"]+)"$`, ClusterObjectSetLifecycleUpdate)
105105
sc.Step(`^(?i)ClusterExtension is available$`, ClusterExtensionIsAvailable)

‎test/upgrade-e2e/features/operator-upgrade.feature‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,6 @@ Feature: Operator upgrade verification
4343

4444
Scenario: ClusterExtension remains functional after OLM upgrade
4545
Given ClusterExtension is reconciled
46-
When ClusterExtension is updated to version "1.0.1"
46+
When ClusterExtension version is updated to "1.0.1"
4747
Then ClusterExtension is available
4848
And bundle "${PACKAGE:test}.1.0.1" is installed in version "1.0.1"

‎test/upgrade-e2e/upgrade_test.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ var opts = godog.Options{
1818
Output: colors.Colored(os.Stdout),
1919
Concurrency: 1,
2020
NoColors: true,
21+
Strict: true,
2122
}
2223

2324
func init() {

0 commit comments

Comments
 (0)