Skip to content

Fix intermittent test: TestNotifier/MultipleSubscribersStress#865

Merged
brandur merged 1 commit into
masterfrom
brandur-fix-notifier-stress-test
Apr 28, 2025
Merged

Fix intermittent test: TestNotifier/MultipleSubscribersStress#865
brandur merged 1 commit into
masterfrom
brandur-fix-notifier-stress-test

Conversation

@brandur
Copy link
Copy Markdown
Contributor

@brandur brandur commented Apr 26, 2025

Try to fix one of the most commonly failing intermittent tests we have
left, which is the stress test in the notifier [1].

--- FAIL: TestNotifier (0.00s)
    --- FAIL: TestNotifier/MultipleSubscribersStress (0.98s)
        notifier_test.go:409: Generated schema "notifier_2025_04_26t20_08_50_schema_04" with migrations [1 2 3 4 5 6] on line "main" in 376.579906ms [4 generated] [0 reused]
        notifier_test.go:434: Sending notification on "test_topic1": msg0
        notifier_test.go:434: Sending notification on "test_topic1": msg1
        notifier_test.go:434: Sending notification on "test_topic1": msg2
        notifier_test.go:434: Sending notification on "test_topic1": msg3
        notifier_test.go:434: Sending notification on "test_topic1": msg4
        notifier_test.go:434: Sending notification on "test_topic1": msg5
        notifier_test.go:434: Sending notification on "test_topic1": msg6
        notifier_test.go:472: Channel  0 contains   5 message(s)
        notifier_test.go:472: Channel  1 contains   7 message(s)
        notifier_test.go:472: Channel  2 contains   0 message(s)
        notifier_test.go:477:
                Error Trace:	/home/runner/work/river/river/internal/notifier/notifier_test.go:477
                Error:      	Should NOT be empty, but was 0xc000244770
                Test:       	TestNotifier/MultipleSubscribersStress
        riverdbtest.go:277: Checked in schema "notifier_2025_04_26t20_08_50_schema_04"; 1 idle schema(s) [4 generated] [2 reused]
FAIL
FAIL	github.com/riverqueue/river/internal/notifier	2.025s

Rebuild the test a little so that each listen goroutine waits for at
least one message to come through instead of only an arbitrary sleep.
The sleep ~always passes locally, but it seems to be that in CI it's
possible for the sending goroutine to be entirely parked around when a
listen/unlisten loop is happening.

[1] https://github.com/riverqueue/river/actions/runs/14684674639/job/41211780010

Try to fix one of the most commonly failing intermittent tests we have
left, which is the stress test in the notifier [1].

    --- FAIL: TestNotifier (0.00s)
        --- FAIL: TestNotifier/MultipleSubscribersStress (0.98s)
            notifier_test.go:409: Generated schema "notifier_2025_04_26t20_08_50_schema_04" with migrations [1 2 3 4 5 6] on line "main" in 376.579906ms [4 generated] [0 reused]
            notifier_test.go:434: Sending notification on "test_topic1": msg0
            notifier_test.go:434: Sending notification on "test_topic1": msg1
            notifier_test.go:434: Sending notification on "test_topic1": msg2
            notifier_test.go:434: Sending notification on "test_topic1": msg3
            notifier_test.go:434: Sending notification on "test_topic1": msg4
            notifier_test.go:434: Sending notification on "test_topic1": msg5
            notifier_test.go:434: Sending notification on "test_topic1": msg6
            notifier_test.go:472: Channel  0 contains   5 message(s)
            notifier_test.go:472: Channel  1 contains   7 message(s)
            notifier_test.go:472: Channel  2 contains   0 message(s)
            notifier_test.go:477:
                    Error Trace:	/home/runner/work/river/river/internal/notifier/notifier_test.go:477
                    Error:      	Should NOT be empty, but was 0xc000244770
                    Test:       	TestNotifier/MultipleSubscribersStress
            riverdbtest.go:277: Checked in schema "notifier_2025_04_26t20_08_50_schema_04"; 1 idle schema(s) [4 generated] [2 reused]
    FAIL
    FAIL	github.com/riverqueue/river/internal/notifier	2.025s

Rebuild the test a little so that each listen goroutine waits for at
least one message to come through instead of only an arbitrary sleep.
The sleep ~always passes locally, but it seems to be that in CI it's
possible for the sending goroutine to be entirely parked around when a
listen/unlisten loop is happening.

[1] https://github.com/riverqueue/river/actions/runs/14684674639/job/41211780010
@brandur brandur requested a review from bgentry April 26, 2025 21:42
@brandur
Copy link
Copy Markdown
Contributor Author

brandur commented Apr 28, 2025

Thanks!

@brandur brandur merged commit 6af6deb into master Apr 28, 2025
39 of 40 checks passed
@brandur brandur deleted the brandur-fix-notifier-stress-test branch April 28, 2025 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants