Skip to content

test: functional test config and soak test#135

Merged
corpo-iwillspeak merged 2 commits into
mainfrom
test/functional-test-config-and-soak
Jun 26, 2026
Merged

test: functional test config and soak test#135
corpo-iwillspeak merged 2 commits into
mainfrom
test/functional-test-config-and-soak

Conversation

@corpo-iwillspeak

@corpo-iwillspeak corpo-iwillspeak commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Removes hardcoded defaults for ATHENA_HOST, OAUTH_AUTH_URL, and OAUTH_AUDIENCE in functional test config — these now fall back to SDK defaults when unset, rather than overriding with stale hard-coded values
  • Adds proactive_refresh_threshold=0.25 to CredentialHelper construction so tokens refresh before expiry during long test runs
  • Adds a new test_classify_single_soak.py that sends 1000 classify_single requests and asserts a ≥95% KnownCSAM- hash-check classification success rate

Test plan

  • Run functional tests against a live environment to confirm existing behaviour is unchanged
  • Run the new soak test (pytest -m soak) to validate the success-rate assertion

🤖 Generated with Claude Code

…oak test

Env vars (ATHENA_HOST, OAUTH_AUTH_URL, OAUTH_AUDIENCE) now fall back to their
SDK defaults when unset rather than overriding with stale hard-coded values.
Adds proactive_refresh_threshold to CredentialHelper and a new classify_single
soak test that validates hash-check classification success rate over 1000 iterations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@corpo-iwillspeak corpo-iwillspeak force-pushed the test/functional-test-config-and-soak branch from 22cdcb4 to 1067cee Compare June 25, 2026 16:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the functional-test configuration to rely on SDK defaults when certain env vars are unset, and adds a new long-running soak test for classify_single to track KnownCSAM-* hash-check classification presence over many requests.

Changes:

  • Functional test config: stop hardcoding defaults for ATHENA_HOST, OAUTH_AUTH_URL, and OAUTH_AUDIENCE and let SDK defaults apply when unset.
  • Functional test auth: adjust CredentialHelper construction (incl. proactive refresh threshold setting).
  • Add a new soak test that runs 1000 classify_single calls and asserts a minimum hash-check classification success rate.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
tests/functional/test_classify_single_soak.py Adds a new soak test for repeated classify_single calls and success-rate assertion for KnownCSAM-* classifications.
tests/functional/conftest.py Updates functional-test env var handling to avoid overriding SDK defaults; adjusts how CredentialHelper and AthenaOptions are constructed.
Comments suppressed due to low confidence (1)

tests/functional/test_classify_single_soak.py:67

  • The assertion message hard-codes "below 95%" even though the threshold is defined by MIN_HASH_CHECK_SUCCESS_RATE. If the constant changes, the message becomes misleading.
        assert success_rate >= MIN_HASH_CHECK_SUCCESS_RATE, (
            f"Hash check success rate {success_rate:.1%} is below 95%. "
            f"{len(failures)} failures:\n" + "\n".join(failures[:20])
        )


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/functional/test_classify_single_soak.py
Comment thread tests/functional/test_classify_single_soak.py
Comment thread tests/functional/conftest.py
Comment thread tests/functional/test_classify_single_soak.py
@corpo-iwillspeak corpo-iwillspeak merged commit 37ee4dc into main Jun 26, 2026
12 checks passed
@corpo-iwillspeak corpo-iwillspeak deleted the test/functional-test-config-and-soak branch June 26, 2026 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants