Skip to content

[WebPubSub] Fix live tests with AAD-based test resources#46626

Open
MoChilia wants to merge 22 commits intomainfrom
csy/fix-livetest
Open

[WebPubSub] Fix live tests with AAD-based test resources#46626
MoChilia wants to merge 22 commits intomainfrom
csy/fix-livetest

Conversation

@MoChilia
Copy link
Copy Markdown
Member

Description

  • Update Web PubSub live test setup to use SDK-created test resources and AAD authentication instead of preconfigured connection-string secrets.
  • Add bounded waits/retries around timing-sensitive Web PubSub client reconnect, recovery, group rejoin, and callback scenarios.
  • Ignore integration_sample.py in sample validation because it requires multi-client live WebSocket behavior.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

MoChilia and others added 15 commits April 27, 2026 11:29
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
@MoChilia MoChilia marked this pull request as ready for review April 30, 2026 08:28
Copilot AI review requested due to automatic review settings April 30, 2026 08:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Web PubSub Python SDK live-test and sample setup to rely on test-resources provisioning and AAD auth (instead of connection-string secrets), while improving resiliency of timing-sensitive WebPubSub client tests and excluding a multi-client integration sample from automated sample validation.

Changes:

  • Introduce test-resources.bicep and adjust pipeline/test preparers to use endpoint-based configuration and AAD credentials.
  • Add bounded waits/retries/timeouts across several live tests to reduce flakiness (reconnect/recovery/group rejoin/list connections/etc.).
  • Exclude integration_sample.py from sample validation runs.

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
sdk/webpubsub/tests.yml Removes preconfigured Web PubSub secret env vars; relies on live test resource outputs.
sdk/webpubsub/test-resources.bicep Adds infra template to provision Web PubSub resources and output endpoints for tests.
sdk/webpubsub/azure-mgmt-webpubsub/dev_requirements.txt Adds mgmt-resource dependency for resource provisioning workflows.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/testcase_async.py Removes connection-string client creation path; uses AAD credential flow.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/testcase.py Removes connection-string client creation path; updates preparer defaults (adds socketio endpoint).
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_smoke_async.py Switches tests to endpoint-based client creation (AAD).
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_smoke.py Switches tests to endpoint-based client creation (AAD).
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_reverse_proxy_async.py Switches reverse-proxy live call to endpoint-based client creation.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_reverse_proxy.py Switches reverse-proxy live call to endpoint-based client creation.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_live_api_coverage_async.py Adds retries/timeouts and uses socketio endpoint for SocketIO token coverage.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_live_api_coverage.py Adds retries/timeouts and uses socketio endpoint for SocketIO token coverage.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_list_connections_async.py Uses endpoint auth + adds wait for server-side connection registration.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_list_connections.py Uses endpoint auth + adds wait for server-side connection registration.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_generated_api_coverage_async.py Updates API coverage tests to use endpoint-based client creation.
sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_generated_api_coverage.py Updates API coverage tests to use endpoint-based client creation.
sdk/webpubsub/azure-messaging-webpubsubservice/samples/send_messages_connection_string_apim_proxy.py Switches sample to AAD credential + endpoint env var.
sdk/webpubsub/azure-messaging-webpubsubservice/samples/send_messages_connection_string.py Switches sample to AAD credential + endpoint env var.
sdk/webpubsub/azure-messaging-webpubsubservice/samples/integration_sample.py Switches integration sample to AAD credential + endpoint env var.
sdk/webpubsub/azure-messaging-webpubsubservice/samples/get_client_access_token_async.py Removes connection-string path; demonstrates AAD-only token flow.
sdk/webpubsub/azure-messaging-webpubsubservice/samples/get_client_access_token.py Removes connection-string path; demonstrates AAD-only token flow.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/testcase_async.py Switches WebPubSubClient test harness to endpoint-based AAD token provider.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/testcase.py Switches WebPubSubClient test harness to endpoint-based AAD token provider + preparer defaults.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_smoke_async.py Updates tests to use endpoint-based setup; adds bounded waits/retries.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_smoke.py Updates tests to use endpoint-based setup; adds bounded waits/retries.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_send_concurrently_async.py Uses endpoint-based setup for concurrent send test.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_send_concurrently.py Uses endpoint-based setup + waits for connection stabilization before concurrent sends.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_recovery_async.py Adds bounded waits for connection id and callbacks; switches to endpoint-based setup.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_recovery.py Adds bounded waits for connection id and callbacks; switches to endpoint-based setup.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_no_recovery_no_connect_async.py Switches to endpoint-based setup; adds waits to confirm disconnect/no message received.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_no_recovery_no_connect.py Switches to endpoint-based setup; adds waits to confirm disconnect/no message received.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_auto_connect_async.py Switches to endpoint-based setup; adds bounded waits for reconnect and callbacks.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_auto_connect.py Switches to endpoint-based setup; adds bounded waits/retries for reconnect + rejoin/send.
sdk/webpubsub/azure-messaging-webpubsubclient/tests/conftest.py Updates sanitizers to sanitize endpoint instead of connection string.
sdk/webpubsub/azure-messaging-webpubsubclient/samples/send_compare_between_async_sync.py Switches sample token providers to AAD + endpoint; modernizes asyncio entrypoint.
sdk/webpubsub/azure-messaging-webpubsubclient/samples/hello_world_exceptions.py Switches sample to AAD + endpoint for token generation.
sdk/webpubsub/azure-messaging-webpubsubclient/samples/hello_world_async.py Switches sample to AAD + endpoint for token generation.
sdk/webpubsub/azure-messaging-webpubsubclient/samples/hello_world.py Switches sample to AAD + endpoint for token generation.
sdk/webpubsub/azure-messaging-webpubsubclient/dev_requirements.txt Uses local azure-messaging-webpubsubservice instead of pinned version.
eng/tools/azure-sdk-tools/azpysdk/samples.py Ignores integration_sample.py during sample validation for this package.
Comments suppressed due to low confidence (1)

sdk/webpubsub/azure-messaging-webpubsubclient/samples/send_compare_between_async_sync.py:42

  • AsyncDefaultAzureCredential and the async service client are created but never closed. This can leak network resources and emit warnings in sample runs; prefer using async with (or explicitly closing) for both the credential and the async client.
async def client_access_url_provider_async():
    credential = AsyncDefaultAzureCredential()
    service_client_async = WebPubSubServiceClientAsync(
        endpoint=os.getenv("WEBPUBSUB_ENDPOINT", ""), hub="hub", credential=credential
    )
    return (await service_client_async.get_client_access_token(
        roles=["webpubsub.joinLeaveGroup", "webpubsub.sendToGroup"]
    ))["url"]

Comment thread sdk/webpubsub/azure-messaging-webpubsubservice/samples/integration_sample.py Outdated
Comment thread sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_smoke_async.py Outdated
Comment thread sdk/webpubsub/test-resources.bicep
Comment thread sdk/webpubsub/azure-messaging-webpubsubclient/tests/test_smoke.py Outdated
Comment thread sdk/webpubsub/azure-messaging-webpubsubclient/tests/conftest.py
Comment thread sdk/webpubsub/azure-messaging-webpubsubservice/samples/get_client_access_token.py Outdated
Comment thread sdk/webpubsub/azure-messaging-webpubsubservice/tests/test_smoke.py Outdated
@MoChilia
Copy link
Copy Markdown
Member Author

MoChilia commented May 7, 2026

Hi @ChenxiJiang333, @mccoyp, @msyyc, @scbedd, could you please review this pr? Your code owner review is required. Thank you!

var webpubsubName = 'e2e-${baseName}'
var socketioName = 'e2e-socketio-${baseName}'
// Web PubSub Service Owner
var ownerRoleId = '12cf5a90-567b-43ae-8102-96cf46c7d9b4'
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sensitive info of azure resources?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is non-sensitive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants