Skip to content

test(bidirectional): Add integ test#15

Merged
JackYPCOnline merged 3 commits into
mehtarac:mainfrom
mkmeral:bidi-integ-test
Nov 3, 2025
Merged

test(bidirectional): Add integ test#15
JackYPCOnline merged 3 commits into
mehtarac:mainfrom
mkmeral:bidi-integ-test

Conversation

@mkmeral
Copy link
Copy Markdown

@mkmeral mkmeral commented Oct 29, 2025

Description

Adds integration tests for bidirectional streaming agents using Nova Sonic with a thread-based architecture that mimics real-world usage patterns.

Key Features

  • Thread-based test context manager - Manages continuous audio streaming with background threads that simulate real microphone input
  • Multi-turn conversation support - Enables natural back-and-forth conversations with follow-up questions
  • Amazon Polly integration - Generates test audio via text-to-speech, eliminating need for physical audio devices or pre-recorded files
  • Audio caching - Caches generated audio for fast test re-runs
  • Automatic lifecycle management - Context manager handles agent start/end and thread cleanup

Architecture

The test infrastructure uses a context manager (BidirectionalTestContext) that manages two background threads:

  1. Input thread - Continuously streams audio (real speech + silence padding) or sends text/other data
  2. Event collection thread - Captures all model outputs (text, audio, tool calls)

This approach enables realistic testing of continuous streaming scenarios without requiring physical audio hardware.

Usage Example

async with BidirectionalTestContext(agent, audio_generator) as ctx:
    # Turn 1
    await ctx.say("What is 5 + 3?")
    await ctx.wait_for_response()
    
    # Turn 2 - follow-up question
    await ctx.say("Now multiply that by 2")
    await ctx.wait_for_response()
    
    # Validate
    text_outputs = ctx.get_text_outputs()
    assert "8" in " ".join(text_outputs)
    assert "16" in " ".join(text_outputs)

Files Added

  • tests_integ/bidirectional_streaming/test_bidirectional_agent.py - Integration tests
  • tests_integ/bidirectional_streaming/utils/test_context.py - Thread-based context manager
  • tests_integ/bidirectional_streaming/utils/audio_generator.py - Polly integration with caching
  • tests_integ/bidirectional_streaming/conftest.py - Pytest fixtures
  • tests_integ/bidirectional_streaming/__init__.py - Module init
  • tests_integ/bidirectional_streaming/utils/__init__.py - Utils module init

Related Issues

Documentation PR

Type of Change

New feature

Testing

How have you tested the change?

  • Ran hatch run prepare - all checks pass
  • Ran integration tests: pytest tests_integ/bidirectional_streaming/ - 2 tests pass
  • Verified multi-turn conversations work correctly
  • Verified tool execution (calculator) works
  • Verified audio I/O streaming works
  • Verified session lifecycle management works
  • No linting errors
  • No type checking errors

Test Results

tests_integ/bidirectional_streaming/test_bidirectional_agent.py::test_single_turn_conversation PASSED
tests_integ/bidirectional_streaming/test_bidirectional_agent.py::test_multi_turn_conversation PASSED

2 passed in 20.52s

Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Comment thread tests_integ/bidirectional_streaming/test_bidirectional_agent.py
Copy link
Copy Markdown
Collaborator

@JackYPCOnline JackYPCOnline left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: check tool call after we emit those events.

@JackYPCOnline JackYPCOnline merged commit d2f3271 into mehtarac:main Nov 3, 2025
2 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants