Skip to content

DO NOT MERGE: test graceful OOM shutdown#942

Closed
Patrick Nilan (pnilan) wants to merge 1 commit intodevin/1773091366-add-memory-monitorfrom
test/graceful-oom-shutdown
Closed

DO NOT MERGE: test graceful OOM shutdown#942
Patrick Nilan (pnilan) wants to merge 1 commit intodevin/1773091366-add-memory-monitorfrom
test/graceful-oom-shutdown

Conversation

@pnilan
Copy link
Copy Markdown
Contributor

Summary

  • Adds an intentional memory leak in AirbyteEntrypoint.read() to test the memory monitor's graceful shutdown behavior
  • Each consumed record is duplicated 500x into a growing list to rapidly inflate memory
  • Tests that the 85% warning log and 95% critical MemoryLimitExceeded exception fire correctly in a container

Test plan

  • Deploy a connector using this branch in a memory-bounded container
  • Observe warning log at ~85% memory usage
  • Observe graceful shutdown with MemoryLimitExceeded at ~95%
  • Verify queued messages are flushed in the finally block before exit

DO NOT MERGE — test-only change to validate memory monitor behavior.

Duplicates each consumed record 500x into a growing list to rapidly
inflate memory and trigger the MemoryMonitor warning/critical thresholds.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 9, 2026

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@test/graceful-oom-shutdown#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch test/graceful-oom-shutdown

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@pnilan
Copy link
Copy Markdown
Contributor Author

Patrick Nilan (pnilan) commented Mar 9, 2026

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/22878700839

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 9, 2026

PyTest Results (Fast)

3 910 tests  ±0   3 898 ✅ ±0   6m 50s ⏱️ ±0s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit a17d00d. ± Comparison against base commit 5f7e16d.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 9, 2026

PyTest Results (Full)

3 913 tests  ±0   3 901 ✅ ±0   10m 40s ⏱️ -32s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit a17d00d. ± Comparison against base commit 5f7e16d.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant