chore: increase MAX_CONNECTION_POOL_SIZE from 20 to 40

[devin-ai-integration]

Summary

Increases MAX_CONNECTION_POOL_SIZE from 20 to 40 in airbyte_cdk/sources/http_config.py and updates the two corresponding test assertions.

With connectors now supporting max_concurrency up to 40 (e.g., source-intercom PR), the previous pool size of 20 causes urllib3 warnings:

Connection pool is full, discarding connection: api.intercom.io. Connection pool size: 20

Connections still succeed but bypass the pool (no reuse), reducing efficiency. This change aligns the pool size with the maximum concurrency connectors can configure.

Review & Testing Checklist for Human

• Verify that 40 concurrent pooled connections per connector instance is acceptable from a memory/socket resource perspective, especially for deployments running many connectors simultaneously
• Consider whether this constant should instead be made dynamically configurable (tied to the connector's actual max_concurrency setting) rather than a static increase — the existing comment in http_config.py already notes this as a future improvement

Notes

Requested by: gl_anatolii.yatsuk
Devin session

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1772449759-increase-max-connection-pool-size#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1772449759-increase-max-connection-pool-size

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[github-actions]

PyTest Results (Fast)

4 012 tests  ±0   4 001 ✅ ±0   7m 45s ⏱️ +2s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit bdc6622. ± Comparison against base commit 4aaafcf.

♻️ This comment has been updated with latest results.

[github-actions]

PyTest Results (Full)

4 015 tests  ±0   4 003 ✅ ±0   11m 22s ⏱️ +23s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit bdc6622. ± Comparison against base commit 4aaafcf.

♻️ This comment has been updated with latest results.

[Anatolii Yatsuk (tolik0)]

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/22575084479

[Anatolii Yatsuk (tolik0)]

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/24029908686

[devin-ai-integration]

❌ Cannot revive Devin session - the session is too old. Please start a new session instead.

[Copilot]

Pull request overview

Increases the default HTTP connection pool size to better match connectors that can run with higher concurrency, avoiding urllib3 “Connection pool is full” warnings and improving connection reuse.

Changes:

• Bumped MAX_CONNECTION_POOL_SIZE from 20 → 40 in the shared HTTP config.
• Updated two unit tests that assert the configured connection pool size.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
airbyte_cdk/sources/http_config.py	Raises the shared connection pool size constant used by the HTTP client adapter.
unit_tests/sources/streams/http/test_http.py	Updates the expected adapter pool configuration assertion.
unit_tests/sources/streams/http/test_http_client.py	Updates the expected adapter pool configuration assertion for HttpClient.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Anatolii Yatsuk (tolik0)]

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/24089021469