Source Salesforce: Update streams.py for adapting changes with AsyncRetriever by btkcodedev · Pull Request #55186 · airbytehq/airbyte

btkcodedev (btkcodedev) · 2025-03-04T13:09:55Z

What

Closes https://github.com/airbytehq/airbyte-internal-issues/issues/11791

How

In specific:

upgrade the CDK version to at least 6.36.3
rename the urls_extractor to download_target_extractor, here and here
move off the stream_slice interpolation context to use the {{creation_response['id']}}, here
make sure the CAT passes, as before
make sure the Regression Tests pass, as before
no source behaviour changes are expected

vercel · 2025-03-04T13:10:00Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 19, 2025 9:11pm

btkcodedev (btkcodedev) · 2025-03-04T13:11:06Z

/bump-version type="minor" changelog="Update manifest for adapting changes with AsyncRetriever"

Bump Version job started... Check job output.

✅ Changes applied successfully. (611224c)

Maxime Carbonneau-Leclerc (maxi297) · 2025-03-04T14:18:16Z

Tests are failing because of TypeError: AsyncHttpJobRepository.__init__() got an unexpected keyword argument 'download_target_extractor' (and maybe for other reasons). Let me know when this is ready for a review and I'll check it out!

Baz (bazarnov) · 2025-03-04T18:13:19Z

Tests are failing because of TypeError: AsyncHttpJobRepository.__init__() got an unexpected keyword argument 'download_target_extractor' (and maybe for other reasons). Let me know when this is ready for a review and I'll check it out!

Because of this, here:

airbyte-cdk = "^5.10.2"

The version should be:

airbyte-cdk = "^6.33.3"

btkcodedev (btkcodedev) · 2025-03-05T01:46:03Z

Thanks Baz (@bazarnov) That helps!!

Maxime Carbonneau-Leclerc (maxi297) · 2025-03-18T15:35:03Z

Hey btkcodedev (@btkcodedev), thanks for the contribution! It turns out that this was more complex than anticipated. Hence, I pushed some changes. The tests will still fail. This is expected at this point. We need this fix from the CDK. Once the CDK release is done, we can up the version here and the tests should be green. At that point, I'll ask a review from another extensibility member

Maxime Carbonneau-Leclerc (maxi297) · 2025-03-19T01:13:20Z

This recent change has created a regression where we don't retry on timed out jobs. I've started a thread internally to try to understand why we have this change

Maxime Carbonneau-Leclerc (maxi297) · 2025-03-19T18:36:42Z

Everything has been updated. CATs are failing but they are failing the same way on master so I think I'm fine with this. Baz (@bazarnov) can you have a review on this?

Baz (bazarnov)

Looks great!

The CAT issues are about the "expected records." Some "IDs" are not matching, which is a relatively straightforward problem that can be easily resolved by updating the exptected_records. btkcodedev (@btkcodedev)

Baz (bazarnov) · 2025-03-19T20:30:44Z

/approve-regression-tests

Check job output.

✅ Approving regression tests

btkcodedev (btkcodedev) · 2025-03-19T20:51:10Z

Baz (@bazarnov) Can we declare ActiveFeatureLicenseMetric as empty stream?

Ref:
[95m158 : �[0m �[90m[5m15.2s] �[0m�[90m|�[0m INFO control-read:models.py:396 Reading records for stream ActiveFeatureLicenseMetric
�[95m158 : �[0m �[90m[5m15.2s] �[0m�[90m|�[0m WARNING control-read:models.py:398 No records found for stream ActiveFeatureLicenseMetric
�[95m158 : �[0m �[90m[5m15.2s] �[0m�[90m|�[0m INFO test_all_pks_are_produced_in_target_version_without_state[CONNECTION c3b48b1]:test_read.py:74 Retrieving primary keys for stream ActiveFeatureLicenseMetric on target version.

btkcodedev (btkcodedev) · 2025-03-19T21:02:56Z

/format-fix

Format-fix job started... Check job output.

✅ Changes applied successfully. (a2f13c8)

Baz (bazarnov) · 2025-03-20T08:33:38Z

/approve-regression-tests

Check job output.

✅ Approving regression tests

Maxime Carbonneau-Leclerc (maxi297)

Merging this. Thanks btkcodedev (@btkcodedev) !

…etriever (airbytehq#55186) Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com> Co-authored-by: maxi297 <maxime@airbyte.io>

Update streams.py according to the description

6a2042a

btkcodedev (btkcodedev) requested a review from a team as a code owner March 4, 2025 13:09

Octavia Squidington III (octavia-squidington-iii) added area/connectors Connector related issues connectors/source/salesforce labels Mar 4, 2025

chore: bump-version

611224c

Octavia Squidington III (octavia-squidington-iii) added the area/documentation Improvements or additions to documentation label Mar 4, 2025

vercel Bot deployed to Preview March 4, 2025 13:21 View deployment

update streams.py

849c72c

vercel Bot deployed to Preview March 4, 2025 15:56 View deployment

btkcodedev (btkcodedev) added 2 commits March 4, 2025 21:35

update CDK version

7fa63b5

update naming

a5b8c0d

vercel Bot deployed to Preview March 4, 2025 16:12 View deployment

update cdk version

2c577c2

vercel Bot deployed to Preview March 5, 2025 01:48 View deployment

add missing dependency

7a3a208

vercel Bot deployed to Preview March 5, 2025 02:21 View deployment

update CDK

dfe071e

vercel Bot deployed to Preview March 13, 2025 21:35 View deployment

Merge branch 'master' into btkcodedev/salesforceUpdateAsyncRetriever

39fdaf5

vercel Bot deployed to Preview March 18, 2025 14:31 View deployment

Fix tests because of CDK upgrade

8dab2f5

vercel Bot deployed to Preview March 18, 2025 15:36 View deployment

Up cdk version

25e9d92

vercel Bot deployed to Preview March 19, 2025 17:32 View deployment

Maxime Carbonneau-Leclerc (maxi297) requested a review from Baz (bazarnov) March 19, 2025 18:36

Baz (bazarnov) approved these changes Mar 19, 2025

View reviewed changes

update expected records

4c88417

vercel Bot deployed to Preview March 19, 2025 20:21 View deployment

update ActiveFeatureLicenseMetric as empty stream

aa9dc68

vercel Bot deployed to Preview March 19, 2025 20:59 View deployment

chore: auto-fix lint and format issues

a2f13c8

vercel Bot deployed to Preview March 19, 2025 21:11 View deployment

Maxime Carbonneau-Leclerc (maxi297) approved these changes Mar 20, 2025

View reviewed changes

Maxime Carbonneau-Leclerc (maxi297) merged commit 5c54e29 into master Mar 20, 2025

Maxime Carbonneau-Leclerc (maxi297) deleted the btkcodedev/salesforceUpdateAsyncRetriever branch March 20, 2025 12:48

Conversation

btkcodedev (btkcodedev) commented Mar 4, 2025

What

How

Uh oh!

vercel Bot commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

btkcodedev (btkcodedev) commented Mar 4, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Maxime Carbonneau-Leclerc (maxi297) commented Mar 4, 2025

Uh oh!

Baz (bazarnov) commented Mar 4, 2025

Uh oh!

btkcodedev (btkcodedev) commented Mar 5, 2025

Uh oh!

Maxime Carbonneau-Leclerc (maxi297) commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Maxime Carbonneau-Leclerc (maxi297) commented Mar 19, 2025

Uh oh!

Maxime Carbonneau-Leclerc (maxi297) commented Mar 19, 2025

Uh oh!

Baz (bazarnov) left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Baz (bazarnov) commented Mar 19, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

btkcodedev (btkcodedev) commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

btkcodedev (btkcodedev) commented Mar 19, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Baz (bazarnov) commented Mar 20, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Maxime Carbonneau-Leclerc (maxi297) left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

vercel Bot commented Mar 4, 2025 •

edited

Loading

btkcodedev (btkcodedev) commented Mar 4, 2025 •

edited by github-actions Bot

Loading

Maxime Carbonneau-Leclerc (maxi297) commented Mar 18, 2025 •

edited

Loading

Baz (bazarnov) left a comment •

edited

Loading

Baz (bazarnov) commented Mar 19, 2025 •

edited by github-actions Bot

Loading

btkcodedev (btkcodedev) commented Mar 19, 2025 •

edited

Loading

btkcodedev (btkcodedev) commented Mar 19, 2025 •

edited by github-actions Bot

Loading

Baz (bazarnov) commented Mar 20, 2025 •

edited by github-actions Bot

Loading