fix(source-tiktok-marketing): add buying_types filter to campaigns stream#76062
Conversation
…ream The TikTok Marketing API /campaign/get/ endpoint defaults to returning only AUCTION buying type campaigns. This adds an explicit buying_types filter to include RESERVATION_TOP_VIEW and RESERVATION_REACH_FREQUENCY campaigns as well. The filter is always sent regardless of the include_deleted setting, ensuring all campaign types are returned. Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
|
Deploy preview for airbyte-docs ready!
Deployed with vercel-action |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
|
↪️ Triggering Reason: CI is green (35 passed, 0 failed). Fix adds |
|
|
Prove-Fix Validation —
|
| Check | Result |
|---|---|
| Viability | PASS — Fix adds buying_types filter to campaigns stream request params |
| Safety | PASS — No malicious code, single-line Jinja template change |
| Breaking Change | PASS — Non-breaking (patch version bump 5.0.5 to 5.0.6) |
| Reversibility | PASS — Can be safely rolled back by removing the filter |
Pre-release
- Docker tag:
5.0.6-preview.a454306 - Publish workflow: Run #23978190735 — SUCCESS
Regression Tests
- Comparison mode (target vs control): Run #23978229578
- Status: In Progress (READ step running)
Evidence: Fix Disproven
BLUF: The fix uses an invalid TikTok API enum value (RESERVATION_REACH_FREQUENCY), causing the campaigns stream to return 0 records.
Local Docker test (side-by-side comparison):
| Version | Campaign Records | Status |
|---|---|---|
| v5.0.5 (baseline) | 6 records | OK |
| v5.0.6-preview.a454306 (pre-release) | 0 records | REGRESSION |
Root Cause: The TikTok Marketing API returns error code 40002 when RESERVATION_REACH_FREQUENCY is passed in the buying_types filter:
filtering.buying_types.2: one or more value of the param is not acceptable,
correct is ['RESERVATION_RF', 'AUCTION', 'RESERVATION_TOP_VIEW'],
error is RESERVATION_REACH_FREQUENCY
Required Fix: In manifest.yaml line 281, change RESERVATION_REACH_FREQUENCY to RESERVATION_RF.
The valid buying_types values per the TikTok API are:
AUCTION(correct in PR)RESERVATION_TOP_VIEW(correct in PR)RESERVATION_RF(NOTRESERVATION_REACH_FREQUENCYas used in PR)
Recommendation
This fix needs a code change before it can be merged. The buying_types array in the Jinja template should use RESERVATION_RF instead of RESERVATION_REACH_FREQUENCY. After the fix, a new pre-release should be published and re-tested.
Last updated: 2026-04-04 13:38 UTC
Prove-Fix Validation:
|
| Check | Result | Notes |
|---|---|---|
| Viability | PASS | Fix adds buying_types filter to campaigns stream — addresses root cause (missing TopView/R&F campaigns) |
| Safety | PASS | Manifest-only YAML change, no custom Python code, no suspicious patterns |
| Breaking Change | NOT BREAKING | Additive filter, no schema changes, no field removals, no state format changes |
| Reversibility | REVERSIBLE | Patch version bump (5.0.5 → 5.0.6), no state migration needed |
Evidence Plan
Proving evidence (fix works):
- Regression tests pass in comparison mode (dev vs published baseline)
- Live connection sync succeeds and campaigns stream returns records including non-AUCTION buying types (TopView, R&F)
- No new errors or regressions introduced
Disproving evidence (fix does not work):
- Regression tests fail or show regressions
- Live sync fails or campaigns stream returns only AUCTION campaigns (same as before fix)
- New errors appear in sync logs
Workflows Triggered
| Workflow | Status | URL |
|---|---|---|
| Pre-release publish (5.0.6-preview.a454306) | In progress | View |
| Regression tests (comparison mode) | Triggered | View |
Candidate Connections for Live Testing
- 2 internal Airbyte org connections with campaigns stream enabled identified
- Multiple external connections with recent successful syncs available as fallback
- All candidates: not pinned, active, recent successful syncs
Next Steps
- Wait for pre-release publish to complete
- Wait for regression tests to complete and review results
- Escalate for live connection testing approval
- Pin candidate connection(s) to pre-release, trigger sync, monitor results
- Generate final evidence report
Last updated: 2026-04-04 11:48 UTC
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| $ref: "#/definitions/requester" | ||
| request_parameters: | ||
| filtering: '{{ {"secondary_status": "CAMPAIGN_STATUS_ALL"}|string if config.get("include_deleted", False) }}' | ||
| filtering: '{{ {"secondary_status": "CAMPAIGN_STATUS_ALL", "buying_types": ["AUCTION", "RESERVATION_TOP_VIEW", "RESERVATION_REACH_FREQUENCY"]}|string if config.get("include_deleted", False) else {"buying_types": ["AUCTION", "RESERVATION_TOP_VIEW", "RESERVATION_REACH_FREQUENCY"]}|string }}' |
There was a problem hiding this comment.
Fix Disproven: The TikTok Marketing API rejects RESERVATION_REACH_FREQUENCY as an invalid buying_types value. The correct enum value is RESERVATION_RF.
Evidence from local Docker testing (pre-release 5.0.6-preview.a454306 vs baseline 5.0.5):
- Baseline (v5.0.5): 6 campaign records returned
- Pre-release (v5.0.6-preview): 0 campaign records returned
API error:
filtering.buying_types.2: one or more value of the param is not acceptable,
correct is ['RESERVATION_RF', 'AUCTION', 'RESERVATION_TOP_VIEW'],
error is RESERVATION_REACH_FREQUENCY
Required change: Replace all occurrences of RESERVATION_REACH_FREQUENCY with RESERVATION_RF in the buying_types filter (both in the if and else branches of this Jinja template).
Detailed evidence in oncall issue airbytehq/oncall#11857.
🔍 Prove-Fix Validation ReportConnector:
|
| Check | Result |
|---|---|
| Viability | ✅ Fix addresses reported issue (TopView campaigns missing) |
| Design Intent | ✅ Not intentional behavior — TikTok API defaults to AUCTION only |
| Safety | ✅ No malicious code or suspicious patterns |
| Breaking Change | ✅ Non-breaking (patch version bump) |
| Reversibility | ✅ Safely reversible (revert single manifest line) |
Regression Test Results
Two independent regression test runs, both in comparison mode (target vs control):
| Run | Workflow | Result | Campaigns (Control → Target) |
|---|---|---|---|
| 1 | 23978195556 | 5 → 0 (Δ -5) | |
| 2 | 23978229578 | ❌ REGRESSION DETECTED (target exit code 1) | 5 → 0 (Δ -5) |
All other streams (33 streams) showed identical record counts between control and target — the regression is isolated to the campaigns stream.
Root Cause Analysis
The test account's 5 campaigns all have buying_type: null (older campaigns created before TikTok introduced buying type categorization). The Jinja filter in the fix:
# When include_deleted=False (default):
filtering: '{"buying_types": ["AUCTION", "RESERVATION_TOP_VIEW", "RESERVATION_REACH_FREQUENCY"]}'This explicitly restricts the API response to only campaigns matching those buying types. Campaigns with null buying type are excluded by the TikTok API, causing them to disappear from results.
Recommended Fix
The buying_types filter should not be sent when include_deleted is False (the default path), since the original behavior (no filter) already returns all campaign types. Instead, only add the buying_types filter to the include_deleted=True path alongside the secondary_status filter.
Alternatively, investigate whether TikTok's API supports a way to include campaigns with no buying type alongside the explicitly typed ones.
Live Connection Testing
NOT performed. Regression testing revealed the fix introduces a data loss regression. Pinning customer connections to this pre-release would cause campaigns to disappear from sync results.
…PI calls per type - Fix RESERVATION_REACH_FREQUENCY -> RESERVATION_RF (correct enum value) - RESERVATION_TOP_VIEW cannot be combined with other buying types per API - Add ListPartitionRouter to make separate API calls per buying type - Update tests to verify per-buying-type partitioning Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
| mock_advertisers_slices(http_mocker, self.config()) | ||
| self._mock_campaigns_for_all_buying_types(http_mocker, include_deleted=False) | ||
|
|
||
| output = read(get_source(config=self.config(), state=None), self.config(), self.catalog()) |
There was a problem hiding this comment.
This is a false positive — CodeQL is resolving get_source to source-gitlab/unit_tests/conftest.py instead of the local source-tiktok-marketing/unit_tests/conftest.py, which does accept state=None. The import on line 11 (from ..conftest import get_source) resolves to the tiktok-marketing conftest, not gitlab's.
| self._mock_campaigns_for_all_buying_types(http_mocker, include_deleted=True) | ||
|
|
||
| output = read( | ||
| get_source(config=self.config(include_deleted=True), state=None), |
There was a problem hiding this comment.
Same false positive as above — CodeQL is resolving to the wrong connector's get_source. The actual get_source imported from source-tiktok-marketing/unit_tests/conftest.py accepts state=None.
|
/publish-connectors-prerelease
|
…5.0.7 Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
|
/ai-canary-prerelease
|
Canary Prerelease: StartingBeginning canary prerelease testing for Plan:
|
|
Canary Prerelease: Phase 2-4 UpdatePrerelease: Phase 2: Breaking Change & Reversibility Gate — PASSED
Phase 3: Canary Connection SelectionSelected 8 connections across 6 distinct workspaces with diverse configurations:
Diversity: 3 tiers, 3 dataplanes (US, US-Central, EU), 4 destination types, 6 workspaces. Phase 4: Awaiting HITL ApprovalApproval requested via Slack For full customer details, see the linked private issue. |
|
/approve-regression-tests |
Canary Prerelease: Status Update — Awaiting ApprovalStatus: BLOCKED — awaiting HITL approval Approval was requested via Slack What's ready:
What's needed:
A reminder is set to check back tomorrow morning. If you can approve the canary request in For full customer details, see the linked private issue. |
Canary Prerelease: Phase 4 Complete — All 8 Connections PinnedPrerelease: All 8 canary connections have been successfully pinned:
Phase 5: Monitoring now starting. Will trigger syncs for idle connections and post updates every 1-2 hours. For full customer details, see the linked private issue. |
Canary Monitoring Update (2026-04-29 ~13:00 UTC, +1 hour)Prerelease:
Totals: 15 post-pin syncs observed, 15 succeeded, 0 failed (100% success rate) Overall status: HEALTHY — No errors, no regressions, record counts and byte volumes consistent with pre-pin baselines. Two hourly connections pending their first post-pin sync. Next update at ~14:00 UTC (+2 hours). For full customer details, see the linked private issue. |
Canary Prerelease: Final ReportConnector: SummaryThe prerelease Detailed Results
Total syncs observed: 25 Canary VerdictOverall Status: PASS ✅ The prerelease performed well across all canary connections. Recommend proceeding to formal release. Next steps:
For full customer details, see the linked private issue. |
Anatolii Yatsuk (tolik0)
deleted the
devin/1775172450-fix-tiktok-campaigns-buying-types
branch
3 participants
What
Resolves https://github.com/airbytehq/oncall/issues/11857
The campaigns stream was only returning
AUCTIONbuying type campaigns because the TikTok Marketing API's/campaign/get/endpoint defaults toAUCTION-only when no explicitbuying_typesfilter is provided. TopView (RESERVATION_TOP_VIEW) and Reach & Frequency (RESERVATION_RF) campaigns were silently excluded.How
Added a
ListPartitionRouterto the campaigns stream inmanifest.yamlthat iterates over each buying type (AUCTION,RESERVATION_TOP_VIEW,RESERVATION_RF), making a separate API call per type. This is required because the TikTok API does not allowRESERVATION_TOP_VIEWto be combined with other buying type values in a single request.The
filteringrequest parameter now referencesstream_partition.buying_typeto include the current partition's buying type:include_deleted=False: sends{"buying_types": ["<buying_type>"]}include_deleted=True: sends{"secondary_status": "CAMPAIGN_STATUS_ALL", "buying_types": ["<buying_type>"]}The
single_id_partition_router($ref) was inlined into the campaigns stream definition so that theListPartitionRouterandSingleAdvertiserIdPerPartitionrouter can be composed together (cartesian product of buying_type × advertiser_id).Version bumped from 5.0.5 → 5.0.6 (patch, non-breaking).
Declarative-First Evaluation
This connector is
language:manifest-only. The fix uses only declarative YAML components (ListPartitionRouter+ Jinja template) — no custom Python component needed.Review guide
airbyte-integrations/connectors/source-tiktok-marketing/manifest.yaml— the core fix. ThefilteringJinja template (line 281) now referencesstream_partition.buying_type. Thepartition_router(lines 286–308) is expanded from a$refto an inline list containing the newListPartitionRouterand the existingSingleAdvertiserIdPerPartition.airbyte-integrations/connectors/source-tiktok-marketing/unit_tests/integration/test_campaigns.py— new integration tests usingHttpMocker. Mocks all 3 buying-type requests per advertiser and verifies the exactfilteringquery parameter for bothinclude_deletedsettings.airbyte-integrations/connectors/source-tiktok-marketing/metadata.yaml— version bump.docs/integrations/sources/tiktok-marketing.md— changelog entry.Recommended human review checklist
{"buying_types": [stream_partition.buying_type]}|stringin Jinja produces the same JSON-formatted output that the API expects. The tests usejson.dumps()as the expected value — confirm these match at runtime (the CDK's|stringfilter behavior on dicts containing variables).CartesianProductStreamSliceremits warnings that "Parent state handling is not supported". Verify that incremental sync / state management still works correctly for the campaigns stream with the new partition router composition.single_id_partition_routerdefinition was inlined for the campaigns stream only. If the shared definition changes in the future, the campaigns stream won't automatically inherit those changes.Note on CodeQL bot comments: CodeQL flagged
state=Noneinget_source()calls as an unsupported parameter, but this is a false positive — it incorrectly resolvedget_sourcetosource-gitlab/unit_tests/conftest.pyinstead of the localsource-tiktok-marketing/unit_tests/conftest.py, which does acceptstate=None.User Impact
Users syncing the
campaignsstream will now receive TopView and Reach & Frequency campaigns in addition to Auction campaigns. This is additive — no existing data is removed or changed. Sync duration may increase slightly due to additional API calls per advertiser (3× instead of 1×).Can this PR be safely reverted and rolled back?
Link to Devin session: https://app.devin.ai/sessions/868948e6499b40f29a285000146a0b5b