feat: add no-ai filtering support for my feed

[nimrodkra]

Summary

• add validated AI tag and title-word blocklists for the No-AI experiment
• extend the feed query and My Feed resolver to merge No-AI filters without changing saved feed settings
• make the local/v1 path honor title-word blocking so dev behavior is closer to production

Test plan

• pnpm run lint
• pnpm run build

Made with Cursor

[pulumi]

🍹 The Update (preview) for dailydotdev/api/prod (at e805896) was successful.

✨ Neo Explanation

This is a standard application deployment rolling out a new build to all 7 services and 37 scheduled jobs, accompanied by database and ClickHouse schema migration jobs that clean up the previous release's migration jobs and create new ones for this version.

Root Cause Analysis

A new version of the API application has been built and is being deployed to production. Every workload in the cluster references the container image by version tag, so publishing a new image triggers updates across all Deployments, CronJobs, and migration Jobs simultaneously.

Dependency Chain

The new container image version cascades uniformly across the entire fleet:

• 7 Deployments (main API, background worker, WebSocket, private, temporal, personalized digest, worker jobs) are all rolling out the new image
• 37 CronJobs are updated to reference the new image version in their job templates — these will take effect on their next scheduled run
• 2 one-time migration Jobs (database and ClickHouse) are being replaced: the old versions tied to the previous release are deleted, and new ones for the current release are created to run any schema migrations required by this code change

Risk analysis

No stateful resources (databases, storage buckets, persistent volumes) are being replaced or deleted. The migration Jobs run schema changes against the database and ClickHouse, which carries inherent risk if the migrations are destructive or irreversible, but this is standard practice for this deployment pattern. Deployments use rolling updates by default in Kubernetes, so no downtime is expected for the running services.

Resource Changes

    Name                                                       Type                           Operation
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                      kubernetes:apps/v1:Deployment  update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-af9ea149                       kubernetes:batch/v1:Job        create
+   vpc-native-api-clickhouse-migration-af9ea149               kubernetes:batch/v1:Job        create
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment  update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
-   vpc-native-api-clickhouse-migration-e28a81e8               kubernetes:batch/v1:Job        delete
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-expire-super-agent-trial-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-squad-posts-analytics-refresh-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tags-str-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-highlights-cron                         kubernetes:batch/v1:CronJob    update
-   vpc-native-api-db-migration-e28a81e8                       kubernetes:batch/v1:Job        delete
~   vpc-native-update-views-cron                               kubernetes:batch/v1:CronJob    update
... and 9 other changes

[nimrodkra]

@claude please review this pr

[claude]

Claude finished @nimrodkra's task in 2m 45s —— View job

---

PR Review

• Gathering context
• Reading changed files
• Analyzing code
• Posting review

---

The overall approach is clean — the withConfigTransform pattern on FeedGenerator is a nice abstraction, and correctly separating v1 (local SQL path) from v2+ (feed service path) is the right way to handle both cases. A few issues worth addressing before merging:

---

Issues

🔴 Performance — 82 individual ILIKE conditions on v1 path

src/common/feedGenerator.ts:709-720

For noAi=true on a v1 feed, 82 separate NOT ILIKE '%word%' WHERE clauses are added to the SQL query. PostgreSQL must evaluate each one as a full string scan on every row. For large feeds this will be a meaningful regression.

A single regex condition is dramatically cheaper:

if (filters?.blockedWords?.length) {
  const pattern = filters.blockedWords
    .map((w) => w.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'))
    .join('|');
  newBuilder = newBuilder.andWhere(
    `COALESCE(${alias}.title, '') !~* :blockedWordsPattern`,
    { blockedWordsPattern: pattern },
  );
}

Fix this →

---

🔴 High false-positive risk — Cursor, RAG, inference, transformer

src/common/noAiFilter.ts

Several words in NO_AI_BLOCKED_WORDS are common general programming terms:

Word	Legitimate non-AI context
Cursor (line 115)	CSS cursor property, cursor pagination, database cursors, text cursors
RAG (line 132)	Red-Amber-Green status in project management
inference (line 149)	TypeScript type inference, statistical inference, logical inference
transformer (line 146)	Babel/esbuild AST transformers, electrical transformers

A post titled "Custom CSS cursor effects" or "TypeScript type inference tips" would be blocked for users who want to avoid AI content. Recommend removing or scoping these to compound forms only (e.g., AI inference, RAG pipeline instead of bare RAG/inference).

---

🟡 False positives — person names

src/common/noAiFilter.ts:164-167

Elon Musk and Jensen Huang are primarily associated with Tesla/SpaceX and Nvidia gaming, respectively. Many non-AI articles (Nvidia GPU releases, Tesla news, SpaceX launches) mention them. Consider dropping these or being more targeted.

---

🟡 No tests

The PR adds non-trivial business logic (withNoAiFilters, wrapGeneratorWithNoAi, mergeUniqueStrings, the new SQL filter in anonymousFeedBuilder) but includes no tests. Per the repo's testing strategy, integration tests are strongly preferred.

At minimum, an integration test for the feed query with noAi: true asserting that posts with blocked tags/words are excluded would give confidence this works end-to-end on the v1 path.

---

🟡 No feature flag — any client can toggle this

This is described as a "No-AI experiment" but there's no GrowthBook flag controlling access. Any client can pass noAi: true to get the filtered feed. If this is intended as a controlled rollout/experiment, consider gating it server-side so you can measure impact without exposing it broadly.

---

🟡 Helper functions defined after use

src/schema/feeds.ts:1411-1438

mergeUniqueStrings, withNoAiFilters, and wrapGeneratorWithNoAi are defined after feedResolverV1 which calls withNoAiFilters. This works at runtime (the call is inside a callback that runs later), but it's confusing to read and conflicts with the convention of defining helpers before their consumers. Move these three helpers above feedResolverV1.

Fix this →

---

🟢 Positive notes

• withConfigTransform is a clean, reusable hook on FeedGenerator that doesn't bleed experiment logic into the generator itself.
• mergeUniqueStrings correctly deduplicates user preferences with the experiment blocklist, so existing user preferences are preserved.
• The split between v1 (SQL) and v2+ (feed service) is handled correctly with separate code paths.

---

[nimrodkra]

Addressed the review on this branch:

• replaced the v1 path's per-word NOT ILIKE chain with a single regex filter so we don't stack dozens of title scans
• moved the No-AI helper functions above feedResolverV1 for readability
• added an integration test covering feed(noAi: true) on the local/v1 path for both blocked tags and blocked title words

On the false-positive points: I'm intentionally keeping the broader title-word blocklist. Product direction here is to optimize for not leaking AI content even if that means occasionally filtering adjacent content. That tradeoff is deliberate for this first experiment.

Verification completed:

• pnpm run lint
• pnpm run build

I also attempted:

• NODE_ENV=test npx jest tests/feeds.ts --testEnvironment=node --runInBand

That local test run is blocked in this environment because Redis is unavailable (ECONNREFUSED 127.0.0.1:6379).

[idoshamun]

We dont need this here. anonymousFeed shouldn't support the ai filter anyway. We should limit the blast radius to my feed

[nimrodkra]

Removed the local SQL title-word filtering and reverted the anonymous-side expansion. The No-AI experiment now stays on the supported My Feed path only.

[idoshamun]

Resolver v1 is also out of scope

[nimrodkra]

Reverted the v1 resolver merge as requested. noAi is now only applied on the v2 feed-service path, and the test was moved to assert the v2 request config instead.

[nimrodkra]

Addressed idoshamun's requested scope reduction on this branch:

• removed the local SQL / anonymous-side title-word filtering
• reverted the v1 resolver merge
• kept noAi applied only on the supported v2 feed-service path for feed
• updated the test to assert the v2 request contains the injected blocked tags and title words

Verification:

• pnpm run lint
• pnpm run build

I also retried the feed Jest suite, but it is still blocked in this environment because local Redis is unavailable (ECONNREFUSED 127.0.0.1:6379).