Skip to content

Implement reddit activity collector#285

Merged
wpak-ai merged 5 commits into
cppalliance:developfrom
AuraMindNest:feature/reddit-activity-tracker
Jun 12, 2026
Merged

Implement reddit activity collector#285
wpak-ai merged 5 commits into
cppalliance:developfrom
AuraMindNest:feature/reddit-activity-tracker

Conversation

@AuraMindNest

@AuraMindNest AuraMindNest commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Close #270.
Close #271.
Close #272.
Close #273.

Summary

Implements the full reddit_activity_tracker collector for r/cpp (PR2), building on the earlier foundation stub:

  • RedditUser profile — new RedditUser BaseProfile in cppa_user_tracker; submissions/comments use FK to RedditUser instead of inline author/author_id.
  • Schema renamereddit_idreddit_submission_id / reddit_comment_id on submission and comment tables.
  • Incremental scrapingrun_reddit_activity_tracker resumes from max created_utc in DB (or --since / 30-day default when empty) through now.
  • Comment-first discovery — PullPush comment search with Reddit /r/cpp/comments fallback, plus submission discovery via search + /new and /api/info for missing posts.
  • Full comment trees/api/morechildren expansion for complete per-submission comment fetch.
  • DB + workspace writes — upserts via services.py; per-record JSON under workspace/reddit_activity_tracker/users/, submissions/, comments/ (overwrite on duplicate id).
  • Fetcher upgradespost(), rate-limit header tracking, proactive pause; auth via client credentials or REDDIT_BEARER_TOKEN / REDDIT_SESSION_COOKIE (parity with reddit-scraper).
  • Daily schedulerun_reddit_activity_tracker added to config/boost_collector_schedule.yaml (17:00 UTC).

Apps touched

  • reddit_activity_tracker
  • cppa_user_tracker (RedditUser model + migration)
  • config (settings, boost_collector_schedule.yaml)
  • .env.example
  • docs/service_api (reddit_activity_tracker, cppa_user_tracker)

Test plan

  • python -m pytest reddit_activity_tracker/tests/ -v (32 tests)
  • uv run pytest --cov --cov-fail-under=90 (full suite: 2980 passed)
  • uv run pre-commit run -a (lint/format/import-linter/service-layer checks)
  • uv run pyright
  • uv run python scripts/validate_collector_scaffold.py
  • App command smoke-tested:
python manage.py migrate
python manage.py run_reddit_activity_tracker --since 2026-06-12

Docs / coupling

  • cross-app-dependencies.md updated (new FK: RedditSubmission.user / RedditComment.usercppa_user_tracker.RedditUser)
  • python scripts/generate_service_docs.py run (reddit_activity_tracker.services, cppa_user_tracker.services)
  • App README or docs/ updated (if behavior or ops changed)

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AuraMindNest, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 57 minutes and 7 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7fd8dac1-041c-4f8b-afc8-01a2f49c12a1

📥 Commits

Reviewing files that changed from the base of the PR and between 68d28f0 and 49434c2.

📒 Files selected for processing (25)
  • .env.example
  • config/boost_collector_schedule.yaml
  • config/settings.py
  • core/_version.py
  • cppa_user_tracker/admin.py
  • cppa_user_tracker/migrations/0009_reddituser_alter_baseprofile_type.py
  • cppa_user_tracker/models.py
  • cppa_user_tracker/services.py
  • cppa_user_tracker/tests/test_services.py
  • docs/service_api/cppa_user_tracker.md
  • docs/service_api/reddit_activity_tracker.md
  • reddit_activity_tracker/admin.py
  • reddit_activity_tracker/fetcher.py
  • reddit_activity_tracker/management/commands/run_reddit_activity_tracker.py
  • reddit_activity_tracker/migrations/0002_replace_author_with_user_fk.py
  • reddit_activity_tracker/models.py
  • reddit_activity_tracker/services.py
  • reddit_activity_tracker/tests/conftest.py
  • reddit_activity_tracker/tests/test_collector_integration.py
  • reddit_activity_tracker/tests/test_fetcher.py
  • reddit_activity_tracker/tests/test_models.py
  • reddit_activity_tracker/tests/test_run_reddit_activity_tracker_command.py
  • reddit_activity_tracker/tests/test_services.py
  • reddit_activity_tracker/tests/test_workspace.py
  • reddit_activity_tracker/workspace.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@AuraMindNest AuraMindNest changed the title Implement incremental r/cpp collector Implement reddit activity collector Jun 12, 2026
@AuraMindNest AuraMindNest self-assigned this Jun 12, 2026
snowfox1003
snowfox1003 previously approved these changes Jun 12, 2026
@snowfox1003 snowfox1003 self-requested a review June 12, 2026 18:05
@wpak-ai wpak-ai merged commit 884ef63 into cppalliance:develop Jun 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants