Skip to content

fix: codebase audit — bugs, dead code, hardening, test gaps#85

Merged
GeiserX merged 3 commits into
mainfrom
fix/codebase-audit-safe-wave
Jun 11, 2026
Merged

fix: codebase audit — bugs, dead code, hardening, test gaps#85
GeiserX merged 3 commits into
mainfrom
fix/codebase-audit-safe-wave

Conversation

@GeiserX

@GeiserX GeiserX commented Jun 11, 2026

Copy link
Copy Markdown
Owner

Summary

Whole-codebase multi-agent audit (9 auditors: architecture, security, complexity, dead code, error handling, test health, consistency, performance) followed by an independent verification pass. This PR lands the safe, high-value findings; four large/risky items are deferred to dedicated follow-up PRs (listed below).

904 tests passing (was 887 — +17 regression tests), ruff + format clean. Every fix was verified to fail against the pre-fix code.

Correctness / silent-failure bugs

  • Collectors — silent earnings corruption (Critical): a missing balance field used to become float(data.get("balance", 0)) → a fake 0.0 persisted as a real reading, corrupting deltas and producing false spikes when an upstream API shape changed. Now a missing field surfaces an error (visible in the alert bell); legitimate zeros are preserved. 11 collectors + anyone non-numeric guard.
  • metrics — dead-worker alerting (3 auditors): _refresh_gauges read w.get("last_seen"), a column that never existed (it's last_heartbeat), so the staleness gauge was never set. Fixed.
  • Scheduler blindness: added an APScheduler EVENT_JOB_ERROR | EVENT_JOB_MISSED listener + max_instances/coalesce/misfire_grace_time so a crashing job is logged, not silently stopped.
  • Stale alert bug: _run_collection now publishes a synthetic alert on crash so the bell can't show green during a total collection outage.
  • auth: narrowed except (BadSignature, Exception)except BadData and moved the epoch check out of the try, so genuine bugs surface instead of masquerading as silent logouts (legit bad/expired tokens still return 401).

Security hardening

  • Worker-proxy errors no longer echo raw worker response text / internal URLs into the HTTP detail (topology + SSRF-oracle leak); real detail logged server-side only.

Performance

  • database.py: added created_at indexes for the daily retention purge (health_events grows ~4.6M rows/yr → was a full-scan write-lock stall); collapsed upsert_earnings to a single ON CONFLICT DO UPDATE.

Consistency / correctness

  • BaseCollector is now a real abc.ABC (@abstractmethod collect); proxyrack _fetch_balance returns only a Response.
  • _schema.yml: documented status: dropped, cashout.method: auto, collector.type: auto, docker.privileged; removed dead referral.bonus. dropped services are now hidden from the available catalog.
  • exchange_rates: exposed rates_stale() + log non-200 rate fetches.

Dead code

  • Deleted unreferenced orchestrator.deploy_service (also had a latent port-parse crash) and reset_docker_status; docker_available() now re-probes instead of caching False forever. Deleted unused _require_auth. Removed abandoned JS worker-cache + unwired goToCollectorSettings; alert()toast(). .gitignore now covers tooling/coverage artifacts.

Tests

+17 regression tests, each verified to fail against the pre-fix code: per-collector missing-balance (incl. legitimate-zero cases), heartbeat gauge populated from last_heartbeat (with a negative case proving the old last_seen payload leaves it unset), rates_stale() states.

Deferred to follow-up PRs (intentionally NOT in this sweep)

  • SSRF allowlist for worker URLs — the fleet runs on RFC1918/Tailscale, so blocking private ranges by default needs an opt-in model.
  • /api/config secret-readback + change-password/session-invalidation — changes the Settings UX contract + adds a feature; warrants its own careful PR.
  • SQLite connection pooling (database.py connection-per-query) — 36-function refactor.
  • main.py router split (1848-line god module) — large structural refactor.

Audit headline scores

Architecture 6 · Security 6.5 · Complexity 5 · Dead Code 8.5 · Error Handling 6.5 · Test Health 6 · Consistency 7 · Performance 5

Test plan

  • CI: tests + ruff green
  • CodeRabbit clean

Summary by CodeRabbit

Release Notes

  • New Features

    • Added self-service password change and admin password reset endpoints for user accounts
    • Added fleet API key reveal feature with owner-only access control
  • Security

    • Credentials and secrets now display as write-only fields in configuration
    • Sessions are invalidated when a user changes their password
    • Implemented SSRF protections for worker URL validation with configurable policies
    • API keys no longer auto-display; revealed only through explicit owner action
  • Improvements

    • Enhanced error handling in earnings collectors for missing or malformed API responses
    • Optimized database connection management with pooling
    • Improved configuration masking to prevent secret exposure

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@GeiserX, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 5 minutes and 31 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7c3fed79-dc4c-4314-a009-8eabdc87647c

📥 Commits

Reviewing files that changed from the base of the PR and between 8f631a2 and cc449d3.

📒 Files selected for processing (42)
  • .gitguardian.yaml
  • .gitignore
  • CHANGELOG.md
  • SECURITY.md
  • app/auth.py
  • app/collectors/anyone.py
  • app/collectors/base.py
  • app/collectors/bitping.py
  • app/collectors/bytelixir.py
  • app/collectors/earnapp.py
  • app/collectors/earnfm.py
  • app/collectors/honeygain.py
  • app/collectors/iproyal.py
  • app/collectors/mystnodes.py
  • app/collectors/proxyrack.py
  • app/collectors/repocket.py
  • app/collectors/salad.py
  • app/collectors/traffmonetizer.py
  • app/database.py
  • app/deps.py
  • app/exchange_rates.py
  • app/main.py
  • app/metrics.py
  • app/orchestrator.py
  • app/routers/__init__.py
  • app/routers/auth.py
  • app/routers/pages.py
  • app/routers/users.py
  • app/static/js/app.js
  • app/templates/base.html
  • app/templates/fleet.html
  • docs/fleet.md
  • docs/getting-started.md
  • services/_schema.yml
  • tests/conftest.py
  • tests/test_auth_extended.py
  • tests/test_collectors_extended.py
  • tests/test_coverage_gaps.py
  • tests/test_exchange_rates.py
  • tests/test_exchange_rates_extended.py
  • tests/test_main_routes.py
  • tests/test_metrics.py
📝 Walkthrough

Walkthrough

Security-focused release hardening authentication (per-user password-epoch invalidation), database (connection pooling, config masking), collectors (strict field validation), worker URLs (SSRF protection with DNS rebinding defense), and API key exposure (lazy reveal). Includes router refactoring, password-change endpoints, and comprehensive test coverage.

Changes

Security hardening and API changes

Layer / File(s) Summary
Per-user password-change session invalidation via epochs
app/auth.py
decode_session_token now catches BadData (replacing BadSignature), checks per-user password-change epoch (_USER_PWD_EPOCH), and rejects tokens issued before a user's epoch in addition to the global session epoch. New set_user_pwd_epoch(uid, changed_at) records when a user changes password.
Shared SQLite connection pooling and config masking
app/database.py
Event-loop–scoped connection caching via _BorrowedConnection; connect_shared() and close_shared() manage lifecycle. Schema indexes added for earnings(created_at) and health_events(created_at). get_config_masked() returns non-secret config as-is plus _secrets map (never leaks secrets). list_users_with_pwd_epoch() pre-warms auth epoch cache. upsert_earnings() uses single INSERT ... ON CONFLICT ... DO UPDATE with balance-change detection.
Strict balance-field validation across collectors
app/collectors/base.py, bitping.py, earnapp.py, earnfm.py, honeygain.py, iproyal.py, mystnodes.py, proxyrack.py, repocket.py, salad.py, traffmonetizer.py, anyone.py, bytelixir.py
BaseCollector becomes ABC with @abc.abstractmethod on collect(). All collectors shift from coercing missing balance fields to 0, to raising ValueError for missing fields; exception handlers return EarningsResult(balance=0.0, error=...). ProxyRack refactors _fetch_balance() as typed helper. Docstrings clarified.
Route handler refactoring and shared dependency injection
app/deps.py, app/routers/auth.py, app/routers/pages.py, app/routers/users.py, app/main.py
New app/deps.py exports shared templates, guard dependencies (_require_auth_api, _require_writer, _require_owner, _require_private_network), and _login_redirect(). New routers extract handlers: auth.py (/login, /register, /logout, /onboarding with rate limiting and first-user gating), pages.py (/, /setup, /catalog, /settings owner-only, /fleet), users.py (GET /api/users, PATCH role updates, DELETE with guards). All reference shared state via app.main.
Worker URL SSRF hardening with policy enforcement and DNS rebinding protection
app/main.py
Comprehensive SSRF guard: blocks metadata/loopback IPs, normalizes IPv4-mapped IPv6, resolves hostnames with DNS-rebinding re-validation, enforces optional strict-mode allowlist (CIDRs/suffixes/exact matches) from CASHPILOT_WORKER_ALLOWED_HOSTS, permits metadata escape hatch via CASHPILOT_WORKER_ALLOW_METADATA. Worker proxy error handlers (command/deploy/logs) log warnings and return uniform HTTPException instead of raw worker errors. Mounts /static, imports routers and deps.
Password-change endpoints with epoch invalidation and session re-minting
app/main.py
POST /api/users/me/password (self-service): verifies current password, enforces min length and non-equality, updates password and per-user epoch, re-mints session cookie. POST /api/users/{user_id}/password (admin reset): owner-only, skips current-password check, updates epoch. Both validate policies and return error details.
Fleet API key lazy-load and owner-only reveal flow
app/main.py, app/templates/fleet.html
GET /api/fleet/api-key returns metadata (is_set, source) without plaintext. New POST /api/fleet/api-key/reveal (owner-only) returns key, audit-logged. Frontend loads key status on page load, fetches plaintext only on explicit reveal via new endpoint. Template masks key initially, renders owner controls (Reveal, Copy).
Config API and env-info masking for secret exposure prevention
app/main.py, app/database.py
GET /api/config returns masked config (non-secrets as-is, secrets as _secrets boolean map). GET /api/env-info treats secret rows as read-only: omits values, reports is_set, special handling for CASHPILOT_SECRET_KEY (always marked set/read-only, never leaked).
Write-only secret inputs and password-change UI
app/static/js/app.js, app/templates/base.html, app/templates/fleet.html
Credential/env-var secret fields render empty values with placeholders from _secrets (set vs unset), add autocomplete="new-password". New Change Password modal with openChangePasswordModal(), submitPasswordChange() (client validation, disable-on-save, error handling, toast). Modal submits on Enter. Collector secret inputs similarly use _secrets. Base template adds password-change dropdown and modal.
Exchange rate staleness detection and refresh warnings
app/exchange_rates.py
New STALE_THRESHOLD and rates_stale() detect when cached rates exceed threshold. refresh() emits logger.warning() on non-200 CoinGecko/Frankfurter responses instead of silently ignoring.
Docker availability detection with positive-only caching
app/orchestrator.py
docker_available() memoizes only successful pings; failures force re-probe on next call (handling transient issues). Shortened docstring. Removed deploy_service() export.
Worker heartbeat staleness metric using last_heartbeat field
app/metrics.py
Switched from last_seen to last_heartbeat (ISO datetime); parses and normalizes UTC timezone.
Comprehensive test coverage for auth epochs, collectors, routes, and features
tests/conftest.py, tests/test_auth_extended.py, tests/test_collectors_extended.py, tests/test_main_routes.py, tests/test_coverage_gaps.py, tests/test_exchange_rates.py, tests/test_exchange_rates_extended.py, tests/test_metrics.py
Autouse fixture drains shared DB connections per test. Auth tests: epoch rejection, per-UID invalidation. Collector tests: ABC enforcement, missing-field error assertions across 10 collectors. Main routes: 500+ lines adding password endpoints, secret masking, fleet key reveal, SSRF validation (metadata/loopback/RFC1918/DNS rebind/strict mode), scheduler wiring, guards, allowlist parsing, rate-limit metrics. Docker caching tests. Exchange-rate staleness and warning tests. Worker heartbeat staleness test.
Documentation and configuration updates
.gitguardian.yaml, .gitignore, CHANGELOG.md, SECURITY.md, docs/fleet.md, docs/getting-started.md, services/_schema.yml, app/main.py
CHANGELOG [Unreleased] section documents security/perf/additions. SECURITY.md adds SSRF section (blocking, DNS rebinding, policy). Fleet docs clarify lazy key reveal and Worker URL Validation. Getting-started adds password-change tip. .gitguardian excludes tests/**. .gitignore adds pytest/coverage/tool artifacts. Schema updates status/referral/Docker/cashout/collector docs. Collection-failure alert uses sanitized entry. Service filtering hides dropped. Lifespan warms services/exchange rates, adds scheduler listener and job hardening.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • GeiserX/CashPilot#16: Worker proxy SSRF hardening and URL validation logic overlaps at app/main.py worker routing.
  • GeiserX/CashPilot#62: EarnFM collector auth/response handling; this PR additionally tightens balance-field requirement.
  • GeiserX/CashPilot#57: Session epoch and token iat validation logic supports this PR's per-user epoch invalidation in app/auth.py.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/codebase-audit-safe-wave

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@GeiserX

GeiserX commented Jun 11, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.95902% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.54%. Comparing base (8f631a2) to head (cc449d3).

Files with missing lines Patch % Lines
app/database.py 52.23% 32 Missing ⚠️
app/main.py 89.80% 16 Missing ⚠️
app/collectors/earnapp.py 75.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #85      +/-   ##
==========================================
+ Coverage   90.99%   91.54%   +0.54%     
==========================================
  Files          26       30       +4     
  Lines        2811     3063     +252     
==========================================
+ Hits         2558     2804     +246     
- Misses        253      259       +6     
Files with missing lines Coverage Δ
app/auth.py 95.87% <100.00%> (+1.55%) ⬆️
app/collectors/anyone.py 93.22% <100.00%> (+0.36%) ⬆️
app/collectors/base.py 95.12% <100.00%> (+0.25%) ⬆️
app/collectors/bitping.py 94.00% <100.00%> (+0.38%) ⬆️
app/collectors/bytelixir.py 79.72% <ø> (ø)
app/collectors/earnfm.py 93.61% <100.00%> (+0.43%) ⬆️
app/collectors/honeygain.py 97.91% <100.00%> (+0.13%) ⬆️
app/collectors/iproyal.py 89.09% <100.00%> (+0.62%) ⬆️
app/collectors/mystnodes.py 96.38% <100.00%> (+0.13%) ⬆️
app/collectors/proxyrack.py 100.00% <100.00%> (ø)
... and 12 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@gitguardian

gitguardian Bot commented Jun 11, 2026

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
33907540 Triggered Username Password d6f1af5 tests/test_main_routes.py View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Comment thread tests/test_main_routes.py Fixed
@GeiserX GeiserX force-pushed the fix/codebase-audit-safe-wave branch 2 times, most recently from 3381d8a to eb38c1a Compare June 11, 2026 14:11
…e, DB pooling, fixes

Consolidated wave from a whole-codebase audit + follow-up research. Full
suite 950 passing, ruff clean. Every behavior change is guarded by a
regression test verified to fail against the pre-fix code.

Security:
- SSRF guard on worker URLs: cloud-metadata IPs (IPv4 169.254.169.254 +
  IPv6 fd00:ec2::254) always blocked; IPv6 loopback/link-local + IPv4-mapped
  (::ffff:) bypasses closed; DNS-rebinding guard resolves hostnames and
  re-validates the resolved IP before every outbound request. Default policy
  "permissive" keeps LAN (RFC1918) + Tailscale (CGNAT 100.64.0.0/10) workers
  working with zero config; opt-in "strict" mode via CASHPILOT_WORKER_URL_POLICY
  + CASHPILOT_WORKER_ALLOWED_HOSTS (CIDRs + *.suffix); CASHPILOT_WORKER_ALLOW_METADATA
  escape hatch.
- Write-only secrets: GET /api/config returns a {key: is_set} map (no decrypted
  values); /api/env-info drops secret values (CASHPILOT_SECRET_KEY never leaves
  the server); fleet key revealed only via owner-only POST /api/fleet/api-key/reveal
  (audit-logged, uid only).
- Change-password + session invalidation: self-service POST /api/users/me/password
  (re-mints caller cookie) + owner reset POST /api/users/{id}/password; a change
  bumps a per-user epoch (warmed at startup) so older tokens for that user are
  rejected.

Performance:
- SQLite connection sharing (one connection per event loop via borrowed-handle
  proxy) — ~3x faster dashboard loads, far less write contention. Per-loop
  keying preserves test isolation.

Audit fixes (bugs / dead code / consistency):
- Collectors: missing balance field surfaces an error instead of persisting a
  fake 0.0 (11 collectors); BaseCollector is a real ABC; proxyrack returns a
  consistent Response.
- metrics: read the real last_heartbeat column (was last_seen) so dead-worker
  alerting fires.
- Scheduler: error/missed-job listener + max_instances/coalesce/misfire_grace_time.
- _run_collection publishes a synthetic alert on crash (no green bell during an
  outage). Worker-proxy errors no longer leak internal URLs into HTTP detail.
- auth: narrowed except (BadSignature, Exception) to BadData.
- database: created_at indexes for retention purge; single upsert_earnings.
- _schema.yml: documented status:dropped, cashout.method:auto, collector.type:auto,
  docker.privileged; removed dead referral.bonus; dropped services hidden from catalog.
- exchange_rates: rates_stale() + non-200 logging. Deleted dead orchestrator
  deploy_service/reset_docker_status, main _require_auth, JS worker-cache +
  goToCollectorSettings.

Refactor:
- Partial router split: users/auth/pages → app/routers/* with shared deps in
  app/deps.py, calling shared symbols through app.main so patch seams hold and
  no auth guard is lost. Fleet/services/earnings/deploy/config stay in main.py
  for a dedicated follow-up PR.

Docs: CHANGELOG, SECURITY.md SSRF section, fleet.md env vars, getting-started.
Adds ~150 regression tests (881 -> 950).
@GeiserX GeiserX force-pushed the fix/codebase-audit-safe-wave branch from eb38c1a to d6f1af5 Compare June 11, 2026 14:29
@GeiserX GeiserX merged commit bc23621 into main Jun 11, 2026
7 of 8 checks passed
@GeiserX GeiserX deleted the fix/codebase-audit-safe-wave branch June 11, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants