Skip to content

Commit baa1aba

Browse files
test(worker): Tier-1 job integration round-trips (billing/deletion) — toward worker coverage floor (#89)
* test(worker): Tier-1 job integration round-trips (billing/deletion) Add real-Postgres integration round-trips for the two highest-value Tier-1 (data-loss / money-adjacent) worker jobs flagged in INTEGRATION-COVERAGE-PLAN-2026-06-04.md §5 #1, building on the Wave-1 testhelpers harness (#87) and the non-short CI integration gate (#88). billing_reconciler — terminal downgrade: - seed a real pro team with a Razorpay subscription id, run Work() with a sub-keyed stub fetcher reporting `cancelled`, assert the LIVE row flipped plan_tier pro→hobby (terminalDowngradeTier, never 'free' — D28 F1) AND a subscription.canceled audit_log row was emitted. - complement: an `active`-at-current-tier team is left untouched (no spurious UPDATE / upgrade email every tick). team_deletion_executor — purge cascade (the §5 #1 single biggest gap): - seed a team past its 30-day grace window + secret-bearing resources + PII-bearing users + a deployment, run Work() with provisioner/S3/k8s nil (fail-open), assert the persisted cascade: status→tombstoned, resources.connection_url→NULL / key_prefix→'', users.email scrubbed to the deleted-<id>@tombstoned.invalid placeholder, team.tombstoned audit emitted. - complement: an in-grace team (5 days) is NOT swept — secrets intact, no tombstone (truehomie-class "destroy ran when it shouldn't" guard). Gate-filter finding: all functions carry "Integration" so the #88 gate (`go test ./internal/... -run Integration`) executes them; the Wave-1 #87 tests already match (TestIntegration_* / TestPropagation_*Integration*) so NO rename was needed. All seeds use testhelpers.SetupTestDB's -short/ no-DB skip guard, so `make gate` stays green and they run only under the non-short integration job. Harness additions (internal/testhelpers/billing_deletion.go): team sub/tier/status + audit-by-team + pending-deletion + secret-resource + user seeds/readbacks. SeedTeamWithSubscription seeds a low-sorting team id so the reconciler's ORDER BY id LIMIT 100 candidate scan always reaches it regardless of DB pollution (deterministic on fresh CI and busy local DB). Integration-cov delta (jobs pkg, -run Integration, -coverpkg jobs): 5.6% → 8.3% (+2.7pp). Touched: billing_reconciler.Work 45.3%, updatePlanTier 75%, emitCancelAudit 66.7%; team_deletion_executor.Work 76.2%, processTeam 40.8%, emitTombstoned 80%, fetchCandidates 83.3% (all 0% from the integration suite before). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(testhelpers): in-package coverage for billing_deletion.go harness billing_deletion.go is exercised only by internal/jobs integration tests, so Go credits its line coverage to internal/jobs, not internal/testhelpers — the diff-cover --fail-under=100 patch gate read it as 0% (142 missing lines), reddening PR #89's coverage check. Same root cause/fix as the merged #87 testhelpers.go fix. Add billing_deletion_smoke_test.go giving the harness its own in-package coverage: a DB-backed happy-path round-trip (gated on DB reachability via SetupTestDB, runs in coverage.yml which exports TEST_DATABASE_URL + applies api migrations), a closed-DB error-arms test, lowSortingTeamID unit coverage, and a withoutGithubIDColumn seam exercising SeedUser's fallback INSERT branch. Route each helper's failure arm through the package tFatalf seam (matching testhelpers.go) so the error arms are coverable via seam-swap — a test seam, not a behavioural change; real callers still get genuine t.Fatalf. billing_deletion.go is now 100.0% covered in-package (go tool cover -func). No threshold lowered, no waiver added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent e9d618a commit baa1aba

4 files changed

Lines changed: 780 additions & 0 deletions

File tree

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
package jobs
2+
3+
// billing_reconciler_integration_test.go — REAL-Postgres integration test for
4+
// the billing reconciler's terminal-downgrade round-trip.
5+
//
6+
// Per INTEGRATION-COVERAGE-PLAN-2026-06-04.md §5 #1, billing_reconciler is a
7+
// Tier-1 (money-adjacent) job whose DB round-trip was fake-tested only — the
8+
// sibling billing_reconciler_test.go drives Work() with a stubGrace and asserts
9+
// the stub's call counts, never the persisted teams row. THIS test seeds a real
10+
// paid (pro) team carrying a Razorpay subscription id, runs Work() against a
11+
// live platform Postgres with a stub fetcher that reports the subscription
12+
// `cancelled` (a terminal status), and asserts the worker actually wrote:
13+
//
14+
// - teams.plan_tier flipped pro → "hobby" (terminalDowngradeTier — never the
15+
// ephemeral "free" tier; the cross-repo contract D28 F1), AND
16+
// - a subscription.canceled audit_log row was emitted for the team (the
17+
// event the email forwarder dispatches the cancellation email from).
18+
//
19+
// The Razorpay leg stays stubbed (no live Razorpay — CLAUDE P0: recurring not
20+
// enabled, plan §4 BLOCKED-bypass: drive the state directly). The DB leg —
21+
// the candidate SELECT, the UpdatePlanTier UPDATE, the audit INSERT — is REAL.
22+
// This is the convergence a sqlmock test cannot prove: that the downgrade
23+
// UPDATE's WHERE id = $2 matched the live row and the audit row landed.
24+
//
25+
// Determinism on a shared local DB: the reconciler scans EVERY team with a
26+
// non-empty stripe_customer_id, not just ours. The stub fetcher therefore
27+
// keys on subscription_id — it returns `cancelled` ONLY for the seeded team's
28+
// subscription and a benign `active`-at-hobby (a no-op for any already-hobby
29+
// team) for everything else, so the sweep cannot mutate unrelated rows in a way
30+
// that perturbs this assertion. grace is left nil so the real dbGracePeriodOpener
31+
// runs against the live DB (its TerminateActiveGracePeriod is an idempotent
32+
// no-op when the team has no active grace row).
33+
//
34+
// GATING: testhelpers.SetupTestDB skips under -short / no-DB, so the regular
35+
// `make gate` stays green without a Postgres; the non-short CI integration job
36+
// runs it for real.
37+
38+
import (
39+
"context"
40+
"testing"
41+
42+
"github.com/google/uuid"
43+
"github.com/riverqueue/river"
44+
"github.com/riverqueue/river/rivertype"
45+
46+
"instant.dev/worker/internal/testhelpers"
47+
)
48+
49+
// subKeyedFetcher is a subscriptionFetcher that returns a caller-supplied
50+
// status for ONE target subscription id and a benign no-op status (`active` at
51+
// the lowest paid tier) for every other id. Keyed-by-sub so the reconciler's
52+
// full-table sweep cannot perturb the assertion on a shared DB.
53+
type subKeyedFetcher struct {
54+
targetSub string
55+
targetStatus string
56+
planID string
57+
}
58+
59+
func (f *subKeyedFetcher) FetchSubscriptionForReconciler(_ context.Context, subID string) (*reconcilerSubscriptionDetails, error) {
60+
if subID == f.targetSub {
61+
return &reconcilerSubscriptionDetails{
62+
Status: f.targetStatus,
63+
PlanID: f.planID,
64+
PaidCount: 1,
65+
}, nil
66+
}
67+
// Any other team in the shared DB: report `created` — a guaranteed
68+
// no-action status (rzpStatusClassNoAction). This is strictly safer than
69+
// `active`, which (with an unrecognised plan_id) resolves to "hobby" and
70+
// could upgrade an anonymous/free team that happens to carry a
71+
// subscription id in a polluted local DB. `created` mutates nothing, so
72+
// the sweep touches ONLY the seeded target team.
73+
return &reconcilerSubscriptionDetails{Status: "created", PlanID: "", PaidCount: 0}, nil
74+
}
75+
76+
func fakeBillingJob() *river.Job[BillingReconcilerArgs] {
77+
return &river.Job[BillingReconcilerArgs]{JobRow: &rivertype.JobRow{ID: 1}}
78+
}
79+
80+
// TestIntegration_BillingReconciler_TerminalDowngradePersistsHobby seeds a
81+
// pro-tier team with a Razorpay subscription id, runs the reconciler with a
82+
// fetcher that reports that subscription `cancelled`, and asserts the worker
83+
// persisted plan_tier='hobby' to the REAL team row AND emitted the
84+
// subscription.canceled audit row.
85+
func TestIntegration_BillingReconciler_TerminalDowngradePersistsHobby(t *testing.T) {
86+
db, cleanup := testhelpers.SetupTestDB(t)
87+
defer cleanup()
88+
89+
subID := "sub_itest_" + uuid.New().String()[:12]
90+
teamID := testhelpers.SeedTeamWithSubscription(t, db, "pro", subID)
91+
92+
fetcher := &subKeyedFetcher{targetSub: subID, targetStatus: "cancelled"}
93+
// grace == nil → real dbGracePeriodOpener against the live DB.
94+
w := NewBillingReconcilerWorker(db, fetcher, nil)
95+
96+
if err := w.Work(context.Background(), fakeBillingJob()); err != nil {
97+
t.Fatalf("Work: %v", err)
98+
}
99+
100+
// The team's plan_tier must have flipped pro → hobby (terminalDowngradeTier).
101+
got := testhelpers.TeamPlanTier(t, db, teamID)
102+
if got != terminalDowngradeTier {
103+
t.Errorf("plan_tier = %q after terminal downgrade, want %q (terminalDowngradeTier)", got, terminalDowngradeTier)
104+
}
105+
if got == "free" {
106+
t.Error("terminal downgrade landed on 'free' — must be 'hobby' (D28 F1: 'free' strands permanent paid resources)")
107+
}
108+
109+
// A subscription.canceled audit row must have been emitted for the team —
110+
// this is what the event-email forwarder dispatches the cancellation
111+
// confirmation email from.
112+
if n := testhelpers.CountAuditLogByTeam(t, db, teamID, "subscription.canceled"); n < 1 {
113+
t.Errorf("subscription.canceled audit rows = %d, want >= 1 (the downgrade must emit the cancellation audit)", n)
114+
}
115+
}
116+
117+
// TestIntegration_BillingReconciler_ActiveNoDriftLeavesTierUntouched is the
118+
// complement: a paid team Razorpay reports `active` at its CURRENT tier must
119+
// NOT be re-written (no gap → continue) and NO audit row is emitted. This pins
120+
// the live "DB tier already at/above expected" no-op branch against real data —
121+
// a regression that always-writes would burn a tier UPDATE + a spurious
122+
// upgrade email every 15-minute tick. It is also fast: the target sorts first
123+
// and the stub reports a no-action for everything else.
124+
func TestIntegration_BillingReconciler_ActiveNoDriftLeavesTierUntouched(t *testing.T) {
125+
db, cleanup := testhelpers.SetupTestDB(t)
126+
defer cleanup()
127+
128+
subID := "sub_itest_" + uuid.New().String()[:12]
129+
teamID := testhelpers.SeedTeamWithSubscription(t, db, "pro", subID)
130+
131+
// Razorpay reports active; with an empty plan_id the expected tier resolves
132+
// to "hobby" — and the team is already "pro" (>= hobby) → no-op.
133+
fetcher := &subKeyedFetcher{targetSub: subID, targetStatus: "active"}
134+
w := NewBillingReconcilerWorker(db, fetcher, nil)
135+
136+
if err := w.Work(context.Background(), fakeBillingJob()); err != nil {
137+
t.Fatalf("Work: %v", err)
138+
}
139+
140+
// Tier untouched (still pro — NOT downgraded to the resolved-hobby).
141+
if got := testhelpers.TeamPlanTier(t, db, teamID); got != "pro" {
142+
t.Errorf("plan_tier = %q for an at-or-above-expected team, want \"pro\" (untouched)", got)
143+
}
144+
// No upgrade / cancel audit emitted for a no-op.
145+
if n := testhelpers.CountAuditLogByTeam(t, db, teamID, "subscription.upgraded"); n != 0 {
146+
t.Errorf("subscription.upgraded audit rows = %d for a no-op tick, want 0", n)
147+
}
148+
if n := testhelpers.CountAuditLogByTeam(t, db, teamID, "subscription.canceled"); n != 0 {
149+
t.Errorf("subscription.canceled audit rows = %d for a no-op tick, want 0", n)
150+
}
151+
}
Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
package jobs
2+
3+
// team_deletion_executor_integration_test.go — REAL-Postgres integration test
4+
// for the team-deletion purge cascade. This is the §5 #1 single highest-value
5+
// data-loss gap: GDPR Article 17 right-to-be-forgotten teardown that, if it
6+
// silently fails or scrubs the wrong rows, is both a compliance breach and a
7+
// data-loss incident (cf. the 2026-06-03 truehomie-db DROP incident class).
8+
//
9+
// The sibling team_deletion_executor_test.go drives Work() with fake S3 / k8s
10+
// clients and asserts the destruction *attempts* against those fakes. THIS test
11+
// seeds a real team whose 30-day grace window has elapsed, with real resources
12+
// (carrying connection_url + key_prefix secrets), real users (carrying PII),
13+
// and a real deployment, then runs Work() against a live platform Postgres with
14+
// provisioner/S3/k8s all nil (fail-open per NewTeamDeletionExecutorWorker) and
15+
// asserts the PERSISTED purge cascade:
16+
//
17+
// - teams.status flipped deletion_requested → tombstoned, tombstoned_at set,
18+
// - resources.connection_url scrubbed to NULL, key_prefix blanked to '',
19+
// - users.email scrubbed to the deleted-<id>@tombstoned.invalid placeholder
20+
// (NULL would violate the NOT NULL UNIQUE constraint),
21+
// - a team.tombstoned audit_log row emitted.
22+
//
23+
// The S3 / k8s / gRPC legs are nil-skipped (CI has no bucket / cluster /
24+
// provisioner; the job is fail-open by design). The DB leg — the candidate
25+
// scan (deletion_requested_at + 30d < now()), the status flip, the
26+
// scrub transaction, the audit emit — is REAL and is the integration target.
27+
// A sqlmock test passes whether or not the scrub UPDATE's WHERE team_id = $1
28+
// matched the seeded rows; this proves it did, against the real schema's
29+
// constraints (the users.email NOT NULL UNIQUE that forces the placeholder
30+
// instead of a NULL).
31+
//
32+
// GATING: testhelpers.SetupTestDB skips under -short / no-DB.
33+
34+
import (
35+
"context"
36+
"testing"
37+
38+
"github.com/riverqueue/river"
39+
"github.com/riverqueue/river/rivertype"
40+
41+
"instant.dev/worker/internal/testhelpers"
42+
)
43+
44+
func fakeTeamDeletionJob() *river.Job[TeamDeletionExecutorArgs] {
45+
return &river.Job[TeamDeletionExecutorArgs]{JobRow: &rivertype.JobRow{ID: 1}}
46+
}
47+
48+
// TestIntegration_TeamDeletionExecutor_PurgeCascadeTombstones seeds a team
49+
// past its 30-day grace window with secret-bearing resources, PII-bearing
50+
// users, and a deployment, runs the executor with no external clients, and
51+
// asserts the full persisted tombstone cascade.
52+
func TestIntegration_TeamDeletionExecutor_PurgeCascadeTombstones(t *testing.T) {
53+
db, cleanup := testhelpers.SetupTestDB(t)
54+
defer cleanup()
55+
56+
// Grace window elapsed (40 > 30 days) → the candidate scan selects it.
57+
teamID := testhelpers.SeedTeamPendingDeletion(t, db, "pro", 40)
58+
resID := testhelpers.SeedResourceWithSecret(t, db, teamID, "postgres")
59+
userID := testhelpers.SeedUser(t, db, teamID, "purge-itest-"+teamID.String()[:8]+"@example.com")
60+
// A deployment so fetchTeamDeployAppIDs has a row to enumerate (the k8s
61+
// leg is nil-skipped, but the row must not break the pure-DB cascade).
62+
_ = testhelpers.SeedDeployment(t, db, teamID, "healthy", "app-purge-itest")
63+
64+
// provisioner / s3 / k8s all nil → fail-open; pure-DB cascade only.
65+
w := NewTeamDeletionExecutorWorker(db, nil, nil, nil, "")
66+
67+
if err := w.Work(context.Background(), fakeTeamDeletionJob()); err != nil {
68+
t.Fatalf("Work: %v", err)
69+
}
70+
71+
// 1. Team tombstoned.
72+
if got := testhelpers.TeamStatus(t, db, teamID); got != "tombstoned" {
73+
t.Errorf("team status = %q after purge, want \"tombstoned\"", got)
74+
}
75+
76+
// 2. Resource secrets scrubbed.
77+
connURL, keyPrefix := testhelpers.ResourceSecretFields(t, db, resID)
78+
if connURL.Valid {
79+
t.Errorf("resource connection_url = %q after purge, want NULL", connURL.String)
80+
}
81+
if keyPrefix != "" {
82+
t.Errorf("resource key_prefix = %q after purge, want empty", keyPrefix)
83+
}
84+
85+
// 3. User PII scrubbed to the tombstone placeholder (NOT the original email,
86+
// NOT NULL — the NOT NULL UNIQUE constraint forces the placeholder).
87+
email := testhelpers.UserEmail(t, db, userID)
88+
wantPrefix := "deleted-" + userID.String()
89+
if email != wantPrefix+"@tombstoned.invalid" {
90+
t.Errorf("user email = %q after purge, want %q@tombstoned.invalid", email, wantPrefix)
91+
}
92+
93+
// 4. team.tombstoned audit row emitted.
94+
if n := testhelpers.CountAuditLogByTeam(t, db, teamID, auditKindTombstoned); n < 1 {
95+
t.Errorf("%s audit rows = %d, want >= 1", auditKindTombstoned, n)
96+
}
97+
}
98+
99+
// TestIntegration_TeamDeletionExecutor_WithinGraceNotPurged is the safety
100+
// complement: a team still INSIDE its 30-day grace window (the customer can
101+
// still restore) MUST NOT be tombstoned. This pins the candidate scan's time
102+
// predicate against real data — a regression that ignores the grace window
103+
// would destroy data a customer is legally entitled to recover. Asserting the
104+
// no-op is as important as asserting the purge: the truehomie-class incident is
105+
// "the destroy path ran when it should not have."
106+
func TestIntegration_TeamDeletionExecutor_WithinGraceNotPurged(t *testing.T) {
107+
db, cleanup := testhelpers.SetupTestDB(t)
108+
defer cleanup()
109+
110+
// Only 5 days elapsed (< 30) → still restorable → must NOT be swept.
111+
teamID := testhelpers.SeedTeamPendingDeletion(t, db, "pro", 5)
112+
resID := testhelpers.SeedResourceWithSecret(t, db, teamID, "postgres")
113+
114+
w := NewTeamDeletionExecutorWorker(db, nil, nil, nil, "")
115+
116+
if err := w.Work(context.Background(), fakeTeamDeletionJob()); err != nil {
117+
t.Fatalf("Work: %v", err)
118+
}
119+
120+
// Team still in deletion_requested (untouched), NOT tombstoned.
121+
if got := testhelpers.TeamStatus(t, db, teamID); got != "deletion_requested" {
122+
t.Errorf("team status = %q for an in-grace team, want \"deletion_requested\" (must not be swept)", got)
123+
}
124+
125+
// Resource secret still present (NOT scrubbed).
126+
connURL, _ := testhelpers.ResourceSecretFields(t, db, resID)
127+
if !connURL.Valid || connURL.String == "" {
128+
t.Error("resource connection_url was scrubbed for an in-grace team — the grace window predicate failed (data-loss regression)")
129+
}
130+
131+
// No tombstone audit.
132+
if n := testhelpers.CountAuditLogByTeam(t, db, teamID, auditKindTombstoned); n != 0 {
133+
t.Errorf("%s audit rows = %d for an in-grace team, want 0", auditKindTombstoned, n)
134+
}
135+
}

0 commit comments

Comments
 (0)