Speed up dummy-project seeding (preview create-project ~15s → ~1.3s)#1437
Open
BilalG1 wants to merge 12 commits into
Open
Speed up dummy-project seeding (preview create-project ~15s → ~1.3s)#1437BilalG1 wants to merge 12 commits into
BilalG1 wants to merge 12 commits into
Conversation
Replace the per-user CRUD-handler calls in seedDummyUsers with bulk createMany inserts, plus several smaller wins on the seeding path. End-to-end the preview create-project endpoint drops from ~15s to ~1.3s. - seedDummyUsers: build every row up front and insert via one createMany per table inside a single transaction, instead of ~86 sequential adminCreate transactions. Named-user team memberships are bulk-inserted the same way. Idempotency is preserved via a single up-front email lookup, so re-runs against an existing project still skip existing users. - Use node:crypto randomUUID instead of stack-shared generateUuid in the seed paths. The browser-safe polyfill calls crypto.getRandomValues ~31x per UUID, which dominated CPU time when generating thousands of seed UUIDs (~800ms in the activity-event build alone). - seedBulkSignupsAndActivity: skip the redundant back-date UPDATE for freshly-inserted users (createMany already writes correct timestamps), and flush ClickHouse events in larger, parallel batches. - seedDummyProject: run seedBulkSignupsAndActivity concurrently with the lighter remaining steps, and fold seedDummyTransactions into the emails/activity/replays Promise.all. - Remove the now-unused syncSeedUserOauthProviders helper.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughRefactors backend seed scripts for deterministic bulk user creation and idempotent bulk inserts, increases ClickHouse batch sizes and concurrency, narrows backdating to existing users, overlaps bulk activity seeding with project setup, validates preview internals and refreshes owned-projects before navigation, guards layout auto-login with a ref, and adjusts CLI whoami auth resolution. ChangesSeed Data Refactoring & Performance Optimization
Preview Project Redirect Validation
Layout Client Auto-login Guard
Walkthrough UI Wiring & Labels
CLI whoami
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Greptile SummaryThis PR replaces per-user sequential
Confidence Score: 3/5The bulk-insert path produces correct data under the happy path, but the floated The new concurrent execution model fires apps/backend/src/lib/seed-dummy-data.ts — specifically the Important Files Changed
Sequence DiagramsequenceDiagram
participant C as seedDummyProject
participant PU as seedDummyUsers (bulk)
participant BSA as seedBulkSignupsAndActivity
participant PA1 as Promise.all #1 (config overrides)
participant PA2 as Promise.all #2 (emails/txns/replays)
participant CH as ClickHouse
C->>PU: await (sequential — heavy Postgres)
PU-->>C: userEmailToId Map
C->>BSA: fire (Promise, not awaited yet)
activate BSA
C->>PA1: await Promise.all [overrideBranchConfig, overrideEnvConfig, ...]
PA1-->>C: done (or throws ⚠️ — bulkSignupsPromise abandoned)
C->>PA2: await Promise.all [seedDummyTransactions, seedDummyEmails, ...]
PA2-->>C: done (or throws ⚠️ — bulkSignupsPromise abandoned)
BSA->>BSA: createMany ProjectUsers
BSA->>CH: Promise.all batch inserts (10 000-row batches, parallel)
CH-->>BSA: ack
BSA-->>C: resolves
C->>C: await bulkSignupsPromise (line 2092)
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional Suggestion:
seedBulkSignupsAndActivity function creates ProjectUser and ContactChannel rows for new users but does not create corresponding ProjectUserDirectPermission rows with default permissions, breaking expected permission behavior for bulk-seeded users.
In preview mode the dashboard's /projects page renders PreviewProjectRedirect, which POSTs /internal/preview/create-project and then router.push()es to /projects/<new-id>. It never refreshed the client-side owned-projects cache, so the [projectId] route's useAdminApp() read a stale list, failed to find the just-created project, and called notFound() — showing a 404. Refresh the owned-projects cache before navigating, matching what the normal create-project flow in page-client.tsx already does.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/dashboard/src/app/(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx (1)
16-27: ⚡ Quick winAvoid
any/asfor internals extraction; use a type guard instead.Line 16 and Line 24-27 currently bypass the type system. Please narrow from
unknownvia a runtime type guard and return the narrowed value directly.Proposed refactor
+type AppInternals = { + sendRequest: (path: string, options: RequestInit, type: string) => Promise<Response>; + refreshOwnedProjects: () => Promise<void>; +}; + +function isAppInternals(value: unknown): value is AppInternals { + return ( + typeof value === "object" && + value !== null && + "sendRequest" in value && + typeof (value as { sendRequest?: unknown }).sendRequest === "function" && + "refreshOwnedProjects" in value && + typeof (value as { refreshOwnedProjects?: unknown }).refreshOwnedProjects === "function" + ); +} + const appInternals = useMemo(() => { - const internals = Reflect.get(app as any, stackAppInternalsSymbol); - if ( - !internals || - typeof internals.sendRequest !== "function" || - typeof internals.refreshOwnedProjects !== "function" - ) { + const internals = Reflect.get(app, stackAppInternalsSymbol); + if (!isAppInternals(internals)) { throw new Error("The Stack client app cannot send internal requests."); } - return internals as { - sendRequest: (path: string, options: RequestInit, type: string) => Promise<Response>, - refreshOwnedProjects: () => Promise<void>, - }; + return internals; }, [app]);As per coding guidelines: "
**/*.{ts,tsx}: Do NOT useas/any/type casts ... unless you specifically asked the user about it."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/app/`(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx around lines 16 - 27, Replace the unchecked cast around Reflect.get(app as any, stackAppInternalsSymbol) by treating the result as unknown and validating it with a runtime type guard: implement a function isStackAppInternals(value: unknown): value is { sendRequest: (path: string, options: RequestInit, type: string) => Promise<Response>; refreshOwnedProjects: () => Promise<void>; } that checks value is an object and both sendRequest and refreshOwnedProjects are functions, then call Reflect.get(...) into a const internals: unknown, run the guard and throw the same error if it returns false, and finally return the narrowed internals (no use of any/as) so the return type is inferred from the type predicate; reference symbols: stackAppInternalsSymbol, internals, isStackAppInternals, sendRequest, refreshOwnedProjects.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@apps/dashboard/src/app/`(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx:
- Around line 16-27: Replace the unchecked cast around Reflect.get(app as any,
stackAppInternalsSymbol) by treating the result as unknown and validating it
with a runtime type guard: implement a function isStackAppInternals(value:
unknown): value is { sendRequest: (path: string, options: RequestInit, type:
string) => Promise<Response>; refreshOwnedProjects: () => Promise<void>; } that
checks value is an object and both sendRequest and refreshOwnedProjects are
functions, then call Reflect.get(...) into a const internals: unknown, run the
guard and throw the same error if it returns false, and finally return the
narrowed internals (no use of any/as) so the return type is inferred from the
type predicate; reference symbols: stackAppInternalsSymbol, internals,
isStackAppInternals, sendRequest, refreshOwnedProjects.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 46e10b70-cafa-45a2-958a-9199f2c52675
📒 Files selected for processing (1)
apps/dashboard/src/app/(main)/(protected)/(outside-dashboard)/projects/preview-project-redirect.tsx
LayoutClient's auto-login effect had no run-once guard. React StrictMode (on by default in Next dev) double-invokes effects, and both invocations ran while `user` was still null — so in preview mode each generated a fresh `preview-*@Preview.stack-auth.com` email and signed up a *separate* preview user. With two users created per load, the session settles on one while the preview project may have been created for the other, so the project page renders a 404 ("does not have access to it"). Guard the effect with a ref so the auto-login (and preview-user creation) runs at most once per mount.
N2D4
approved these changes
May 19, 2026
Conflict in apps/dashboard/src/app/(main)/(protected)/layout-client.tsx
resolved by combining both sides: keep dev's `useUser({ or: "anonymous-if-exists[deprecated]" })`
option and remote-development-environment early return, plus this branch's
`autoLoginStarted` ref guard that prevents StrictMode from minting duplicate
preview users. Dropped dev's redundant `if (user) return;` inside autoLogin
since the existing `if (user || autoLoginStarted.current) return;` covers it.
…hoami `resolveSessionAuth` takes no arguments; the trailing `flags` was breaking typecheck.
The 7-step walkthrough tour shown in preview mode (NEXT_PUBLIC_STACK_IS_PREVIEW) had several silently-skipping steps after recent dashboard changes: - Overview revamp (#1238) dropped the data-walkthrough hooks for the globe and metrics widgets — re-attach them to the new GlobeSectionWithData column and the AnalyticsChartWidget column in metrics-page.tsx. - Session-replays/data-grid PR (#1424) renamed the anchor to "analytics-replays" and the sidebar nav was relabeled from "Session Replays" to "Replays" (under a collapsed Analytics group) — update the step id and sidebarNavLabel. - The sidebar's collapsible-group expand button is now a <button aria-expanded> *nested inside* the prev-sibling header div, not the prev sibling itself — navigateViaSidebar's expansion logic only matched the prevSibling directly, so step 7 silently skipped. Look inside prevSibling for the chevron too. - The cmdkSearch strings "Authentication" / "Emails" / "Payments" were matching the wrong CmdK result first (auth-methods / email-themes / payments/products/new) — tighten them to "Users" / "Emails sent" / "Products" so they land on the pages that actually have the spotlight anchors. - Loop the walkthrough back to step 1 after step 7 instead of stopping; add sidebarNavLabel "Overview" to step 1 so the loop can navigate back from /session-replays. Verified end-to-end in a real browser: all 7 spotlights render and the loop continues indefinitely across iterations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The internal
preview/create-projectendpoint was taking ~15s becauseseedDummyProjectcreated its dummy users one at a time through the fullusersCrudHandlers.adminCreateCRUD pipeline (one DB transaction + config render per user, ~86 users). This reworks the seeding path to use bulk inserts.End-to-end, the endpoint's server-side handler time drops from ~15,100ms → ~1,300ms (~11× faster).
Seeding changes (
seed-dummy-data.ts)seedDummyUsers— bulk insert. Build every row (ProjectUser,ContactChannel,AuthMethod,ProjectUserOAuthAccount,OAuthAuthMethod, default permissions) up front with pre-generated UUIDs, then insert via onecreateManyper table inside a single transaction — replacing ~86 sequentialadminCreatetransactions. Named-user team memberships are bulk-inserted the same way (TeamMember+TeamMemberDirectPermission). Idempotency is preserved with a single up-front email lookup, so re-runs against an existing project still skip existing users.randomUUID. The seed paths now usenode:crypto'srandomUUID()instead of stack-shared'sgenerateUuid(). The browser-safe polyfill callscrypto.getRandomValues~31× per UUID (once per template char, each with a freshUint8Array(1)); generating thousands of seed UUIDs made that ~800ms of pure CPU in the activity-event build alone.seedBulkSignupsAndActivity. Skip the redundant back-dateUPDATEfor freshly-inserted users (createManyalready writes correctcreatedAt/signedUpAt), and flush ClickHouse events in larger, parallel batches.seedDummyProject. RunseedBulkSignupsAndActivityconcurrently with the lighter remaining steps, and foldseedDummyTransactionsinto the emails/activity/replaysPromise.all.syncSeedUserOauthProvidershelper.The bulk path produces the same rows as the CRUD-handler path (verified row-count equality during development). Webhooks / soft-limit checks are intentionally not fired for seed data, consistent with the rest of the seed.
Also in this PR — preview-mode 404 fix (
preview-project-redirect.tsx)While testing the above, the dashboard 404'd right after a preview project was created. In preview mode the
/projectspage rendersPreviewProjectRedirect, whichPOSTs/internal/preview/create-projectand thenrouter.push()es to/projects/<new-id>— but it never refreshed the client-side owned-projects cache, so the[projectId]route'suseAdminApp()read a stale list, failed to find the just-created project, and callednotFound().Fixed by refreshing the owned-projects cache before navigating, matching what the normal create-project flow in
page-client.tsxalready does. (Pre-existing bug, not caused by the seeding change — but it surfaces the seeding path, so it's bundled here.)Testing
pnpm typecheckandpnpm lintpass for both backend and dashboard. The preview endpoint was exercised repeatedly during development (HTTP 200, projects created and populated correctly).Summary by CodeRabbit
Performance
Refactor
Bug Fixes
UI/UX
Chore