[Refactor][Feat][Fix] Rework Email Section With New Sent Page, Better Drafts Page, and Settings Page (#1221)

### Context

We didn't have an easy place for a user to see their domain statistics
and track their sent emails, either overall or by draft. Additionally,
there was scope creep with the sidebar, where we were supporting more
pages. Our emails landing page was also rather confusing, especially
toggling/ working with different email server types. So, we decide to
add a "sent" page, to track email logs and email statistics, as well as
let users temporarily override their sending limits if need be.
Additionally, a user may want to see a particular email in more detail:
what stage is it in? How did it proceed through time? How can I pause
the sending of this email or change the scheduled time or edit the code?
We allow for that to happen.

### Summary of Changes

#### New Pages
1. **Sent Page:** A Domain Reputation card lets you track how many of
your sent emails were bounced or marked as spam as well as how much
capacity you have left. We also provide a temporary override, where you
can use up to 4 times your capacity for a limited period of time.
Additionally, we provide an email log that lets you see the recently
sent emails. You can also toggle this view from a "list all emails" to
"group by template/draft" which shows stats for each template/draft id
(i.e a bar showing how many emails were sent, are pending, were marked
as spam, were bounced etc, and the total number of emails sent with that
template or draft). Clicking on an email in the list all view takes you
to the "email-viewer" endpoint for that email (see below). Clicking on a
template/draft in the group by view takes you to a page where you can
see the statistics for that template/draft in more detail (the "send"
stage view for that template/draft, as referenced below).
2. **Settings Page:** This is a new page we created because the old
"emails" landing page wasn't doing its job. This page is to track all
the email settings. Currently, we put in 2 sections. A "theme settings"
card where users can see their active theme and click on a button to be
navigated to the themes page. This is necessary as we remove themes from
the sidebar. The other section is a card for email server and domain
configuration - you can change your server type and adjust the settings
or send a test email. It's cleaner and less noisy.
3. **Drafts Page**: There are a lot of changes here. On the landing
page, we actually separate out the drafts into "active drafts" and
"draft history" because drafts are meant to be fire-and-forget, not
reusable. We also add the functionality to create a draft from a
template. This was tricky to manage because templates rely on template
variables which sent to the backend along with the code and injected
during render time. We deal with this by having AI rewrite the template
source code to remove any references to template variables and to make
the draft standalone. The drafts page has been separated into a
stepper-controlled multi stage process:
draft->recipients->schedule->sent. Sent is a read only view that shows
you the statistics of the emails sent using that draft, as mentioned
earlier. You can also see the sent view of a historical draft. You can
also bulk pause/cancel any unsent emails from the sent view of the
drafts.
4. **Sidebar Updates**: The email sidebar now doesn't show "themes" or
"emails" (the old landing page), but it does show "settings" and "sent",
and the default landing page for emails is "sent".
5. **Email Viewer**: When you click on an individual email, you get
navigated here. This has a timeline showing the progress of the email on
the right, and some optional info for the user that's toggleable on the
right bottom, while having either a preview of the email if it's sent or
a way to edit it. You can also change the scheduledAt date of an email
if it hasn't already been sent.

#### Bug Fixes
1. **Search in `TeamMemberSearchTable`**: This was broken. Every time
you tried to enter or remove a character, it would trigger skeleton
loading that overlapped the search bar too, preventing you from
adding/removing more. This was caused because the `useUser` hook
eventually ended up calling a `use` hook, which throws a promise that
triggers a suspense. This, coupled with the fact that the implementation
of `TeamMemberSearchTable` involved a prop-drilling/ dependency
inversion approach to passing down its toolbar to a base table
component, meant the suspense would cover the toolbar too and couldn't
be scoped to just the table. A refactor has gotten rid of the need for
those base components while fixing tables in `payments/customers`,
`teams/team_id`, and `payments/transactions` on top of the existing use
in email drafts recipients stage. We also dedupped some code.
2. **Stale draft fetches on draft landing page**: `useEmailDrafts` uses
an asyncCache to cache the fetched drafts. It is used on the drafts
landing page to render the drafts. When a draft is sent, its `sentAt` is
marked versus when it is still active, it is marked as null. The cache
was stale and so navigating to the landing page after firing off a draft
would errorneously represent that draft as still active and indeed, even
allow you to edit it and fire it again. This violated the principle of
drafts being fire and forget. This has been dealt with by adding
functionality to refresh the draft cache upon firing off a draft.

#### Other Changes

1. We bumped up the base time for the exponential send attempt retry
backoff in `email-queue-step` to 20 seconds. The previous base was two
seconds, and this effectively just made it wait until the next iteration
of the `email-queue-step` cron job or at most an iteration that wasn't
too far away. When an outage with our provider happens, it may take a
while for it to be resolved, so a longer backoff is justified
2. We transitioned the themes page and the templates page to using the
new components, though deeper UI refactors for them were out of scope
for this ticket.
3. We implement a "temporarily increase capacity" button, that bumps up
the throughput/ capacity limit fourfold for a user for a given period of
time. It works like this:

> Clicking the button sets a boost expiredat time.
> When this time is set and still valid, the capacity rate is multiplied
by 4.
> When the button is clicked, trigger a loading spinner until the route
finishes processing.
> When the timer runs out, we reset the button back to its original
state.
> We dont need to wrap the onclick with runAsyncWithAlert because the
component does that already.

4. We add a new default theme: a colorful theme with a lavender base.
This was mainly done so we could have three times in a theme showcase in
the settings page.

### UI Demos

**Sent Page Demo:**


https://github.com/user-attachments/assets/19294a90-bb65-4f00-9a97-111f6c08287f

**Drafts Page Demo**



https://github.com/user-attachments/assets/847609ef-d699-470c-a699-297bb9e17f04

**Settings Page Demo**



https://github.com/user-attachments/assets/190a3829-036a-4f57-89c0-a873bef5a7ce

**Email Viewer Page Demo**



https://github.com/user-attachments/assets/3bc50159-4acb-4865-a4dd-830c84ee4235


---------

Co-authored-by: Konstantin Wohlwend <n2d4xc@gmail.com>

Author: nams1570

apps/backend/prisma/migrations/20260209180554_add_email_capacity_boost_expires_at_to_tenancy/migration.sql

@@ -0,0 +1,3 @@
+-- AlterTable
+ALTER TABLE "Tenancy" ADD COLUMN     "emailCapacityBoostExpiresAt" TIMESTAMP(3);
+

apps/backend/prisma/schema.prisma

@@ -75,6 +75,9 @@ model Tenancy {
   sessionReplayChunks SessionReplayChunk[]
   managedEmailDomains ManagedEmailDomain[]
 
+  // Email capacity boost - when set and in the future, email capacity is multiplied by 4
+  emailCapacityBoostExpiresAt DateTime?
+
   @@unique([projectId, branchId, organizationId])
   @@unique([projectId, branchId, hasNoOrganization])
 }

apps/backend/src/app/api/latest/emails/capacity-boost/route.tsx

@@ -0,0 +1,53 @@
+import { globalPrismaClient } from "@/prisma-client";
+import { createSmartRouteHandler } from "@/route-handlers/smart-route-handler";
+import { KnownErrors } from "@stackframe/stack-shared/dist/known-errors";
+import { adaptSchema, serverOrHigherAuthTypeSchema, yupNumber, yupObject, yupString } from "@stackframe/stack-shared/dist/schema-fields";
+
+const BOOST_DURATION_HOURS = 4;
+
+export const POST = createSmartRouteHandler({
+  metadata: {
+    summary: "Activate email capacity boost",
+    description: "Temporarily increases email capacity by 4x for 4 hours.",
+    tags: ["Emails"],
+  },
+  request: yupObject({
+    auth: yupObject({
+      type: serverOrHigherAuthTypeSchema,
+      tenancy: adaptSchema.defined(),
+    }).defined(),
+    method: yupString().oneOf(["POST"]).defined(),
+  }),
+  response: yupObject({
+    statusCode: yupNumber().oneOf([200]).defined(),
+    bodyType: yupString().oneOf(["json"]).defined(),
+    body: yupObject({
+      expires_at: yupString().defined(),
+    }).defined(),
+  }),
+  handler: async ({ auth }) => {
+    const tenancy = await globalPrismaClient.tenancy.findUniqueOrThrow({
+      where: { id: auth.tenancy.id },
+      select: { emailCapacityBoostExpiresAt: true },
+    });
+
+    if (tenancy.emailCapacityBoostExpiresAt && tenancy.emailCapacityBoostExpiresAt > new Date()) {
+      throw new KnownErrors.EmailCapacityBoostAlreadyActive(tenancy.emailCapacityBoostExpiresAt.toISOString());
+    }
+
+    const expiresAt = new Date(Date.now() + BOOST_DURATION_HOURS * 60 * 60 * 1000);
+
+    await globalPrismaClient.tenancy.update({
+      where: { id: auth.tenancy.id },
+      data: { emailCapacityBoostExpiresAt: expiresAt },
+    });
+
+    return {
+      statusCode: 200,
+      bodyType: "json",
+      body: {
+        expires_at: expiresAt.toISOString(),
+      },
+    };
+  },
+});

apps/backend/src/app/api/latest/emails/delivery-info/route.tsx

@@ -1,6 +1,6 @@
-import { calculateCapacityRate, getEmailDeliveryStatsForTenancy } from "@/lib/email-delivery-stats";
+import { calculateCapacityRate, getEmailCapacityBoostExpiresAt, getEmailDeliveryStatsForTenancy } from "@/lib/email-delivery-stats";
 import { createSmartRouteHandler } from "@/route-handlers/smart-route-handler";
-import { adaptSchema, serverOrHigherAuthTypeSchema, yupNumber, yupObject, yupString } from "@stackframe/stack-shared/dist/schema-fields";
+import { adaptSchema, serverOrHigherAuthTypeSchema, yupBoolean, yupNumber, yupObject, yupString } from "@stackframe/stack-shared/dist/schema-fields";
 
 const windows = [
   { key: "hour" as const },
@@ -34,13 +34,19 @@ export const GET = createSmartRouteHandler({
       }).defined(),
       capacity: yupObject({
         rate_per_second: yupNumber().defined(),
+        boost_multiplier: yupNumber().defined(),
         penalty_factor: yupNumber().defined(),
+        is_boost_active: yupBoolean().defined(),
+        boost_expires_at: yupString().nullable().defined(),
       }).defined(),
     }).defined(),
   }),
   handler: async ({ auth }) => {
-    const stats = await getEmailDeliveryStatsForTenancy(auth.tenancy.id);
-    const capacity = calculateCapacityRate(stats);
+    const [stats, boostExpiresAt] = await Promise.all([
+      getEmailDeliveryStatsForTenancy(auth.tenancy.id),
+      getEmailCapacityBoostExpiresAt(auth.tenancy.id),
+    ]);
+    const capacity = calculateCapacityRate(stats, boostExpiresAt);
 
     return {
       statusCode: 200,
@@ -57,7 +63,10 @@ export const GET = createSmartRouteHandler({
         }, {} as Record<typeof windows[number]["key"], { sent: number, bounced: number, marked_as_spam: number }>),
         capacity: {
           rate_per_second: capacity.ratePerSecond,
+          boost_multiplier: capacity.boostMultiplier,
           penalty_factor: capacity.penaltyFactor,
+          is_boost_active: capacity.isBoostActive,
+          boost_expires_at: boostExpiresAt?.toISOString() ?? null,
         },
       },
     };

apps/backend/src/app/api/latest/emails/outbox/crud.tsx

@@ -88,6 +88,10 @@ function prismaModelToCrud(prismaModel: EmailOutbox): EmailOutboxCrud["Server"][
     variables: (prismaModel.extraRenderVariables ?? {}) as Record<string, any>,
     skip_deliverability_check: prismaModel.shouldSkipDeliverabilityCheck,
     scheduled_at_millis: prismaModel.scheduledAt.getTime(),
+    // Source tracking for grouping emails by template/draft
+    created_with: (prismaModel.createdWith === "DRAFT" ? "draft" : "programmatic-call") as "draft" | "programmatic-call",
+    email_draft_id: prismaModel.emailDraftId,
+    email_programmatic_call_template_id: prismaModel.emailProgrammaticCallTemplateId,
     send_retries: prismaModel.sendRetries,
     next_send_retry_at_millis: prismaModel.nextSendRetryAt?.getTime() ?? null,
     send_attempt_errors: sendAttemptErrors,

apps/backend/src/app/api/latest/emails/send-email/route.tsx

@@ -9,11 +9,6 @@ import { adaptSchema, jsonSchema, serverOrHigherAuthTypeSchema, templateThemeIdS
 import { getEnvVariable } from "@stackframe/stack-shared/dist/utils/env";
 import { StatusError, throwErr } from "@stackframe/stack-shared/dist/utils/errors";
 
-type UserResult = {
-  user_id: string,
-  user_email?: string,
-};
-
 const bodyBase = yupObject({
   user_ids: yupArray(yupString().defined()).optional(),
   all_users: yupBoolean().oneOf([true]).optional(),
@@ -25,6 +20,9 @@ const bodyBase = yupObject({
   is_high_priority: yupBoolean().optional().meta({
     openapiField: { description: "Marks the email as high priority so it jumps the queue." }
   }),
+  scheduled_at_millis: yupNumber().optional().meta({
+    openapiField: { description: "When to send the email. If not specified, the email will be sent immediately." }
+  }),
 });
 
 export const POST = createSmartRouteHandler({
@@ -83,7 +81,7 @@ export const POST = createSmartRouteHandler({
 
     const prisma = await getPrismaClientForTenancy(auth.tenancy);
 
-    const variables = "variables" in body ? body.variables ?? {} : {};
+    let variables: Record<string, any> = "variables" in body ? body.variables ?? {} : {};
 
     let overrideSubject: string | undefined = undefined;
     if (body.subject) {
@@ -159,6 +157,8 @@ export const POST = createSmartRouteHandler({
       }
     }
 
+    const scheduledAt = body.scheduled_at_millis ? new Date(body.scheduled_at_millis) : new Date();
+
     await sendEmailToMany({
       createdWith: createdWith,
       tenancy: auth.tenancy,
@@ -168,7 +168,7 @@ export const POST = createSmartRouteHandler({
       themeId: selectedThemeId === null ? null : (selectedThemeId === undefined ? auth.tenancy.config.emails.selectedThemeId : selectedThemeId),
       isHighPriority: isHighPriority,
       shouldSkipDeliverabilityCheck: false,
-      scheduledAt: new Date(),
+      scheduledAt: scheduledAt,
       overrideSubject: overrideSubject,
       overrideNotificationCategoryId: overrideNotificationCategoryId,
     });

apps/backend/src/app/api/latest/internal/rewrite-template-source/route.tsx

@@ -0,0 +1,43 @@
+import { rewriteTemplateSourceWithAI } from "@/lib/email-template-rewrite";
+import { createSmartRouteHandler } from "@/route-handlers/smart-route-handler";
+import { KnownErrors } from "@stackframe/stack-shared";
+import { adaptSchema, yupNumber, yupObject, yupString } from "@stackframe/stack-shared/dist/schema-fields";
+
+export const POST = createSmartRouteHandler({
+  metadata: {
+    summary: "Rewrite email template source for email draft creation",
+    description: "Rewrites email template TSX into standalone draft TSX using AI and runtime validation.",
+    hidden: true,
+    tags: ["Internal", "AI"],
+  },
+  request: yupObject({
+    auth: yupObject({
+      type: yupString().oneOf(["admin"]).defined(),
+      tenancy: adaptSchema.defined(),
+    }).defined(),
+    body: yupObject({
+      template_tsx_source: yupString().defined(),
+    }).defined(),
+  }),
+  response: yupObject({
+    statusCode: yupNumber().oneOf([200]).defined(),
+    bodyType: yupString().oneOf(["json"]).defined(),
+    body: yupObject({
+      tsx_source: yupString().defined(),
+    }).defined(),
+  }),
+  handler: async ({ body }) => {
+    const rewriteResult = await rewriteTemplateSourceWithAI(body.template_tsx_source);
+    if (rewriteResult.status === "error") {
+      throw new KnownErrors.TemplateSourceRewriteError(rewriteResult.error);
+    }
+
+    return {
+      statusCode: 200,
+      bodyType: "json",
+      body: {
+        tsx_source: rewriteResult.data,
+      },
+    };
+  },
+});

apps/backend/src/lib/email-delivery-stats.tsx

@@ -30,16 +30,22 @@ if (!Number.isFinite(defaultEmailCapacityPerHour)) {
   throw new StackAssertionError(`Invalid STACK_DEFAULT_EMAIL_CAPACITY_PER_HOUR environment variable: ${getEnvVariable("STACK_DEFAULT_EMAIL_CAPACITY_PER_HOUR", "<not set>")}`);
 }
 
-export function calculateCapacityRate(stats: EmailDeliveryStats) {
+const BOOST_MULTIPLIER = 4;
+
+export function calculateCapacityRate(stats: EmailDeliveryStats, boostExpiresAt?: Date | null) {
   const penaltyFactor = Math.min(
     calculatePenaltyFactor(stats.week.sent, stats.week.bounced, stats.week.markedAsSpam),
     calculatePenaltyFactor(stats.day.sent, stats.day.bounced, stats.day.markedAsSpam),
     calculatePenaltyFactor(stats.hour.sent, stats.hour.bounced, stats.hour.markedAsSpam)
   );
   const hourlyBaseline = defaultEmailCapacityPerHour + (4 * stats.month.sent / 30 / 24);  // default capacity + 4x the average throughput during the last month
   const ratePerHour = Math.max(hourlyBaseline * penaltyFactor, defaultEmailCapacityPerHour / 4);  // multiply by penalty factor, at least 1/4th of the default capacity
-  const ratePerSecond = ratePerHour / 60 / 60;
-  return { ratePerSecond, penaltyFactor };
+
+  const isBoostActive = boostExpiresAt != null && boostExpiresAt > new Date();
+  const boostMultiplier = isBoostActive ? BOOST_MULTIPLIER : 1;
+
+  const ratePerSecond = (ratePerHour / 60 / 60) * boostMultiplier;
+  return { ratePerSecond, boostMultiplier, penaltyFactor, isBoostActive };
 }
 
 const deliveryStatsQuery = (tenancyId: string): RawQuery<EmailDeliveryStats> => ({
@@ -107,3 +113,12 @@ export async function getEmailDeliveryStatsForTenancy(tenancyId: string, tx?: Pr
   const client = tx ?? globalPrismaClient;
   return await rawQuery(client, deliveryStatsQuery(tenancyId));
 }
+
+export async function getEmailCapacityBoostExpiresAt(tenancyId: string, tx?: PrismaClientTransaction): Promise<Date | null> {
+  const client = tx ?? globalPrismaClient;
+  const tenancy = await client.tenancy.findUniqueOrThrow({
+    where: { id: tenancyId },
+    select: { emailCapacityBoostExpiresAt: true },
+  });
+  return tenancy.emailCapacityBoostExpiresAt;
+}

apps/backend/src/lib/email-queue-step.tsx

@@ -1,5 +1,5 @@
 import { EmailOutbox, EmailOutboxSkippedReason, Prisma } from "@/generated/prisma/client";
-import { calculateCapacityRate, getEmailDeliveryStatsForTenancy } from "@/lib/email-delivery-stats";
+import { calculateCapacityRate, getEmailCapacityBoostExpiresAt, getEmailDeliveryStatsForTenancy } from "@/lib/email-delivery-stats";
 import { getEmailThemeForThemeId, renderEmailsForTenancyBatched } from "@/lib/email-rendering";
 import { EmailOutboxRecipient, getEmailConfig, } from "@/lib/emails";
 import { generateUnsubscribeLink, getNotificationCategoryById, hasNotificationEnabled, listNotificationCategories } from "@/lib/notification-categories";
@@ -21,7 +21,7 @@ const MAX_RENDER_BATCH = 50;
 
 const MAX_SEND_ATTEMPTS = 5;
 
-const SEND_RETRY_BACKOFF_BASE_MS = 2000;
+const SEND_RETRY_BACKOFF_BASE_MS = 20000;
 
 const calculateRetryBackoffMs = (attemptCount: number): number => {
   return (Math.random() + 0.5) * SEND_RETRY_BACKOFF_BASE_MS * Math.pow(2, attemptCount);
@@ -555,13 +555,21 @@ async function prepareSendPlan(deltaSeconds: number): Promise<TenancySendBatch[]
 
   const plan: TenancySendBatch[] = [];
   for (const entry of tenancyIds) {
-    const stats = await getEmailDeliveryStatsForTenancy(entry.tenancyId);
-    const capacity = calculateCapacityRate(stats);
-    const quota = stochasticQuota(capacity.ratePerSecond * deltaSeconds);
-    if (quota <= 0) continue;
-    const rows = await claimEmailsForSending(globalPrismaClient, entry.tenancyId, quota);
-    if (rows.length === 0) continue;
-    plan.push({ tenancyId: entry.tenancyId, rows, capacityRatePerSecond: capacity.ratePerSecond });
+    try {
+      const [stats, boostExpiresAt] = await Promise.all([
+        getEmailDeliveryStatsForTenancy(entry.tenancyId),
+        getEmailCapacityBoostExpiresAt(entry.tenancyId),
+      ]);
+      const capacity = calculateCapacityRate(stats, boostExpiresAt);
+      const quota = stochasticQuota(capacity.ratePerSecond * deltaSeconds);
+      if (quota <= 0) continue;
+      const rows = await claimEmailsForSending(globalPrismaClient, entry.tenancyId, quota);
+      if (rows.length === 0) continue;
+      plan.push({ tenancyId: entry.tenancyId, rows, capacityRatePerSecond: capacity.ratePerSecond });
+    } catch (error) {
+      captureError("email-queue-step-prepare-send-plan-for-tenancy-error", error);
+      continue;
+    }
   }
   return plan;
 }

apps/backend/src/lib/email-rendering.test.tsx

@@ -685,3 +685,4 @@ describe('renderEmailWithTemplate', () => {
     `);
   });
 });
+