Skip to content

Commit d14317c

Browse files
BilalG1N2D4
andauthored
batch sending (#875)
<!-- Make sure you've read the CONTRIBUTING.md guidelines: https://github.com/stack-auth/stack-auth/blob/dev/CONTRIBUTING.md --> <!-- ELLIPSIS_HIDDEN --> ---- > [!IMPORTANT] > Implement batch email rendering and sending with enhanced processing and error handling. > > - **Behavior**: > - Implement batch email rendering and sending in `route.tsx` using `renderEmailsWithTemplateBatched()` and `sendEmailResendBatched()`. > - Add per-recipient notification category resolution and unsubscribe link generation. > - Support templates from IDs, raw HTML, or drafts with dynamic theme handling. > - Enhanced result reporting, including users without primary emails. > - **Functions**: > - Add `renderEmailsWithTemplateBatched()` in `email-rendering.tsx` for batch email rendering. > - Add `sendEmailResendBatched()` in `emails.tsx` for batch email sending. > - Add `getChunks()` in `arrays.tsx` to split arrays into chunks. > - **Tests**: > - Add timed waits in `send-email.test.ts` and `unsubscribe-link.test.ts` to stabilize email delivery checks. > - **Dependencies**: > - Add `@react-email/render` and `resend` to `package.json` for email rendering and sending. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=stack-auth%2Fstack-auth&utm_source=github&utm_medium=referral)<sup> for ff1dea6. You can [customize](https://app.ellipsis.dev/stack-auth/settings/summaries) this summary. It will automatically update as commits are pushed.</sup> ---- <!-- ELLIPSIS_HIDDEN --> <!-- RECURSEML_SUMMARY:START --> ## Review by RecurseML _🔍 Review performed on [3c34140..1267879](3c34140...1267879cfdb4e44705adbfad2795a2b41cdc8a71)_ | Severity | Location | Issue | |----------|----------|-------| | ![Medium](https://img.shields.io/badge/Medium-yellow?style=plastic) | [apps/e2e/tests/backend/endpoints/api/v1/send-email.test.ts:593](#875 (comment)) | Asynchronous wait function not wrapped with runAsynchronously | | ![Medium](https://img.shields.io/badge/Medium-yellow?style=plastic) | [apps/e2e/tests/backend/endpoints/api/v1/send-email.test.ts:743](#875 (comment)) | Asynchronous wait function not wrapped with runAsynchronously | <details> <summary>✅ Files analyzed, no issues (4)</summary> • `apps/backend/src/app/api/latest/emails/send-email/route.tsx` • `apps/backend/src/lib/email-rendering.tsx` • `apps/backend/src/lib/emails.tsx` • `packages/stack-shared/src/utils/arrays.tsx` </details> <details> <summary>⏭️ Files skipped (low suspicion) (2)</summary> • `apps/backend/package.json` • `pnpm-lock.yaml` </details> [![Need help? Join our Discord](https://img.shields.io/badge/Need%20help%3F%20Join%20our%20Discord-5865F2?style=plastic&logo=discord&logoColor=white)](https://discord.gg/n3SsVDAW6U) <!-- RECURSEML_SUMMARY:END --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - New Features - Batch email rendering and sending to multiple recipients with background processing. - Per-recipient notification category resolution and unsubscribe link generation. - Support for templates from IDs, raw HTML, or drafts with dynamic theme handling. - Enhanced result reporting, including users without primary emails. - Chores - Added dependencies for email rendering and bulk sending. - Tests - Stabilized email delivery checks with timed waits across relevant e2e tests. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Konsti Wohlwend <n2d4xc@gmail.com>
1 parent 39a4b05 commit d14317c

8 files changed

Lines changed: 383 additions & 68 deletions

File tree

apps/backend/package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@
6161
"@prisma/adapter-pg": "^6.12.0",
6262
"@prisma/client": "^6.12.0",
6363
"@prisma/instrumentation": "^6.12.0",
64+
"@react-email/render": "^1.2.1",
6465
"@sentry/nextjs": "^10.11.0",
6566
"@simplewebauthn/server": "^11.0.0",
6667
"@stackframe/stack": "workspace:*",
@@ -85,6 +86,7 @@
8586
"posthog-node": "^4.1.0",
8687
"react": "19.0.0",
8788
"react-dom": "19.0.0",
89+
"resend": "^6.0.1",
8890
"semver": "^7.6.3",
8991
"sharp": "^0.32.6",
9092
"stripe": "^18.3.0",

apps/backend/src/app/api/latest/emails/send-email/route.tsx

Lines changed: 121 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,16 @@
1-
import { createTemplateComponentFromHtml, getEmailThemeForTemplate, renderEmailWithTemplate } from "@/lib/email-rendering";
2-
import { getEmailConfig, sendEmail } from "@/lib/emails";
1+
import { getEmailDraft, themeModeToTemplateThemeId } from "@/lib/email-drafts";
2+
import { createTemplateComponentFromHtml, getEmailThemeForTemplate, renderEmailsWithTemplateBatched } from "@/lib/email-rendering";
3+
import { getEmailConfig, sendEmail, sendEmailResendBatched } from "@/lib/emails";
34
import { getNotificationCategoryByName, hasNotificationEnabled } from "@/lib/notification-categories";
45
import { getPrismaClientForTenancy } from "@/prisma-client";
56
import { createSmartRouteHandler } from "@/route-handlers/smart-route-handler";
7+
import { runAsynchronouslyAndWaitUntil } from "@/utils/vercel";
8+
import { KnownErrors } from "@stackframe/stack-shared";
69
import { adaptSchema, serverOrHigherAuthTypeSchema, templateThemeIdSchema, yupArray, yupBoolean, yupMixed, yupNumber, yupObject, yupRecord, yupString, yupUnion } from "@stackframe/stack-shared/dist/schema-fields";
10+
import { getChunks } from "@stackframe/stack-shared/dist/utils/arrays";
711
import { getEnvVariable } from "@stackframe/stack-shared/dist/utils/env";
812
import { StatusError, throwErr } from "@stackframe/stack-shared/dist/utils/errors";
913
import { unsubscribeLinkVerificationCodeHandler } from "../unsubscribe-link/verification-handler";
10-
import { KnownErrors } from "@stackframe/stack-shared";
11-
import { getEmailDraft, themeModeToTemplateThemeId } from "@/lib/email-drafts";
1214

1315
type UserResult = {
1416
user_id: string,
@@ -108,96 +110,147 @@ export const POST = createSmartRouteHandler({
108110
throw new KnownErrors.UserIdDoesNotExist(missingUserIds[0]);
109111
}
110112
const userMap = new Map(users.map(user => [user.projectUserId, user]));
111-
const userSendErrors: Map<string, string> = new Map();
112113
const userPrimaryEmails: Map<string, string> = new Map();
113-
114114
for (const user of userMap.values()) {
115115
const primaryEmail = user.contactChannels.find((c) => c.isPrimary === "TRUE")?.value;
116-
if (!primaryEmail) {
117-
userSendErrors.set(user.projectUserId, "User does not have a primary email");
118-
continue;
116+
if (primaryEmail) {
117+
userPrimaryEmails.set(user.projectUserId, primaryEmail);
119118
}
120-
userPrimaryEmails.set(user.projectUserId, primaryEmail);
119+
}
120+
121+
const results: UserResult[] = Array.from(userMap.values()).map((user) => ({
122+
user_id: user.projectUserId,
123+
user_email: userPrimaryEmails.get(user.projectUserId) ?? user.contactChannels.find((c) => c.isPrimary === "TRUE")?.value,
124+
}));
125+
126+
const BATCH_SIZE = 100;
121127

122-
let currentNotificationCategory = defaultNotificationCategory;
128+
const resolveCategoriesForUsers = async (usersWithPrimary: typeof users) => {
129+
const currentCategories = new Map<string, ReturnType<typeof getNotificationCategoryByName>>();
123130
if (!("html" in body)) {
124-
// We have to render email twice in this case, first pass is to get the notification category
125-
const renderedTemplateFirstPass = await renderEmailWithTemplate(
126-
templateSource,
127-
themeSource,
128-
{
129-
user: { displayName: user.displayName },
130-
project: { displayName: auth.tenancy.project.display_name },
131-
variables,
132-
},
133-
);
134-
if (renderedTemplateFirstPass.status === "error") {
135-
userSendErrors.set(user.projectUserId, "There was an error rendering the email");
136-
continue;
131+
const firstPassInputs = usersWithPrimary.map((user) => ({
132+
user: { displayName: user.displayName },
133+
project: { displayName: auth.tenancy.project.display_name },
134+
variables,
135+
}));
136+
137+
const chunks = getChunks(firstPassInputs, BATCH_SIZE);
138+
const userChunks = getChunks(usersWithPrimary, BATCH_SIZE);
139+
for (let i = 0; i < chunks.length; i++) {
140+
const chunk = chunks[i];
141+
const correspondingUsers = userChunks[i];
142+
const rendered = await renderEmailsWithTemplateBatched(templateSource, themeSource, chunk);
143+
if (rendered.status === "error") {
144+
continue;
145+
}
146+
const outputs = rendered.data;
147+
for (let j = 0; j < outputs.length; j++) {
148+
const output = outputs[j];
149+
const user = correspondingUsers[j];
150+
const category = getNotificationCategoryByName(output.notificationCategory ?? "");
151+
currentCategories.set(user.projectUserId, category);
152+
}
137153
}
138-
const notificationCategory = getNotificationCategoryByName(renderedTemplateFirstPass.data.notificationCategory ?? "");
139-
if (!notificationCategory) {
140-
userSendErrors.set(user.projectUserId, "Notification category not found with given name");
141-
continue;
154+
} else {
155+
for (const user of usersWithPrimary) {
156+
currentCategories.set(user.projectUserId, defaultNotificationCategory);
142157
}
143-
currentNotificationCategory = notificationCategory;
144158
}
159+
return currentCategories;
160+
};
145161

146-
const isNotificationEnabled = await hasNotificationEnabled(auth.tenancy, user.projectUserId, currentNotificationCategory.id);
147-
if (!isNotificationEnabled) {
148-
userSendErrors.set(user.projectUserId, "User has disabled notifications for this category");
149-
continue;
150-
}
162+
const getAllowedUsersWithUnsub = async (usersWithPrimary: typeof users, currentCategories: Map<string, ReturnType<typeof getNotificationCategoryByName>>) => {
163+
const allowed = await Promise.all(usersWithPrimary.map(async (user) => {
164+
const category = currentCategories.get(user.projectUserId) ?? defaultNotificationCategory;
165+
const enabled = await hasNotificationEnabled(auth.tenancy, user.projectUserId, category.id);
166+
return enabled ? { user, category } : null;
167+
})).then(r => r.filter((x): x is { user: typeof users[number], category: NonNullable<ReturnType<typeof getNotificationCategoryByName>> } => Boolean(x)));
151168

152-
let unsubscribeLink: string | undefined = undefined;
153-
if (currentNotificationCategory.can_disable) {
169+
const unsubLinks = new Map<string, string | undefined>();
170+
await Promise.all(allowed.map(async ({ user, category }) => {
171+
if (!category.can_disable) {
172+
unsubLinks.set(user.projectUserId, undefined);
173+
return;
174+
}
154175
const { code } = await unsubscribeLinkVerificationCodeHandler.createCode({
155176
tenancy: auth.tenancy,
156177
method: {},
157178
data: {
158179
user_id: user.projectUserId,
159-
notification_category_id: currentNotificationCategory.id,
180+
notification_category_id: category.id,
160181
},
161182
callbackUrl: undefined
162183
});
163184
const unsubUrl = new URL(getEnvVariable("NEXT_PUBLIC_STACK_API_URL"));
164185
unsubUrl.pathname = "/api/v1/emails/unsubscribe-link";
165186
unsubUrl.searchParams.set("code", code);
166-
unsubscribeLink = unsubUrl.toString();
167-
}
187+
unsubLinks.set(user.projectUserId, unsubUrl.toString());
188+
}));
189+
return { allowed, unsubLinks };
190+
};
168191

169-
const renderedEmail = await renderEmailWithTemplate(
170-
templateSource,
171-
themeSource,
172-
{
173-
user: { displayName: user.displayName },
174-
project: { displayName: auth.tenancy.project.display_name },
175-
variables,
176-
unsubscribeLink,
177-
},
178-
);
179-
if (renderedEmail.status === "error") {
180-
userSendErrors.set(user.projectUserId, "There was an error rendering the email");
181-
continue;
192+
const renderAndSendBatches = async (finalUsers: typeof users, unsubLinks: Map<string, string | undefined>) => {
193+
const finalInputs = finalUsers.map((user) => ({
194+
user: { displayName: user.displayName },
195+
project: { displayName: auth.tenancy.project.display_name },
196+
variables,
197+
unsubscribeLink: unsubLinks.get(user.projectUserId),
198+
}));
199+
200+
const inputChunks = getChunks(finalInputs, BATCH_SIZE);
201+
const userChunks = getChunks(finalUsers, BATCH_SIZE);
202+
203+
for (let i = 0; i < inputChunks.length; i++) {
204+
const chunk = inputChunks[i];
205+
const correspondingUsers = userChunks[i];
206+
const rendered = await renderEmailsWithTemplateBatched(templateSource, themeSource, chunk);
207+
if (rendered.status === "error") {
208+
continue;
209+
}
210+
const outputs = rendered.data;
211+
const emailOptions = outputs.map((output, idx) => {
212+
const user = correspondingUsers[idx];
213+
const email = userPrimaryEmails.get(user.projectUserId);
214+
if (!email) return null;
215+
return {
216+
tenancyId: auth.tenancy.id,
217+
emailConfig,
218+
to: email,
219+
subject: body.subject ?? output.subject ?? "",
220+
html: output.html,
221+
text: output.text,
222+
};
223+
}).filter((option): option is NonNullable<typeof option> => Boolean(option));
224+
225+
if (emailConfig.host === "smtp.resend.com") {
226+
await sendEmailResendBatched(emailConfig.password, emailOptions);
227+
} else {
228+
await Promise.allSettled(emailOptions.map(option => sendEmail(option)));
229+
}
182230
}
183-
try {
184-
await sendEmail({
185-
tenancyId: auth.tenancy.id,
186-
emailConfig,
187-
to: primaryEmail,
188-
subject: body.subject ?? renderedEmail.data.subject ?? "",
189-
html: renderedEmail.data.html,
190-
text: renderedEmail.data.text,
231+
};
232+
233+
runAsynchronouslyAndWaitUntil((async () => {
234+
const usersArray = Array.from(userMap.values());
235+
236+
const usersWithPrimary = usersArray.filter(u => userPrimaryEmails.has(u.projectUserId));
237+
const currentCategories = await resolveCategoriesForUsers(usersWithPrimary);
238+
const { allowed, unsubLinks } = await getAllowedUsersWithUnsub(usersWithPrimary, currentCategories);
239+
const finalUsers = allowed.map(({ user }) => user);
240+
await renderAndSendBatches(finalUsers, unsubLinks);
241+
242+
if ("draft_id" in body) {
243+
await prisma.emailDraft.update({
244+
where: {
245+
tenancyId_id: {
246+
tenancyId: auth.tenancy.id,
247+
id: body.draft_id,
248+
},
249+
},
250+
data: { sentAt: new Date() },
191251
});
192-
} catch {
193-
userSendErrors.set(user.projectUserId, "Failed to send email");
194252
}
195-
}
196-
197-
const results: UserResult[] = Array.from(userMap.values()).map((user) => ({
198-
user_id: user.projectUserId,
199-
user_email: userPrimaryEmails.get(user.projectUserId) ?? user.contactChannels.find((c) => c.isPrimary === "TRUE")?.value,
200-
}));
253+
})());
201254

202255
if ("draft_id" in body) {
203256
await prisma.emailDraft.update({

apps/backend/src/lib/email-rendering.tsx

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { get, has } from '@stackframe/stack-shared/dist/utils/objects';
66
import { Result } from "@stackframe/stack-shared/dist/utils/results";
77
import { deindent } from "@stackframe/stack-shared/dist/utils/strings";
88
import { Tenancy } from './tenancies';
9+
import { getEnvVariable } from '@stackframe/stack-shared/dist/utils/env';
910

1011
export function getActiveEmailTheme(tenancy: Tenancy) {
1112
const themeList = tenancy.config.emails.themes;
@@ -125,6 +126,86 @@ export async function renderEmailWithTemplate(
125126
return Result.ok(output.data.result as { html: string, text: string, subject: string, notificationCategory: string });
126127
}
127128

129+
export async function renderEmailsWithTemplateBatched(
130+
templateOrDraftComponent: string,
131+
themeComponent: string,
132+
inputs: Array<{
133+
user: { displayName: string | null },
134+
project: { displayName: string },
135+
variables?: Record<string, any>,
136+
unsubscribeLink?: string,
137+
}>,
138+
): Promise<Result<Array<{ html: string, text: string, subject?: string, notificationCategory?: string }>, string>> {
139+
const apiKey = getEnvVariable("STACK_FREESTYLE_API_KEY");
140+
141+
const serializedInputs = JSON.stringify(inputs);
142+
143+
const result = await bundleJavaScript({
144+
"/utils.tsx": findComponentValueUtil,
145+
"/theme.tsx": themeComponent,
146+
"/template.tsx": templateOrDraftComponent,
147+
"/render.tsx": deindent`
148+
import { configure } from "arktype/config"
149+
configure({ onUndeclaredKey: "delete" })
150+
import React from 'react';
151+
import { render } from '@react-email/components';
152+
import { type } from "arktype";
153+
import { findComponentValue } from "./utils.tsx";
154+
import * as TemplateModule from "./template.tsx";
155+
const { variablesSchema, EmailTemplate } = TemplateModule;
156+
import { EmailTheme } from "./theme.tsx";
157+
158+
export const renderAll = async () => {
159+
const inputs = ${serializedInputs}
160+
const renderOne = async (input: any) => {
161+
const variables = variablesSchema ? variablesSchema({
162+
...(input.variables || {}),
163+
}) : {};
164+
if (variables instanceof type.errors) {
165+
throw new Error(variables.summary)
166+
}
167+
const EmailTemplateWithProps = <EmailTemplate variables={variables} user={input.user} project={input.project} />;
168+
const Email = <EmailTheme unsubscribeLink={input.unsubscribeLink}>
169+
{ EmailTemplateWithProps }
170+
</EmailTheme>;
171+
return {
172+
html: await render(Email),
173+
text: await render(Email, { plainText: true }),
174+
subject: findComponentValue(EmailTemplateWithProps, "Subject"),
175+
notificationCategory: findComponentValue(EmailTemplateWithProps, "NotificationCategory"),
176+
};
177+
};
178+
179+
return await Promise.all(inputs.map(renderOne));
180+
}
181+
`,
182+
"/entry.js": deindent`
183+
import { renderAll } from "./render.tsx";
184+
export default renderAll;
185+
`,
186+
}, {
187+
keepAsImports: ['arktype', 'react', 'react/jsx-runtime', '@react-email/components'],
188+
externalPackages: { '@stackframe/emails': stackframeEmailsPackage },
189+
format: 'esm',
190+
sourcemap: false,
191+
});
192+
if (result.status === "error") {
193+
return Result.error(result.error);
194+
}
195+
196+
const freestyle = new Freestyle({ apiKey });
197+
const nodeModules = {
198+
"react": "19.1.1",
199+
"@react-email/components": "0.1.1",
200+
"arktype": "2.1.20",
201+
};
202+
const executeResult = await freestyle.executeScript(result.data, { nodeModules });
203+
if (executeResult.status === "error") {
204+
return Result.error(executeResult.error);
205+
}
206+
return Result.ok(executeResult.data.result as Array<{ html: string, text: string, subject?: string, notificationCategory?: string }>);
207+
}
208+
128209
const findComponentValueUtil = `import React from 'react';
129210
export function findComponentValue(element, targetStackComponent) {
130211
const matches = [];

0 commit comments

Comments
 (0)