Skip to content

Commit ad4f983

Browse files
authored
Add plan for mailgun transition for transactional email provider (#628)
# Context We're migrating our transactional email infrastructure from Customer.io to Mailgun, as recommended by our email deliverability providers. As part of this migration, email template content (previously managed in Customer.io's GUI) will move into code in the cloud repo. This PR contains the 12 HTML email templates plus a markdown file laying out the implementation plan. # Complication This is a high-stakes change — transactional emails are customer-facing and business-critical, so we need confidence in both the architecture and our ability to test safely before going live. # Question Does the implementation plan hold up to scrutiny, and are we set up to ship this safely? # Request The markdown file is the primary thing needing review. I'm requesting feedback focused on three areas: 1. Overall architecture — Does the general approach make sense? Any concerns? 2. Template file location — Where in the repo should the 12 HTML email templates live? 3. Pre-launch testing strategy — How should we test this before it goes live?
2 parents 420f583 + 65eb857 commit ad4f983

28 files changed

Lines changed: 2778 additions & 247 deletions

.env.development.local.example

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
# Email provider: 'customerio' (default) or 'mailgun'
2+
# EMAIL_PROVIDER=customerio
3+
# MAILGUN_API_KEY=key-...
4+
# MAILGUN_DOMAIN=app.kilocode.ai
5+
16
# Stripe integration
27
STRIPE_WEBHOOK_SECRET="...extract this from: stripe listen --forward-to http://localhost:3000/api/stripe/webhook"
38

.env.test

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,3 +81,4 @@ STRIPE_KILOCLAW_EARLYBIRD_COUPON_ID=coupon_test_kiloclaw_earlybird
8181
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_invalid_mock_key
8282

8383
CREDIT_CATEGORIES_ENCRYPTION_KEY=
84+
EMAIL_PROVIDER=customerio

package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,12 +111,14 @@
111111
"drizzle-orm": "catalog:",
112112
"event-source-polyfill": "^1.0.31",
113113
"eventsource-parser": "^3.0.6",
114+
"form-data": "^4.0.5",
114115
"jotai": "^2.15.1",
115116
"jotai-minidb": "^0.0.8",
116117
"js-cookie": "^3.0.5",
117118
"js-yaml": "^4.1.1",
118119
"jsonwebtoken": "catalog:",
119120
"lucide-react": "^0.552.0",
121+
"mailgun.js": "^12.7.0",
120122
"monaco-editor": "^0.55.1",
121123
"motion": "^12.23.24",
122124
"next": "^16.1.6",

plans/migrate-email-to-mailgun.md

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
# Migrate Email Sending from Customer.io to Mailgun
2+
3+
## Overview
4+
5+
Replace Customer.io transactional email API with Mailgun's Node.js SDK (`mailgun.js`). Instead of referencing remote Customer.io template IDs, we'll render the local HTML templates in `src/emails/*.html` server-side and send the full HTML body to Mailgun.
6+
7+
## Current Architecture
8+
9+
- **Provider**: Customer.io via `customerio-node` SDK
10+
- **Config**: `CUSTOMERIO_EMAIL_API_KEY` in `config.server.ts`
11+
- **Templates**: Remote, stored in Customer.io, referenced by numeric ID (`'10'`, `'11'`, etc.)
12+
- **Variable substitution**: Done by Customer.io using Liquid syntax (`{{ var }}`, `{% if %}`)
13+
- **Send function**: `send()` in `email.ts` creates `APIClient` + `SendEmailRequest`
14+
15+
## Target Architecture
16+
17+
- **Provider**: Mailgun via `mailgun.js` SDK + `form-data`
18+
- **Config**: `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, and `EMAIL_PROVIDER` in `config.server.ts`
19+
- **Templates**: Local HTML files in `src/emails/*.html`
20+
- **Variable substitution**: Done server-side in Node.js before sending
21+
- **Send function**: `send()` routes to either Customer.io or Mailgun backend based on `EMAIL_PROVIDER` env var
22+
23+
### Dual-Provider Routing
24+
25+
During the transition, `email.ts` delegates to a provider backend selected by the `EMAIL_PROVIDER` env var (`customerio` | `mailgun`, defaulting to `customerio`). The existing Customer.io logic moves to `src/lib/email-customerio.ts`; the new Mailgun logic lives in `src/lib/email-mailgun.ts`.
26+
27+
PostHog feature flags are **not** suitable for this toggle because some emails target logged-out users (magic link, invitations) where there is no user context for flag evaluation. A simple env var is the right mechanism.
28+
29+
```typescript
30+
import { sendViaCustomerIo } from './email-customerio';
31+
import { sendViaMailgun } from './email-mailgun';
32+
33+
type SendParams = {
34+
to: string;
35+
templateName: TemplateName;
36+
// Record<string, unknown> in PR 1 to support customerio's native types (numbers, booleans).
37+
// Tightened to Record<string, string> in PR 2 once renderTemplate is added.
38+
templateVars: Record<string, unknown>;
39+
};
40+
41+
function send(params: SendParams) {
42+
if (EMAIL_PROVIDER === 'mailgun') {
43+
// PR 2: looks up subjects[templateName], calls renderTemplate, then sendViaMailgun
44+
return sendViaMailgun();
45+
}
46+
return sendViaCustomerIo({
47+
transactional_message_id: templates[params.templateName],
48+
to: params.to,
49+
message_data: params.templateVars,
50+
identifiers: { email: params.to },
51+
reply_to: 'hi@kilocode.ai',
52+
});
53+
}
54+
```
55+
56+
Each `send*Email` function builds `templateVars` and calls `send()` once. Customer.io receives its native `transactional_message_id` + `message_data` interface. Mailgun (PR 2) renders HTML locally and looks up the subject from `subjects[templateName]` — no subject arg needed at call sites. All emails use `{ email: to }` as the Customer.io identifier.
57+
58+
### Template Rendering
59+
60+
A `renderTemplate()` function reads an HTML file from `src/emails/{name}.html` and replaces `{{ variable_name }}` placeholders with provided values:
61+
62+
```typescript
63+
function renderTemplate(name: string, vars: Record<string, string>) {
64+
const templatePath = path.join(process.cwd(), 'src', 'emails', `${name}.html`);
65+
const html = fs.readFileSync(templatePath, 'utf-8');
66+
return html.replace(/\{\{\s*(\w+)\s*\}\}/g, (_, key) => {
67+
if (!(key in vars)) {
68+
throw new Error(`Missing template variable '${key}' in email template '${name}'`);
69+
}
70+
return vars[key];
71+
});
72+
}
73+
```
74+
75+
The 3 OSS templates currently use a Liquid conditional `{% if has_credits %}...{% endif %}` which must be replaced with a `{{ credits_section }}` placeholder. The JS builds the snippet:
76+
77+
```typescript
78+
function buildCreditsSection(monthlyCreditsUsd: number): string {
79+
if (monthlyCreditsUsd <= 0) return '';
80+
return `<br />• <strong style="color: #d1d5db">$${monthlyCreditsUsd} USD in Kilo credits</strong>, which reset every 30 days`;
81+
}
82+
```
83+
84+
### Subject Lines
85+
86+
Customer.io stores subjects in the remote template. Mailgun needs them passed explicitly. Rather than adding a subject argument to every `send*Email` call site, subjects are stored in a `subjects` map in `email.ts` keyed by `TemplateName`. The Mailgun branch of `send()` looks up `subjects[templateName]` internally — call sites are unchanged.
87+
88+
| Template name | Subject |
89+
| --------------------------- | --------------------------------------------- |
90+
| `orgSubscription` | "Welcome to Kilo for Teams!" |
91+
| `orgRenewed` | "Kilo: Your Teams Subscription Renewal" |
92+
| `orgCancelled` | "Kilo: Your Teams Subscription is Cancelled" |
93+
| `orgSSOUserJoined` | "Kilo: New SSO User Joined Your Organization" |
94+
| `orgInvitation` | "Kilo: Teams Invitation" |
95+
| `magicLink` | "Sign in to Kilo Code" |
96+
| `balanceAlert` | "Kilo: Low Balance Alert" |
97+
| `autoTopUpFailed` | "Kilo: Auto Top-Up Failed" |
98+
| `ossInviteNewUser` | "Kilo: OSS Sponsorship Offer" |
99+
| `ossInviteExistingUser` | "Kilo: OSS Sponsorship Offer" |
100+
| `ossExistingOrgProvisioned` | "Kilo: OSS Sponsorship Offer" |
101+
| `deployFailed` | "Kilo: Your Deployment Failed" |
102+
103+
### Template Variables
104+
105+
All templates use `{{ variable }}` interpolation. `year` is always `String(new Date().getFullYear())`. `credits_section` is built in JS (HTML snippet or empty string).
106+
107+
| Template file | Variables |
108+
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
109+
| `orgSubscription.html` | `seats`, `organization_url`, `invoices_url`, `year` |
110+
| `orgRenewed.html` | `seats`, `invoices_url`, `year` |
111+
| `orgCancelled.html` | `invoices_url`, `year` |
112+
| `orgSSOUserJoined.html` | `new_user_email`, `organization_url`, `year` |
113+
| `orgInvitation.html` | `organization_name`, `inviter_name`, `accept_invite_url`, `year` |
114+
| `magicLink.html` | `magic_link_url`, `email`, `expires_in`, `year` |
115+
| `balanceAlert.html` | `minimum_balance`, `organization_url`, `year` |
116+
| `autoTopUpFailed.html` | `reason`, `credits_url`, `year` |
117+
| `ossInviteNewUser.html` | `tier_name`, `seats`, `seat_value`, `credits_section`, `accept_invite_url`, `integrations_url`, `code_reviews_url`, `year` |
118+
| `ossInviteExistingUser.html` | `tier_name`, `seats`, `seat_value`, `credits_section`, `organization_url`, `integrations_url`, `code_reviews_url`, `year` |
119+
| `ossExistingOrgProvisioned.html` | `tier_name`, `seats`, `seat_value`, `credits_section`, `organization_url`, `integrations_url`, `code_reviews_url`, `year` |
120+
| `deployFailed.html` | `deployment_name`, `deployment_url`, `repository`, `year` |
121+
122+
### Admin Email Testing Page
123+
124+
An admin page for sending test emails and previewing output. Controls:
125+
126+
- **Template** — dropdown of all available templates (one entry per `send*Email` function)
127+
- **Provider** — dropdown of implemented providers; only shows providers that are actually wired up (starts as just `customerio`; `mailgun` appears once `email-mailgun.ts` is implemented)
128+
- **Recipient** — text input pre-filled with the logged-in admin's email address, overridable with any valid email
129+
130+
Preview pane (updates when template or provider selection changes):
131+
132+
- **Customer.io**: shows the `message_data` variables that would be sent to the Customer.io API (key/value pairs, not rendered HTML)
133+
- **Mailgun**: shows the fully rendered HTML email in an iframe
134+
135+
Submitting sends a test email using hardcoded representative fixture data for the selected template, routed through the selected provider (ignoring the `EMAIL_PROVIDER` env var for this request). This page is useful both during QA and long-term for testing new templates.
136+
137+
### Environment Variables
138+
139+
- `EMAIL_PROVIDER` — Which email backend to use: `customerio` or `mailgun`. Defaults to `customerio` when unset. Setting an unrecognised value will throw at startup.
140+
- `MAILGUN_API_KEY` — Mailgun API key (starts with `key-...`)
141+
- `MAILGUN_DOMAIN` — Mailgun sending domain (`app.kilocode.ai`)
142+
143+
### Dependencies
144+
145+
Install: `pnpm add mailgun.js form-data` (keep `customerio-node` until cleanup)
146+
147+
## Migration Strategy
148+
149+
### PR 1: Routing Shell + Admin Testing Page ✅ Done
150+
151+
- ~~Add `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, `EMAIL_PROVIDER` to `src/lib/config.server.ts`~~
152+
- ~~Extract existing Customer.io send logic into `src/lib/email-customerio.ts` with `sendViaCustomerIo()`~~
153+
- ~~Create `src/lib/email-mailgun.ts` as a stub (throws "not yet implemented")~~
154+
- ~~Add routing layer in `email.ts`: `send()` takes `{ to, templateName, templateVars }` and delegates to the active provider~~
155+
- ~~Refactor all `send*Email` functions to use `templateName` + `templateVars` instead of building `SendEmailRequestOptions` directly~~
156+
- ~~Remove `inviteCode` from `OrganizationInviteEmailData` and `OssInviteEmailData`; all sends use `{ email: to }` as the Customer.io identifier~~
157+
- ~~Export `templates`, `subjects`, `TemplateName` from `email.ts`~~
158+
- ~~Build the admin testing page with template/provider/recipient controls and preview pane~~
159+
- ~~Hardcode representative fixture data for each template~~
160+
- ~~The provider dropdown only shows `customerio` at this point~~
161+
- ~~Add `EMAIL_PROVIDER=customerio` to `.env.local`, `.env.test`, and `.env.development.local.example`~~
162+
- Deploy with `EMAIL_PROVIDER=customerio` — no production change
163+
164+
### PR 2: Mailgun Send Logic + Template Rendering ✅ Done (merged into PR 1 branch)
165+
166+
Pre-requisite template work:
167+
168+
- ~~Replace `{{ "now" | date: "%Y" }}` with `{{ year }}` in all 12 templates~~
169+
- ~~Re-include `src/emails/*.html` and `src/emails/AGENTS.md`~~
170+
- ~~Replace the `{% if has_credits %}...{% endif %}` block with `{{ credits_section }}` in the 3 OSS templates (`ossInviteNewUser.html`, `ossInviteExistingUser.html`, `ossExistingOrgProvisioned.html`)~~
171+
172+
Implementation:
173+
174+
- ~~Install `mailgun.js` + `form-data`~~
175+
- ~~Implement `sendViaMailgun({ to, subject, html })` in `src/lib/email-mailgun.ts`~~
176+
- ~~Add `renderTemplate()` and `buildCreditsSection()` to `email.ts`~~
177+
- ~~Update `send()` in `email.ts`: mailgun branch looks up `subjects[templateName]`, calls `renderTemplate(templateName, templateVars)`, then `sendViaMailgun()`. Tighten `templateVars` to `Record<string, string>` — update all `send*Email` call sites (OSS functions replace `has_credits: boolean` + `monthly_credits_usd: number` with `credits_section: buildCreditsSection(...)`)~~
178+
- ~~Add `mailgun` to the providers list in `email-testing-router.ts`; update `getPreview` to return rendered HTML for mailgun; update `sendTest` to call `sendViaMailgun` directly~~
179+
- ~~The admin page now shows `mailgun` in the provider dropdown with a full HTML iframe preview~~
180+
- Deploy with `EMAIL_PROVIDER=customerio` — still no production change
181+
182+
**QA** (no PR): Use the admin testing page to send each template via Mailgun to real inboxes. Compare rendered output against the Customer.io versions. Verify all variable substitution, styling, links.
183+
184+
### PR 3: Flip to Mailgun
185+
186+
- Set `EMAIL_PROVIDER=mailgun` in production
187+
- Monitor delivery, bounce rates, rendering across email clients
188+
189+
### PR 4: Cleanup
190+
191+
After the provider switch is confirmed stable:
192+
193+
- Remove `customerio-node` from `package.json`
194+
- Remove `CUSTOMERIO_EMAIL_API_KEY` from `config.server.ts`
195+
- Remove the `templates` map (no longer needed)
196+
- Remove `SendEmailRequestOptions` type imports
197+
- Remove `EMAIL_PROVIDER` routing logic from `email.ts` and delete `src/lib/email-customerio.ts`
198+
- Note: `src/lib/external-services.ts` also references Customer.io for user deletion — that's a separate concern and uses different API keys (`CUSTOMERIO_SITE_ID`, `CUSTOMERIO_API_KEY`)
199+
200+
## Files Changed
201+
202+
| File | PR | Change |
203+
| ----------------------------------------------------------- | --- | --------------------------------------------------------------------------------------------------- |
204+
| `src/lib/email.ts` | 1 | Routing via `send({ templateName, templateVars })`; exports `templates`, `subjects`, `TemplateName` |
205+
| `src/lib/email-customerio.ts` | 1 | Extracted `sendViaCustomerIo()`; minimal PII-free logging |
206+
| `src/lib/email-mailgun.ts` | 1→2 | Stub in PR 1 (throws); full `sendViaMailgun({ to, subject, html })` implementation in PR 2 |
207+
| `src/lib/config.server.ts` | 1 | Add `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, `EMAIL_PROVIDER` with runtime validation guard |
208+
| `src/routers/admin/email-testing-router.ts` | 1 | New tRPC router with `getTemplates`, `getProviders`, `getPreview`, `sendTest` |
209+
| `src/app/admin/email-testing/page.tsx` | 1 | New admin page; Customer.io variable preview; mailgun iframe preview added in PR 2 |
210+
| `src/app/admin/components/AppSidebar.tsx` | 1 | Add Email Testing nav link |
211+
| `src/routers/admin-router.ts` | 1 | Register `emailTestingRouter` |
212+
| `.env.local`, `.env.test`, `.env.development.local.example` | 1 | Add `EMAIL_PROVIDER=customerio` |
213+
| `src/emails/*.html` | 2 | OSS templates: replace Liquid credits conditional with `{{ credits_section }}` |
214+
| `package.json` | 2 | Add `mailgun.js` + `form-data` (keep `customerio-node` until PR 4) |

pnpm-lock.yaml

Lines changed: 28 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

src/app/admin/components/AppSidebar.tsx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import {
1515
MessageSquare,
1616
Bot,
1717
Sparkles,
18+
MailCheck,
1819
FileSearch,
1920
GitPullRequest,
2021
UserX,
@@ -133,6 +134,11 @@ const productEngineeringItems: MenuItem[] = [
133134
url: '/admin/kiloclaw',
134135
icon: () => <Server />,
135136
},
137+
{
138+
title: () => 'Email Testing',
139+
url: '/admin/email-testing',
140+
icon: () => <MailCheck />,
141+
},
136142
{
137143
title: () => 'Managed Indexing',
138144
url: '/admin/code-indexing',

0 commit comments

Comments
 (0)