|
| 1 | +# Migrate Email Sending from Customer.io to Mailgun |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +Replace Customer.io transactional email API with Mailgun's Node.js SDK (`mailgun.js`). Instead of referencing remote Customer.io template IDs, we'll render the local HTML templates in `src/emails/*.html` server-side and send the full HTML body to Mailgun. |
| 6 | + |
| 7 | +## Current Architecture |
| 8 | + |
| 9 | +- **Provider**: Customer.io via `customerio-node` SDK |
| 10 | +- **Config**: `CUSTOMERIO_EMAIL_API_KEY` in `config.server.ts` |
| 11 | +- **Templates**: Remote, stored in Customer.io, referenced by numeric ID (`'10'`, `'11'`, etc.) |
| 12 | +- **Variable substitution**: Done by Customer.io using Liquid syntax (`{{ var }}`, `{% if %}`) |
| 13 | +- **Send function**: `send()` in `email.ts` creates `APIClient` + `SendEmailRequest` |
| 14 | + |
| 15 | +## Target Architecture |
| 16 | + |
| 17 | +- **Provider**: Mailgun via `mailgun.js` SDK + `form-data` |
| 18 | +- **Config**: `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, and `EMAIL_PROVIDER` in `config.server.ts` |
| 19 | +- **Templates**: Local HTML files in `src/emails/*.html` |
| 20 | +- **Variable substitution**: Done server-side in Node.js before sending |
| 21 | +- **Send function**: `send()` routes to either Customer.io or Mailgun backend based on `EMAIL_PROVIDER` env var |
| 22 | + |
| 23 | +### Dual-Provider Routing |
| 24 | + |
| 25 | +During the transition, `email.ts` delegates to a provider backend selected by the `EMAIL_PROVIDER` env var (`customerio` | `mailgun`, defaulting to `customerio`). The existing Customer.io logic moves to `src/lib/email-customerio.ts`; the new Mailgun logic lives in `src/lib/email-mailgun.ts`. |
| 26 | + |
| 27 | +PostHog feature flags are **not** suitable for this toggle because some emails target logged-out users (magic link, invitations) where there is no user context for flag evaluation. A simple env var is the right mechanism. |
| 28 | + |
| 29 | +```typescript |
| 30 | +import { sendViaCustomerIo } from './email-customerio'; |
| 31 | +import { sendViaMailgun } from './email-mailgun'; |
| 32 | + |
| 33 | +type SendParams = { |
| 34 | + to: string; |
| 35 | + templateName: TemplateName; |
| 36 | + // Record<string, unknown> in PR 1 to support customerio's native types (numbers, booleans). |
| 37 | + // Tightened to Record<string, string> in PR 2 once renderTemplate is added. |
| 38 | + templateVars: Record<string, unknown>; |
| 39 | +}; |
| 40 | + |
| 41 | +function send(params: SendParams) { |
| 42 | + if (EMAIL_PROVIDER === 'mailgun') { |
| 43 | + // PR 2: looks up subjects[templateName], calls renderTemplate, then sendViaMailgun |
| 44 | + return sendViaMailgun(); |
| 45 | + } |
| 46 | + return sendViaCustomerIo({ |
| 47 | + transactional_message_id: templates[params.templateName], |
| 48 | + to: params.to, |
| 49 | + message_data: params.templateVars, |
| 50 | + identifiers: { email: params.to }, |
| 51 | + reply_to: 'hi@kilocode.ai', |
| 52 | + }); |
| 53 | +} |
| 54 | +``` |
| 55 | + |
| 56 | +Each `send*Email` function builds `templateVars` and calls `send()` once. Customer.io receives its native `transactional_message_id` + `message_data` interface. Mailgun (PR 2) renders HTML locally and looks up the subject from `subjects[templateName]` — no subject arg needed at call sites. All emails use `{ email: to }` as the Customer.io identifier. |
| 57 | + |
| 58 | +### Template Rendering |
| 59 | + |
| 60 | +A `renderTemplate()` function reads an HTML file from `src/emails/{name}.html` and replaces `{{ variable_name }}` placeholders with provided values: |
| 61 | + |
| 62 | +```typescript |
| 63 | +function renderTemplate(name: string, vars: Record<string, string>) { |
| 64 | + const templatePath = path.join(process.cwd(), 'src', 'emails', `${name}.html`); |
| 65 | + const html = fs.readFileSync(templatePath, 'utf-8'); |
| 66 | + return html.replace(/\{\{\s*(\w+)\s*\}\}/g, (_, key) => { |
| 67 | + if (!(key in vars)) { |
| 68 | + throw new Error(`Missing template variable '${key}' in email template '${name}'`); |
| 69 | + } |
| 70 | + return vars[key]; |
| 71 | + }); |
| 72 | +} |
| 73 | +``` |
| 74 | + |
| 75 | +The 3 OSS templates currently use a Liquid conditional `{% if has_credits %}...{% endif %}` which must be replaced with a `{{ credits_section }}` placeholder. The JS builds the snippet: |
| 76 | + |
| 77 | +```typescript |
| 78 | +function buildCreditsSection(monthlyCreditsUsd: number): string { |
| 79 | + if (monthlyCreditsUsd <= 0) return ''; |
| 80 | + return `<br />• <strong style="color: #d1d5db">$${monthlyCreditsUsd} USD in Kilo credits</strong>, which reset every 30 days`; |
| 81 | +} |
| 82 | +``` |
| 83 | + |
| 84 | +### Subject Lines |
| 85 | + |
| 86 | +Customer.io stores subjects in the remote template. Mailgun needs them passed explicitly. Rather than adding a subject argument to every `send*Email` call site, subjects are stored in a `subjects` map in `email.ts` keyed by `TemplateName`. The Mailgun branch of `send()` looks up `subjects[templateName]` internally — call sites are unchanged. |
| 87 | + |
| 88 | +| Template name | Subject | |
| 89 | +| --------------------------- | --------------------------------------------- | |
| 90 | +| `orgSubscription` | "Welcome to Kilo for Teams!" | |
| 91 | +| `orgRenewed` | "Kilo: Your Teams Subscription Renewal" | |
| 92 | +| `orgCancelled` | "Kilo: Your Teams Subscription is Cancelled" | |
| 93 | +| `orgSSOUserJoined` | "Kilo: New SSO User Joined Your Organization" | |
| 94 | +| `orgInvitation` | "Kilo: Teams Invitation" | |
| 95 | +| `magicLink` | "Sign in to Kilo Code" | |
| 96 | +| `balanceAlert` | "Kilo: Low Balance Alert" | |
| 97 | +| `autoTopUpFailed` | "Kilo: Auto Top-Up Failed" | |
| 98 | +| `ossInviteNewUser` | "Kilo: OSS Sponsorship Offer" | |
| 99 | +| `ossInviteExistingUser` | "Kilo: OSS Sponsorship Offer" | |
| 100 | +| `ossExistingOrgProvisioned` | "Kilo: OSS Sponsorship Offer" | |
| 101 | +| `deployFailed` | "Kilo: Your Deployment Failed" | |
| 102 | + |
| 103 | +### Template Variables |
| 104 | + |
| 105 | +All templates use `{{ variable }}` interpolation. `year` is always `String(new Date().getFullYear())`. `credits_section` is built in JS (HTML snippet or empty string). |
| 106 | + |
| 107 | +| Template file | Variables | |
| 108 | +| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | |
| 109 | +| `orgSubscription.html` | `seats`, `organization_url`, `invoices_url`, `year` | |
| 110 | +| `orgRenewed.html` | `seats`, `invoices_url`, `year` | |
| 111 | +| `orgCancelled.html` | `invoices_url`, `year` | |
| 112 | +| `orgSSOUserJoined.html` | `new_user_email`, `organization_url`, `year` | |
| 113 | +| `orgInvitation.html` | `organization_name`, `inviter_name`, `accept_invite_url`, `year` | |
| 114 | +| `magicLink.html` | `magic_link_url`, `email`, `expires_in`, `year` | |
| 115 | +| `balanceAlert.html` | `minimum_balance`, `organization_url`, `year` | |
| 116 | +| `autoTopUpFailed.html` | `reason`, `credits_url`, `year` | |
| 117 | +| `ossInviteNewUser.html` | `tier_name`, `seats`, `seat_value`, `credits_section`, `accept_invite_url`, `integrations_url`, `code_reviews_url`, `year` | |
| 118 | +| `ossInviteExistingUser.html` | `tier_name`, `seats`, `seat_value`, `credits_section`, `organization_url`, `integrations_url`, `code_reviews_url`, `year` | |
| 119 | +| `ossExistingOrgProvisioned.html` | `tier_name`, `seats`, `seat_value`, `credits_section`, `organization_url`, `integrations_url`, `code_reviews_url`, `year` | |
| 120 | +| `deployFailed.html` | `deployment_name`, `deployment_url`, `repository`, `year` | |
| 121 | + |
| 122 | +### Admin Email Testing Page |
| 123 | + |
| 124 | +An admin page for sending test emails and previewing output. Controls: |
| 125 | + |
| 126 | +- **Template** — dropdown of all available templates (one entry per `send*Email` function) |
| 127 | +- **Provider** — dropdown of implemented providers; only shows providers that are actually wired up (starts as just `customerio`; `mailgun` appears once `email-mailgun.ts` is implemented) |
| 128 | +- **Recipient** — text input pre-filled with the logged-in admin's email address, overridable with any valid email |
| 129 | + |
| 130 | +Preview pane (updates when template or provider selection changes): |
| 131 | + |
| 132 | +- **Customer.io**: shows the `message_data` variables that would be sent to the Customer.io API (key/value pairs, not rendered HTML) |
| 133 | +- **Mailgun**: shows the fully rendered HTML email in an iframe |
| 134 | + |
| 135 | +Submitting sends a test email using hardcoded representative fixture data for the selected template, routed through the selected provider (ignoring the `EMAIL_PROVIDER` env var for this request). This page is useful both during QA and long-term for testing new templates. |
| 136 | + |
| 137 | +### Environment Variables |
| 138 | + |
| 139 | +- `EMAIL_PROVIDER` — Which email backend to use: `customerio` or `mailgun`. Defaults to `customerio` when unset. Setting an unrecognised value will throw at startup. |
| 140 | +- `MAILGUN_API_KEY` — Mailgun API key (starts with `key-...`) |
| 141 | +- `MAILGUN_DOMAIN` — Mailgun sending domain (`app.kilocode.ai`) |
| 142 | + |
| 143 | +### Dependencies |
| 144 | + |
| 145 | +Install: `pnpm add mailgun.js form-data` (keep `customerio-node` until cleanup) |
| 146 | + |
| 147 | +## Migration Strategy |
| 148 | + |
| 149 | +### PR 1: Routing Shell + Admin Testing Page ✅ Done |
| 150 | + |
| 151 | +- ~~Add `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, `EMAIL_PROVIDER` to `src/lib/config.server.ts`~~ ✅ |
| 152 | +- ~~Extract existing Customer.io send logic into `src/lib/email-customerio.ts` with `sendViaCustomerIo()`~~ ✅ |
| 153 | +- ~~Create `src/lib/email-mailgun.ts` as a stub (throws "not yet implemented")~~ ✅ |
| 154 | +- ~~Add routing layer in `email.ts`: `send()` takes `{ to, templateName, templateVars }` and delegates to the active provider~~ ✅ |
| 155 | +- ~~Refactor all `send*Email` functions to use `templateName` + `templateVars` instead of building `SendEmailRequestOptions` directly~~ ✅ |
| 156 | +- ~~Remove `inviteCode` from `OrganizationInviteEmailData` and `OssInviteEmailData`; all sends use `{ email: to }` as the Customer.io identifier~~ ✅ |
| 157 | +- ~~Export `templates`, `subjects`, `TemplateName` from `email.ts`~~ ✅ |
| 158 | +- ~~Build the admin testing page with template/provider/recipient controls and preview pane~~ ✅ |
| 159 | +- ~~Hardcode representative fixture data for each template~~ ✅ |
| 160 | +- ~~The provider dropdown only shows `customerio` at this point~~ ✅ |
| 161 | +- ~~Add `EMAIL_PROVIDER=customerio` to `.env.local`, `.env.test`, and `.env.development.local.example`~~ ✅ |
| 162 | +- Deploy with `EMAIL_PROVIDER=customerio` — no production change |
| 163 | + |
| 164 | +### PR 2: Mailgun Send Logic + Template Rendering ✅ Done (merged into PR 1 branch) |
| 165 | + |
| 166 | +Pre-requisite template work: |
| 167 | + |
| 168 | +- ~~Replace `{{ "now" | date: "%Y" }}` with `{{ year }}` in all 12 templates~~ ✅ |
| 169 | +- ~~Re-include `src/emails/*.html` and `src/emails/AGENTS.md`~~ ✅ |
| 170 | +- ~~Replace the `{% if has_credits %}...{% endif %}` block with `{{ credits_section }}` in the 3 OSS templates (`ossInviteNewUser.html`, `ossInviteExistingUser.html`, `ossExistingOrgProvisioned.html`)~~ ✅ |
| 171 | + |
| 172 | +Implementation: |
| 173 | + |
| 174 | +- ~~Install `mailgun.js` + `form-data`~~ ✅ |
| 175 | +- ~~Implement `sendViaMailgun({ to, subject, html })` in `src/lib/email-mailgun.ts`~~ ✅ |
| 176 | +- ~~Add `renderTemplate()` and `buildCreditsSection()` to `email.ts`~~ ✅ |
| 177 | +- ~~Update `send()` in `email.ts`: mailgun branch looks up `subjects[templateName]`, calls `renderTemplate(templateName, templateVars)`, then `sendViaMailgun()`. Tighten `templateVars` to `Record<string, string>` — update all `send*Email` call sites (OSS functions replace `has_credits: boolean` + `monthly_credits_usd: number` with `credits_section: buildCreditsSection(...)`)~~ ✅ |
| 178 | +- ~~Add `mailgun` to the providers list in `email-testing-router.ts`; update `getPreview` to return rendered HTML for mailgun; update `sendTest` to call `sendViaMailgun` directly~~ ✅ |
| 179 | +- ~~The admin page now shows `mailgun` in the provider dropdown with a full HTML iframe preview~~ ✅ |
| 180 | +- Deploy with `EMAIL_PROVIDER=customerio` — still no production change |
| 181 | + |
| 182 | +**QA** (no PR): Use the admin testing page to send each template via Mailgun to real inboxes. Compare rendered output against the Customer.io versions. Verify all variable substitution, styling, links. |
| 183 | + |
| 184 | +### PR 3: Flip to Mailgun |
| 185 | + |
| 186 | +- Set `EMAIL_PROVIDER=mailgun` in production |
| 187 | +- Monitor delivery, bounce rates, rendering across email clients |
| 188 | + |
| 189 | +### PR 4: Cleanup |
| 190 | + |
| 191 | +After the provider switch is confirmed stable: |
| 192 | + |
| 193 | +- Remove `customerio-node` from `package.json` |
| 194 | +- Remove `CUSTOMERIO_EMAIL_API_KEY` from `config.server.ts` |
| 195 | +- Remove the `templates` map (no longer needed) |
| 196 | +- Remove `SendEmailRequestOptions` type imports |
| 197 | +- Remove `EMAIL_PROVIDER` routing logic from `email.ts` and delete `src/lib/email-customerio.ts` |
| 198 | +- Note: `src/lib/external-services.ts` also references Customer.io for user deletion — that's a separate concern and uses different API keys (`CUSTOMERIO_SITE_ID`, `CUSTOMERIO_API_KEY`) |
| 199 | + |
| 200 | +## Files Changed |
| 201 | + |
| 202 | +| File | PR | Change | |
| 203 | +| ----------------------------------------------------------- | --- | --------------------------------------------------------------------------------------------------- | |
| 204 | +| `src/lib/email.ts` | 1 | Routing via `send({ templateName, templateVars })`; exports `templates`, `subjects`, `TemplateName` | |
| 205 | +| `src/lib/email-customerio.ts` | 1 | Extracted `sendViaCustomerIo()`; minimal PII-free logging | |
| 206 | +| `src/lib/email-mailgun.ts` | 1→2 | Stub in PR 1 (throws); full `sendViaMailgun({ to, subject, html })` implementation in PR 2 | |
| 207 | +| `src/lib/config.server.ts` | 1 | Add `MAILGUN_API_KEY`, `MAILGUN_DOMAIN`, `EMAIL_PROVIDER` with runtime validation guard | |
| 208 | +| `src/routers/admin/email-testing-router.ts` | 1 | New tRPC router with `getTemplates`, `getProviders`, `getPreview`, `sendTest` | |
| 209 | +| `src/app/admin/email-testing/page.tsx` | 1 | New admin page; Customer.io variable preview; mailgun iframe preview added in PR 2 | |
| 210 | +| `src/app/admin/components/AppSidebar.tsx` | 1 | Add Email Testing nav link | |
| 211 | +| `src/routers/admin-router.ts` | 1 | Register `emailTestingRouter` | |
| 212 | +| `.env.local`, `.env.test`, `.env.development.local.example` | 1 | Add `EMAIL_PROVIDER=customerio` | |
| 213 | +| `src/emails/*.html` | 2 | OSS templates: replace Liquid credits conditional with `{{ credits_section }}` | |
| 214 | +| `package.json` | 2 | Add `mailgun.js` + `form-data` (keep `customerio-node` until PR 4) | |
0 commit comments