Skip to content

mail.mdx

Latest commit

 

History

History
410 lines (317 loc) · 13.2 KB

mail.mdx

File metadata and controls

410 lines (317 loc) · 13.2 KB
Raw
title Email Integration Documentation
description Complete email system with Nodemailer, Pug templates, SMTP configuration, background job integration, and type-safe helper methods for DeployStack Backend.

This document describes the email system integration in DeployStack, including the email service, template system, and recommended usage patterns.

Overview

The email system provides a comprehensive solution for sending templated emails using:

  • Background Job Queue: Async email processing with automatic retry (Recommended)
  • Nodemailer: For SMTP email delivery
  • Pug Templates: For beautiful, maintainable email templates
  • Global Settings Integration: Automatic SMTP configuration from global settings
  • Template Caching: Performance optimization for template rendering

Recommended: Sending Emails via Background Jobs

**For all user-facing email operations, use the background job queue instead of direct email sending.**

The background job approach provides significant benefits over direct email sending:

Why Background Jobs?

Performance Benefits:

  • Instant Response: API endpoints return immediately without waiting for SMTP operations
  • Non-Blocking: Email sending doesn't delay critical user flows like registration or password reset
  • Scalability: Can queue hundreds of emails without impacting application performance

Reliability Benefits:

  • Automatic Retry: Failed emails automatically retry with exponential backoff (3 attempts by default)
  • Error Isolation: Email failures don't crash user-facing operations
  • Rate Limiting: Built-in support for respecting SMTP server rate limits

Observability Benefits:

  • Job Tracking: All email operations tracked in the job queue database
  • Comprehensive Logging: Full logging of email operations, successes, and failures
  • Status Monitoring: Query job status and history for debugging

Background Job Usage

// RECOMMENDED: Queue email as background job
await server.jobQueueService.createJob('send_email', {
  to: 'user@example.com',
  subject: 'Welcome to DeployStack',
  template: 'welcome',
  variables: {
    userName: 'John Doe',
    userEmail: 'user@example.com',
    loginUrl: 'https://cloud.deploystack.io/login'
  }
});

// Job is queued and returns immediately
// Email will be sent by background worker within seconds

Real-World Example: Registration Flow

// In registration route - after user created successfully
const jobQueueService = (server as any).jobQueueService;
if (jobQueueService) {
  await jobQueueService.createJob('send_email', {
    to: user.email,
    subject: 'Verify Your Email Address',
    template: 'email-verification',
    variables: {
      userName: user.username,
      userEmail: user.email,
      verificationUrl: `https://cloud.deploystack.io/verify?token=${token}`,
      expirationTime: '24 hours'
    }
  });
  server.log.info(`Verification email queued for ${user.email}`);
}

// Registration completes instantly
// Email sent by worker in background

When to Use Background Jobs

Use background jobs for:

  • User registration/verification emails
  • Password reset emails
  • Welcome emails
  • Notification emails
  • Batch email operations
  • Any email that doesn't require immediate confirmation

Monitoring Background Jobs

// Check job status in database
const jobs = await db.select()
  .from(queueJobs)
  .where(eq(queueJobs.type, 'send_email'))
  .orderBy(desc(queueJobs.created_at))
  .limit(10);

// Job statuses: 'pending', 'processing', 'completed', 'failed'

For complete details on the job queue system, see the Background Job Queue Documentation.

Direct Email Sending (Advanced)

**Direct email sending is only recommended for specific use cases requiring immediate feedback.**

When to Use Direct Sending

Use EmailService.sendEmail() directly only for:

  • Test Emails: Admin panel SMTP testing requiring immediate success/failure feedback
  • Synchronous Validation: Flows where you must confirm email sent before proceeding
  • Development/Debugging: Local testing and troubleshooting

Direct Usage Example

import { EmailService } from '../email';

// Direct sending - blocks until complete
const result = await EmailService.sendEmail({
  to: 'admin@example.com',
  subject: 'SMTP Test Email',
  template: 'test',
  variables: {
    testDateTime: new Date().toISOString(),
    adminUser: 'Admin',
    appUrl: 'https://cloud.deploystack.io'
  }
}, request.log);

if (result.success) {
  request.log.info('Email sent successfully');
  return { success: true, messageId: result.messageId };
} else {
  request.log.error('Failed to send email:', result.error);
  return { success: false, error: result.error };
}

Performance Consideration

Direct email sending adds 2-5 seconds of latency to your endpoint. This is acceptable for admin testing but not acceptable for user-facing operations like registration.

Architecture

src/email/
├── emailService.ts       # Main email service with SMTP integration
├── templateRenderer.ts   # Pug template compilation and rendering
├── types.ts             # TypeScript interfaces and schemas
├── index.ts             # Module exports
└── templates/
    ├── layouts/
    │   ├── base.pug     # Main email layout
    │   ├── header.pug   # Email header component
    │   └── footer.pug   # Email footer component
    ├── welcome.pug              # Welcome email template
    ├── email-verification.pug   # Email verification template
    ├── password-reset.pug       # Password reset template
    ├── password-changed.pug     # Password change notification
    ├── notification.pug         # General notification template
    └── test.pug                 # SMTP test template

src/workers/
└── emailWorker.ts       # Background job worker for emails

SMTP Configuration

The email system automatically integrates with SMTP global settings. Configure these settings in the admin panel or global settings:

Setting Required Description
smtp.enabled ✅ Enable/disable email sending system-wide
smtp.host ✅ SMTP server hostname (e.g., smtp.gmail.com)
smtp.port ✅ SMTP server port (587 for TLS, 465 for SSL)
smtp.username ✅ SMTP authentication username
smtp.password ✅ SMTP authentication password (encrypted)
smtp.secure ❌ Use SSL/TLS connection (default: true)
smtp.from_name ❌ Default sender name (default: DeployStack)
smtp.from_email ❌ Default sender email (default: username)

Template System

Available Templates

Template Description Required Variables
welcome Welcome email for new users userName, userEmail, loginUrl
email-verification Email address verification userName, userEmail, verificationUrl, expirationTime
password-reset Password reset instructions userName, resetUrl, expirationTime
password-changed Password change notification userEmail, changeTime
notification General notification template title, message
test SMTP configuration testing testDateTime, adminUser, appUrl

Template Variables

All templates have access to these common variables:

  • currentYear: Current year (automatically injected)
  • appName: Application name (default: 'DeployStack')
  • supportEmail: Support email address (if provided)

Custom Templates

To create custom templates:

  1. Add new Pug template in src/email/templates/
  2. Add TypeScript types in src/email/types.ts
  3. Use with background jobs or EmailService.sendEmail()

Color Guidelines

When adding colors to email templates, follow the official DeployStack color system to maintain brand consistency. See the UI Design System page for color specifications.

Advanced Email Options

Both background jobs and direct sending support advanced options:

await jobQueueService.createJob('send_email', {
  to: 'user@example.com',
  subject: 'Important Notification',
  template: 'notification',
  variables: { title: 'Update', message: 'New features available' },
  
  // Advanced options
  from: {
    name: 'DeployStack Support',
    email: 'support@deploystack.io'
  },
  replyTo: 'support@deploystack.io',
  cc: ['admin@deploystack.io'],
  bcc: ['archive@deploystack.io'],
  attachments: [{
    filename: 'guide.pdf',
    content: pdfBuffer,
    contentType: 'application/pdf'
  }]
});

Utility Methods

Test SMTP Connection

const status = await EmailService.testConnection(request.log);
if (status.success) {
  request.log.info('SMTP connection successful');
} else {
  request.log.error('SMTP connection failed:', status.error);
}

Other Utilities

  • getSmtpStatus() - Check SMTP configuration status
  • refreshConfiguration() - Reload SMTP configuration
  • getAvailableTemplates() - Get list of available templates
  • validateTemplate() - Validate template and variables

Error Handling

Background Job Errors

Background jobs automatically retry on failure:

  • Attempt 1: Immediate execution
  • Attempt 2: After 1 second (if first fails)
  • Attempt 3: After 2 seconds (if second fails)
  • Final: Job marked as 'failed' after 3 attempts

Check failed jobs in the database:

SELECT * FROM queue_jobs 
WHERE type = 'send_email' 
  AND status = 'failed'
ORDER BY created_at DESC;

Direct Sending Errors

Direct sending provides immediate error feedback:

const result = await EmailService.sendEmail(options, logger);
if (!result.success) {
  // Handle errors immediately
  logger.error(`Email failed: ${result.error}`);
  
  // Common errors:
  // - 'SMTP not configured'
  // - 'Invalid email address'
  // - 'Connection timeout'
  // - 'Authentication failed'
}

Security & Performance

Security Features

  • Passwords encrypted in global settings
  • Secure connections (TLS/SSL) supported
  • Template security with escaped variable injection
  • No sensitive data in job queue payload

Performance Features

  • Template caching after first compilation
  • Connection pooling (max 5 concurrent connections)
  • Rate limiting (5 emails per 20 seconds)
  • Background processing doesn't block main thread

Migration Guide

Converting Existing Code to Background Jobs

Before (Blocking):

const result = await EmailService.sendWelcomeEmail({
  to: user.email,
  userName: user.name,
  userEmail: user.email,
  loginUrl: 'https://cloud.deploystack.io/login'
}, request.log);

After (Non-Blocking):

await server.jobQueueService.createJob('send_email', {
  to: user.email,
  subject: 'Welcome to DeployStack',
  template: 'welcome',
  variables: {
    userName: user.name,
    userEmail: user.email,
    loginUrl: 'https://cloud.deploystack.io/login'
  }
});

Common Usage Patterns

User Registration

Queue verification email as background job immediately after user creation.

Password Reset

Queue password reset email with secure token link.

System Notifications

Queue notification emails for system events, deployments, or status changes.

Batch Operations

Queue multiple emails with rate limiting to avoid SMTP throttling:

for (let i = 0; i < users.length; i++) {
  await jobQueueService.createJob('send_email', {
    to: users[i].email,
    subject: 'Monthly Newsletter',
    template: 'notification',
    variables: { ... }
  }, {
    scheduledFor: new Date(Date.now() + (i * 1000)) // 1 second apart
  });
}

API Reference

EmailService Methods

Method Description Returns
sendEmail(options, logger) Send an email directly using a template Promise<EmailSendResult>
testConnection(logger) Test SMTP connection Promise<{success: boolean, error?: string}>
getSmtpStatus() Check SMTP configuration status Promise<{configured: boolean, error?: string}>
refreshConfiguration() Reload SMTP configuration Promise<void>
getAvailableTemplates() Get list of available templates string[]
validateTemplate(template, variables) Validate template and variables Promise<TemplateValidationResult>

Job Queue Service Methods

Method Description Returns
createJob(type, payload, options?) Create a background job Promise<Job>
getJobStatus(jobId) Get job status and details Promise<Job>

Related Documentation

Summary

For all user-facing email operations, use background jobs. This provides instant response times, automatic retry, and better reliability. Reserve direct EmailService.sendEmail() calls for admin testing and debugging scenarios requiring immediate feedback.