cleanup

Author: muniter

.gitignore

@@ -3,4 +3,6 @@ node_modules/
 *.sqlite
 *.sqlite3
 dist/
-*.data
+*.data
+queue.index.json
+queue.index.json*

example-queue-project/.email-queue/index.json

@@ -0,0 +1,105 @@
+# Queue Example Project
+
+This example demonstrates how to use `@muniter/queue` in a realistic production setup with separate processes for job producers and consumers.
+
+## Architecture
+
+This example shows the **separation of concerns** pattern recommended for production applications:
+
+- **Queue Configuration** (`queues.ts`) - Centralized queue initialization, configuration, and event listeners
+- **Job Producers** (`add-job.ts`) - Add jobs to queues (e.g., from web servers, API endpoints)
+- **Job Consumers** (`process-jobs.ts`) - Process jobs continuously (e.g., background worker processes)
+- **Shared Types** (`types.ts`) - Job type definitions shared between producers and consumers
+
+## Features Demonstrated
+
+- **Multiple Queue Types**: Database queue (SQLite) and File queue
+- **Type Safety**: Full TypeScript support with job payload validation
+- **Event System**: Comprehensive logging via queue lifecycle events
+- **Worker Management**: Using Worker class for clean process organization
+- **Error Handling**: Simulated failures and proper error events
+- **Delayed Jobs**: Jobs that run after a specified delay
+- **Production Pattern**: Separate entry points for different concerns
+
+## Setup
+
+1. Install dependencies:
+```bash
+npm install
+```
+
+2. Build the project:
+```bash
+npm run build
+```
+
+## Usage
+
+### 1. Start the Job Processors (Workers)
+
+In one terminal, start the background workers that will process jobs:
+
+```bash
+npm run process-jobs
+```
+
+This will:
+- Initialize the database and file queue
+- Register handlers for all job types
+- Start workers that continuously poll for new jobs
+- Display real-time processing logs
+
+### 2. Add Jobs to the Queues
+
+In another terminal, add jobs to be processed:
+
+```bash
+npm run add-job
+```
+
+This will add various types of jobs:
+- Welcome emails
+- Notification emails  
+- Image processing tasks
+- Report generation (with delay)
+- Batch processing jobs
+
+You'll see the jobs being processed in real-time in the worker terminal.
+
+## Job Types
+
+### Email Jobs (File Queue)
+- `welcome-email`: Send welcome emails to new users
+- `notification`: Send system notifications
+
+### General Jobs (Database Queue)  
+- `process-image`: Resize and optimize images
+- `generate-report`: Generate business reports
+
+## Production Deployment
+
+In a real production environment, you would:
+
+1. **Web Server** - Run `add-job.ts` logic in your API endpoints to add jobs
+2. **Worker Processes** - Deploy `process-jobs.ts` as separate background services
+3. **Scaling** - Run multiple worker processes for high throughput
+4. **Monitoring** - Use the event system to send metrics to monitoring services
+5. **Error Handling** - Implement retry logic and dead letter queues
+
+## Files
+
+- `queues.ts` - Centralized queue configuration, initialization, and event listeners
+- `add-job.ts` - Job producer (adds jobs to queues)
+- `process-jobs.ts` - Job consumer (processes jobs continuously)  
+- `types.ts` - Shared job type definitions
+- `sqlite-adapter.ts` - Database adapter implementation
+- `database.ts` - Database initialization
+- `index.ts` - Original combined example (legacy)
+
+## Try It
+
+1. Start workers: `npm run process-jobs`
+2. In another terminal: `npm run add-job`
+3. Watch the jobs get processed in real-time!
+4. Try adding jobs multiple times to see concurrent processing
+5. Stop workers with Ctrl+C

example-queue-project/README.md

@@ -10,7 +10,8 @@
     "dev": "tsx src/index.ts",
     "start": "node dist/index.js",
     "add-job": "tsx src/add-job.ts",
-    "process-jobs": "tsx src/process-jobs.ts"
+    "process-jobs": "tsx src/process-jobs.ts",
+    "demo": "echo 'Run \"npm run process-jobs\" in one terminal, then \"npm run add-job\" in another!'"
   },
   "dependencies": {
     "@muniter/queue": "file:..",

example-queue-project/package.json

@@ -1,5 +1,4 @@
 import sqlite3 from 'sqlite3';
-import { promisify } from 'util';
 import path from 'path';
 import { fileURLToPath } from 'url';
 

example-queue-project/queue-data-events/index.json

@@ -0,0 +1,31 @@
+import { FileQueue } from "@muniter/queue";
+
+interface EmailJobs {
+  'welcome-email': { to: string; name: string };
+  'notification': { to: string; subject: string; body: string };
+}
+
+
+export const emailQueue = new FileQueue<EmailJobs>({
+    path: "./email-queue",
+})
+
+// Register job handlers for email queue
+emailQueue.onJob('welcome-email', async (payload) => {
+    const { to, name } = payload;
+    console.log(`Sending welcome email to ${to} (${name})`);
+    await new Promise(resolve => setTimeout(resolve, 1000));
+
+    if (Math.random() > 0.8) {
+        throw new Error('Email service temporarily unavailable');
+    }
+
+    console.log(`Welcome email sent successfully to ${to}`);
+});
+
+emailQueue.onJob('notification', async (payload) => {
+    const { to, subject, body } = payload;
+    console.log(`Sending notification email to ${to}: ${subject}`);
+    await new Promise(resolve => setTimeout(resolve, 500));
+    console.log(`Notification sent successfully`);
+});