|
5 | 5 |
|
6 | 6 | ## Background Processing |
7 | 7 |
|
8 | | -Long-running tasks shouldn't block HTTP requests. `rustapi-jobs` provides a robust queue system. |
| 8 | +Long-running tasks shouldn't block HTTP requests. `rustapi-jobs` provides a robust queue system that can run in-memory or be backed by Redis/Postgres. |
| 9 | + |
| 10 | +## Usage Example |
| 11 | + |
| 12 | +Here is how to set up a simple background job queue using the in-memory backend. |
| 13 | + |
| 14 | +### 1. Define the Job |
| 15 | + |
| 16 | +Jobs are simple structs that implement `Serialize` and `Deserialize`. |
| 17 | + |
| 18 | +```rust |
| 19 | +use serde::{Deserialize, Serialize}; |
| 20 | +use rustapi_jobs::{Job, JobContext, Result}; |
| 21 | +use std::sync::Arc; |
| 22 | + |
| 23 | +#[derive(Serialize, Deserialize, Debug, Clone)] |
| 24 | +struct EmailJob { |
| 25 | + to: String, |
| 26 | + subject: String, |
| 27 | + body: String, |
| 28 | +} |
| 29 | + |
| 30 | +// Implement the Job trait to define how to process it |
| 31 | +#[async_trait::async_trait] |
| 32 | +impl Job for EmailJob { |
| 33 | + const NAME: &'static str = "email_job"; |
| 34 | + |
| 35 | + async fn run(&self, _ctx: JobContext) -> Result<()> { |
| 36 | + println!("Sending email to {} with subject: {}", self.to, self.subject); |
| 37 | + // Simulate work |
| 38 | + tokio::time::sleep(std::time::Duration::from_millis(100)).await; |
| 39 | + Ok(()) |
| 40 | + } |
| 41 | +} |
| 42 | +``` |
| 43 | + |
| 44 | +### 2. Configure the Queue |
| 45 | + |
| 46 | +In your `main` function, initialize the queue and start the worker. |
9 | 47 |
|
10 | 48 | ```rust |
11 | | -// Define a job |
12 | | -#[derive(Serialize, Deserialize)] |
13 | | -struct EmailJob { to: String } |
| 49 | +use rustapi_jobs::{JobQueue, InMemoryBackend, EnqueueOptions}; |
| 50 | + |
| 51 | +#[tokio::main] |
| 52 | +async fn main() -> Result<(), Box<dyn std::error::Error>> { |
| 53 | + // 1. Create the backend |
| 54 | + let backend = InMemoryBackend::new(); |
| 55 | + |
| 56 | + // 2. Create the queue |
| 57 | + let queue = JobQueue::new(backend); |
14 | 58 |
|
15 | | -// Enqueue it |
16 | | -queue.push(EmailJob { to: "alice@example.com" }).await; |
| 59 | + // 3. Register the job type |
| 60 | + queue.register_job::<EmailJob>(); |
| 61 | + |
| 62 | + // 4. Start the worker in the background |
| 63 | + let worker_queue = queue.clone(); |
| 64 | + tokio::spawn(async move { |
| 65 | + worker_queue.start_workers().await; |
| 66 | + }); |
| 67 | + |
| 68 | + // 5. Enqueue a job |
| 69 | + queue.enqueue(EmailJob { |
| 70 | + to: "user@example.com".into(), |
| 71 | + subject: "Welcome!".into(), |
| 72 | + body: "Thanks for joining.".into(), |
| 73 | + }).await?; |
| 74 | + |
| 75 | + Ok(()) |
| 76 | +} |
17 | 77 | ``` |
18 | 78 |
|
19 | 79 | ## Backends |
20 | 80 |
|
21 | | -- **Memory**: Great for development and testing. |
22 | | -- **Redis**: High throughput persistence. |
23 | | -- **Postgres**: Transactional reliability (acid). |
| 81 | +- **Memory**: Great for development and testing. Zero infrastructure required. |
| 82 | +- **Redis**: High throughput persistence. Recommended for production. |
| 83 | +- **Postgres**: Transactional reliability (ACID). Best if you cannot lose jobs. |
| 84 | + |
| 85 | +## Reliability Features |
24 | 86 |
|
25 | | -## Reliability |
| 87 | +The worker system includes built-in reliability features: |
26 | 88 |
|
27 | | -The worker system features: |
28 | | -- **Exponential Backoff**: Automatic retries for failing jobs. |
29 | | -- **Dead Letter Queue**: Poison jobs are isolated for manual inspection. |
| 89 | +- **Exponential Backoff**: Automatically retries failing jobs with increasing delays. |
| 90 | +- **Dead Letter Queue (DLQ)**: "Poison" jobs that fail repeatedly are isolated for manual inspection. |
| 91 | +- **Concurrency Control**: Limit the number of concurrent workers to prevent overloading your system. |
0 commit comments