Import Render skill resources

Author: ashwinm-oai

plugins/render/skills/render-background-workers/references/graceful-shutdown.md

@@ -0,0 +1,79 @@
+# Graceful shutdown for Render workers
+
+Render stops worker instances during **deploys**, **manual restarts**, and **scale-in** events. Your process must **exit cleanly** within the configured window or work may be **lost** or **duplicated** (depending on your queue’s ack/retry semantics).
+
+## Platform behavior
+
+1. Render sends **`SIGTERM`** to your process.
+2. The platform waits up to **`maxShutdownDelaySeconds`** (**1–300**, **default 30**).
+3. If the process is still running, Render sends **`SIGKILL`** (not catchable).
+
+Configure **`maxShutdownDelaySeconds`** in the **Dashboard** (service settings) or in **`render.yaml`** on the worker service. Set it to cover your **longest job** you are willing to let complete during shutdown (plus buffer for flushing metrics, closing DB pools, etc.).
+
+## General pattern
+
+1. **Stop accepting new jobs** — stop the consumer loop, pause polling, or drain the framework’s internal fetch.
+2. **Finish the current job** or **checkpoint** durable progress so another worker can resume safely.
+3. **Close connections** — Redis/Postgres pools, HTTP clients.
+4. **Exit with code 0** when done.
+
+## Python
+
+**Low-level handler**
+
+```python
+import signal
+import sys
+
+def handle_sigterm(signum, frame):
+    # set a flag; main loop checks it and stops dequeuing
+    global shutting_down
+    shutting_down = True
+
+signal.signal(signal.SIGTERM, handle_sigterm)
+```
+
+**Celery** — use lifecycle signals such as **`worker_shutting_down`** to run cleanup; ensure tasks honor a **soft time limit** or cooperative cancel flag so shutdown can finish within **`maxShutdownDelaySeconds`**.
+
+## Ruby (Sidekiq)
+
+Sidekiq **handles SIGTERM** by default: it stops fetching new work and waits for in-flight jobs up to a **configurable timeout** (`:timeout` in Sidekiq options, in seconds). Align that timeout with Render’s **`maxShutdownDelaySeconds`** (Sidekiq timeout should be **≤** platform delay minus a small margin).
+
+## Node.js
+
+```javascript
+let accept = true;
+
+process.on("SIGTERM", async () => {
+  accept = false;
+  await worker.close(); // BullMQ: stops accepting, waits for active jobs
+  process.exit(0);
+});
+```
+
+**BullMQ** — prefer **`worker.close()`** (and **`queue.close()`** where applicable) so active jobs complete per library defaults; tune **`stalledInterval`** / job locks if you need stricter bounds.
+
+## Go
+
+Use **`signal.NotifyContext`** (or `signal.Notify` + `context.WithCancel`) to cancel a root context passed into your consumer loop and job handlers; wait on **`sync.WaitGroup`** or channels until in-flight work finishes, then exit.
+
+```go
+ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
+defer stop()
+// run consumer until ctx.Done(), then drain workers
+```
+
+## Anti-patterns
+
+- **Ignoring SIGTERM** — the process survives until **`SIGKILL`**, often **mid-job**, causing **lost work** or **stuck** queue entries.
+- **`maxShutdownDelaySeconds` too low** for your p95 job duration — frequent **hard kills** and retries.
+- **No idempotency** — if a job is retried after an ambiguous failure at shutdown, **duplicate side effects** can occur.
+
+## Checklist
+
+| Item | Action |
+|------|--------|
+| Delay | Set **`maxShutdownDelaySeconds`** ≥ longest graceful completion you need |
+| Consumer | On SIGTERM, **stop dequeuing** first |
+| Jobs | **Idempotent** handlers or explicit **checkpoints** |
+| Framework | Use built-in **drain** / **close** APIs (Sidekiq, BullMQ, Celery signals) where available |

plugins/render/skills/render-background-workers/references/queue-framework-setup.md

@@ -0,0 +1,272 @@
+# Queue framework setup on Render
+
+Minimal patterns for running each stack as a **`type: worker`** service. Replace regions, plans, and commands to match your repo.
+
+**Shared Key Value (Redis-style) wiring** — add once, reference from every consumer that needs the broker:
+
+```yaml
+services:
+  - type: keyvalue
+    name: jobs
+    region: oregon
+    plan: starter
+    ipAllowList: []
+```
+
+**Env var** (worker or web that enqueues):
+
+```yaml
+envVars:
+  - key: REDIS_URL
+    fromService:
+      name: jobs
+      type: keyvalue
+      property: connectionString
+```
+
+Set Key Value **maxmemory policy** to **`noeviction`** in the Dashboard so queue keys are never evicted like cache entries.
+
+---
+
+## Celery (Python)
+
+**Install**
+
+```bash
+pip install "celery[redis]"
+```
+
+**Minimal app** (`tasks.py` / package root):
+
+```python
+import os
+from celery import Celery
+
+app = Celery("tasks", broker=os.environ["REDIS_URL"])
+
+@app.task
+def add(x, y):
+    return x + y
+```
+
+**Start command** (adjust module path):
+
+```bash
+celery -A tasks worker --loglevel=info
+```
+
+**Blueprint (worker)**
+
+```yaml
+  - type: worker
+    name: celery-worker
+    runtime: python
+    region: oregon
+    plan: starter
+    buildCommand: pip install -r requirements.txt
+    startCommand: celery -A tasks worker --loglevel=info
+    envVars:
+      - key: REDIS_URL
+        fromService:
+          name: jobs
+          type: keyvalue
+          property: connectionString
+```
+
+---
+
+## Sidekiq (Ruby)
+
+**Gemfile**
+
+```ruby
+gem "sidekiq"
+```
+
+**Initializer** (`config/initializers/sidekiq.rb` or equivalent):
+
+```ruby
+Sidekiq.configure_server do |config|
+  config.redis = { url: ENV.fetch("REDIS_URL") }
+end
+
+Sidekiq.configure_client do |config|
+  config.redis = { url: ENV.fetch("REDIS_URL") }
+end
+```
+
+**Start command**
+
+```bash
+bundle exec sidekiq
+```
+
+**Blueprint (worker)**
+
+```yaml
+  - type: worker
+    name: sidekiq
+    runtime: ruby
+    region: oregon
+    plan: starter
+    buildCommand: bundle install
+    startCommand: bundle exec sidekiq
+    envVars:
+      - key: REDIS_URL
+        fromService:
+          name: jobs
+          type: keyvalue
+          property: connectionString
+```
+
+---
+
+## BullMQ (Node.js)
+
+**Install**
+
+```bash
+npm install bullmq ioredis
+```
+
+**Minimal worker** (`worker.js`):
+
+```javascript
+const { Worker } = require("bullmq");
+const IORedis = require("ioredis");
+
+const connection = new IORedis(process.env.REDIS_URL, {
+  maxRetriesPerRequest: null,
+});
+
+const worker = new Worker("myqueue", async (job) => job.data, { connection });
+```
+
+**Start command**
+
+```bash
+node worker.js
+```
+
+**Blueprint (worker)**
+
+```yaml
+  - type: worker
+    name: bullmq-worker
+    runtime: node
+    region: oregon
+    plan: starter
+    buildCommand: npm ci
+    startCommand: node worker.js
+    envVars:
+      - key: REDIS_URL
+        fromService:
+          name: jobs
+          type: keyvalue
+          property: connectionString
+```
+
+---
+
+## Asynq (Go)
+
+**Module**
+
+```bash
+go get github.com/hibiken/asynq
+```
+
+**Minimal worker** (simplified):
+
+```go
+package main
+
+import (
+    "log"
+    "os"
+
+    "github.com/hibiken/asynq"
+)
+
+func main() {
+    redisOpt, err := asynq.ParseRedisURI(os.Getenv("REDIS_URL"))
+    if err != nil {
+        log.Fatal(err)
+    }
+    srv := asynq.NewServer(redisOpt, asynq.Config{Concurrency: 10})
+    mux := asynq.NewServeMux()
+    // mux.HandleFunc("task:type", handler)
+    if err := srv.Run(mux); err != nil {
+        log.Fatal(err)
+    }
+}
+```
+
+**Start command** (after `go build` in `buildCommand` or run `go run`):
+
+```bash
+./worker
+```
+
+**Blueprint (worker)**
+
+```yaml
+  - type: worker
+    name: asynq-worker
+    runtime: go
+    region: oregon
+    plan: starter
+    buildCommand: go build -o worker ./cmd/worker
+    startCommand: ./worker
+    envVars:
+      - key: REDIS_URL
+        fromService:
+          name: jobs
+          type: keyvalue
+          property: connectionString
+```
+
+---
+
+## Oban (Elixir)
+
+Oban uses **PostgreSQL** as its queue store—**not** Redis. Provision a Render **PostgreSQL** database and wire **`DATABASE_URL`**.
+
+**Config** (`config/runtime.exs` or `config/prod.exs` pattern):
+
+```elixir
+config :my_app, Oban,
+  repo: MyApp.Repo,
+  queues: [default: 10]
+```
+
+**Start command** (typical Release):
+
+```bash
+_build/prod/rel/my_app/bin/my_app start
+```
+
+**Blueprint (worker)** — `fromDatabase` for `DATABASE_URL`:
+
+```yaml
+databases:
+  - name: app-db
+    plan: basic-256mb
+    region: oregon
+
+services:
+  - type: worker
+    name: oban-worker
+    runtime: elixir
+    region: oregon
+    plan: starter
+    buildCommand: mix deps.get && mix release
+    startCommand: _build/prod/rel/my_app/bin/my_app start
+    envVars:
+      - key: DATABASE_URL
+        fromDatabase:
+          name: app-db
+          property: connectionString
+```
+
+Do **not** point Oban at Key Value unless you are using an **experimental or custom** backend; default Oban is **Postgres-only**.