Skip to content

Commit e669e1b

Browse files
merenciamerencia
authored
docs: execution modes + cooperative timeout/cancellation (#186)
* docs: document execution modes and cooperative timeout/cancellation Adds a new "Execution Modes" page covering the fork and runner options, the four fork/runner combinations and when to use each (serverless, tests, SQLite, framework integrations), abortGracePeriodMs, and the cooperative timeout/cancellation model via this.abortSignal. Includes disclaimers for the sharp edges: no-fork removes crash isolation; inline jobs block the event loop and cannot be force-stopped (so timeouts and cancellation only work if the job honors this.abortSignal); grace=0 gives no cooperative window in thread mode; canceling a running inline job is best-effort. Also documents this.abortSignal in the job control page, the new options in the configuration reference, and cross-links from how-it-works and the lifecycle page. * docs: drop references to the not-yet-released @sidequest/nestjs package
1 parent 90ff994 commit e669e1b

6 files changed

Lines changed: 288 additions & 44 deletions

File tree

‎packages/docs/.vitepress/config.mts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ export default defineConfig({
9999
collapsed: false,
100100
items: [
101101
{ text: "Backends", link: "/backends" },
102+
{ text: "Execution Modes", link: "/execution-modes" },
102103
{ text: "Graceful Shutdown", link: "/graceful-shutdown" },
103104
{ text: "Cleanup", link: "/cleanup" },
104105
{ text: "Manual Job Resolution", link: "/manual-resolution" },

‎packages/docs/getting-started/configuration.md‎

Lines changed: 32 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,9 @@ await Sidequest.start({
122122
minThreads: 4,
123123
maxThreads: 8,
124124
idleWorkerTimeout: 10000, // 10 seconds
125+
fork: true, // run the engine in a child process
126+
runner: "thread", // "thread" (worker pool) or "inline"
127+
abortGracePeriodMs: 0, // grace before force-killing a timed-out/canceled thread job
125128

126129
// 4. Migration and startup
127130
skipMigration: false,
@@ -179,32 +182,35 @@ await Sidequest.start({
179182

180183
### Configuration Options
181184

182-
| Option | Description | Default |
183-
| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
184-
| `backend.driver` | Backend driver package name (SQLite, Postgres, MySQL, MongoDB) | `@sidequest/sqlite-backend` |
185-
| `backend.config` | Backend-specific connection string or [Knex configuration object](https://knexjs.org/guide/#configuration-options) | `./sidequest.sqlite` |
186-
| `dashboard.enabled` | Whether to enable the dashboard web interface | `true` |
187-
| `dashboard.port` | Port for the dashboard web interface | `8678` |
188-
| `dashboard.auth` | Basic auth configuration with `user` and `password`. If omitted, no auth is required. | `undefined` |
189-
| `queues` | Array of queue configurations with name, concurrency, priority, and state | `[]` |
190-
| `maxConcurrentJobs` | Maximum number of jobs processed simultaneously across all queues | `10` |
191-
| `minThreads` | Minimum number of worker threads to use | Number of CPU cores |
192-
| `maxThreads` | Maximum number of worker threads to use | `minThreads * 2` |
193-
| `idleWorkerTimeout` | Timeout (milliseconds) for idle workers before they are terminated | `10000` (10 seconds) |
194-
| `skipMigration` | Whether to skip database migration on startup | `false` |
195-
| `releaseStaleJobsIntervalMin` | Frequency (minutes) for releasing stale jobs. Set to `false` to disable | `60` |
196-
| `releaseStaleJobsMaxStaleMs` | Maximum age (milliseconds) for a running job to be considered stale | `600000` (10 minutes) |
197-
| `releaseStaleJobsMaxClaimedMs` | Maximum age (milliseconds) for a claimed job to be considered stale | `60000` (1 minute) |
198-
| `cleanupFinishedJobsIntervalMin` | Frequency (minutes) for cleaning up finished jobs. Set to `false` to disable | `60` |
199-
| `cleanupFinishedJobsOlderThan` | Age (milliseconds) after which finished jobs are deleted | `2592000000` (30 days) |
200-
| `logger.level` | Minimum log level (`debug`, `info`, `warn`, `error`) | `info` |
201-
| `logger.json` | Whether to output logs in JSON format | `false` |
202-
| `gracefulShutdown` | Whether to enable graceful shutdown handling | `true` |
203-
| `jobDefaults` | Default values for new jobs. Used while enqueueing | `undefined` |
204-
| `queueDefaults` | Default values for auto-created queues | `undefined` |
205-
| `manualJobResolution` | Whether to manually resolve job classes. See [Manual Job Resolution](/production/manual-resolution) | `false` |
206-
| `jobsFilePath` | Optional path to the file where job classes are exported. Ignored if `manualJobResolution` is `false`. | `undefined` |
207-
| `jobPollingInterval` | Interval (milliseconds) for polling new jobs to process. Increase this number to reduce DB load at the cost of job start latency. | `100` (100 milliseconds) |
185+
| Option | Description | Default |
186+
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
187+
| `backend.driver` | Backend driver package name (SQLite, Postgres, MySQL, MongoDB) | `@sidequest/sqlite-backend` |
188+
| `backend.config` | Backend-specific connection string or [Knex configuration object](https://knexjs.org/guide/#configuration-options) | `./sidequest.sqlite` |
189+
| `dashboard.enabled` | Whether to enable the dashboard web interface | `true` |
190+
| `dashboard.port` | Port for the dashboard web interface | `8678` |
191+
| `dashboard.auth` | Basic auth configuration with `user` and `password`. If omitted, no auth is required. | `undefined` |
192+
| `queues` | Array of queue configurations with name, concurrency, priority, and state | `[]` |
193+
| `maxConcurrentJobs` | Maximum number of jobs processed simultaneously across all queues | `10` |
194+
| `fork` | Run the engine in a child process (crash isolation). Set `false` to run in-process. See [Execution Modes](/production/execution-modes#fork-process-isolation) | `true` |
195+
| `runner` | How jobs run: `"thread"` (worker pool) or `"inline"` (current thread). See [Execution Modes](/production/execution-modes#runner-thread-pool-vs-inline) | `"thread"` |
196+
| `abortGracePeriodMs` | Grace period (ms) before a timed-out/canceled job's worker **thread** is force-killed. `0` kills immediately. No effect with `runner: "inline"`. See [Execution Modes](/production/execution-modes#cooperative-timeout-and-cancellation) | `0` |
197+
| `minThreads` | Minimum number of worker threads to use (`runner: "thread"` only) | Number of CPU cores |
198+
| `maxThreads` | Maximum number of worker threads to use (`runner: "thread"` only) | `minThreads * 2` |
199+
| `idleWorkerTimeout` | Timeout (milliseconds) for idle workers before they are terminated (`runner: "thread"` only) | `10000` (10 seconds) |
200+
| `skipMigration` | Whether to skip database migration on startup | `false` |
201+
| `releaseStaleJobsIntervalMin` | Frequency (minutes) for releasing stale jobs. Set to `false` to disable | `60` |
202+
| `releaseStaleJobsMaxStaleMs` | Maximum age (milliseconds) for a running job to be considered stale | `600000` (10 minutes) |
203+
| `releaseStaleJobsMaxClaimedMs` | Maximum age (milliseconds) for a claimed job to be considered stale | `60000` (1 minute) |
204+
| `cleanupFinishedJobsIntervalMin` | Frequency (minutes) for cleaning up finished jobs. Set to `false` to disable | `60` |
205+
| `cleanupFinishedJobsOlderThan` | Age (milliseconds) after which finished jobs are deleted | `2592000000` (30 days) |
206+
| `logger.level` | Minimum log level (`debug`, `info`, `warn`, `error`) | `info` |
207+
| `logger.json` | Whether to output logs in JSON format | `false` |
208+
| `gracefulShutdown` | Whether to enable graceful shutdown handling | `true` |
209+
| `jobDefaults` | Default values for new jobs. Used while enqueueing | `undefined` |
210+
| `queueDefaults` | Default values for auto-created queues | `undefined` |
211+
| `manualJobResolution` | Whether to manually resolve job classes. See [Manual Job Resolution](/production/manual-resolution) | `false` |
212+
| `jobsFilePath` | Optional path to the file where job classes are exported. Ignored if `manualJobResolution` is `false`. | `undefined` |
213+
| `jobPollingInterval` | Interval (milliseconds) for polling new jobs to process. Increase this number to reduce DB load at the cost of job start latency. | `100` (100 milliseconds) |
208214

209215
::: danger
210216
If `auth` is not configured and `dashboard: true` is enabled in production, the dashboard will be publicly accessible. This is a security risk and **not recommended**.

‎packages/docs/guide/jobs/lifecycle.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,11 @@ Jobs can be manually canceled at any point before completion:
6767

6868
- Waiting jobs are immediately marked as `canceled`
6969
- Claimed jobs are marked as `canceled` before execution an are prevented from running
70-
- Running jobs receive a cancellation signal and transition to `canceled`
70+
- Running jobs receive a cancellation signal via `this.abortSignal` and transition to `canceled`
71+
72+
::: warning
73+
Stopping a _running_ job depends on the [execution mode](/production/execution-modes#cooperative-timeout-and-cancellation). With the default thread pool the worker is terminated. In `runner: "inline"` mode the job cannot be force-stopped, so it must honor `this.abortSignal`; a running inline job that ignores it finishes with its own result instead of `canceled`. The same applies to job timeouts.
74+
:::
7175

7276
## Best Practices
7377

0 commit comments

Comments
 (0)