From b85599043c495c994757ab42da7ed94cd08e150b Mon Sep 17 00:00:00 2001 From: Matt Sodomsky Date: Sun, 5 Jul 2026 10:08:32 -0500 Subject: [PATCH 1/4] Background tasks: advanced settings docs Documents the background lane and per-task concurrency caps shipping in lightward/mechanic-api#2290: the yield contract, immediate reclassification of queued runs, the guaranteed-progress drip, the backlog limit, and the bulk-operations expiry caveat. Co-Authored-By: Claude Fable 5 --- SUMMARY.md | 1 + core/tasks/advanced-settings/README.md | 1 + .../advanced-settings/background-tasks.md | 27 +++++++++++++++++++ 3 files changed, 29 insertions(+) create mode 100644 core/tasks/advanced-settings/background-tasks.md diff --git a/SUMMARY.md b/SUMMARY.md index d2aa63e..bfa7298 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -78,6 +78,7 @@ * [Documentation](core/tasks/advanced-settings/documentation.md) * [JavaScript](core/tasks/advanced-settings/javascript.md) * [Perform action runs in sequence](core/tasks/advanced-settings/perform-action-runs-in-sequence.md) + * [Background tasks](core/tasks/advanced-settings/background-tasks.md) * [Import and export](core/tasks/import-and-export.md) * [User Form](core/tasks/user-form.md) * [Actions](core/actions/README.md) diff --git a/core/tasks/advanced-settings/README.md b/core/tasks/advanced-settings/README.md index 8319926..1052fe5 100644 --- a/core/tasks/advanced-settings/README.md +++ b/core/tasks/advanced-settings/README.md @@ -10,3 +10,4 @@ Advanced settings are per-task configuration options beyond the task's code and * [Documentation](documentation.md) — add Markdown documentation that is displayed to users alongside the task * [JavaScript](javascript.md) — attach JavaScript to the online storefront or order status page * [Perform action runs in sequence](perform-action-runs-in-sequence.md) — force the task's actions to run one at a time, in order, instead of concurrently +* [Background tasks](background-tasks.md) — have a task's runs wait until your other tasks are caught up, and/or cap how many of its runs may run at once diff --git a/core/tasks/advanced-settings/background-tasks.md b/core/tasks/advanced-settings/background-tasks.md new file mode 100644 index 0000000..3cbec39 --- /dev/null +++ b/core/tasks/advanced-settings/background-tasks.md @@ -0,0 +1,27 @@ +# Background tasks + +Mechanic's [run system](../../runs/) gives every shop a queue of work, performing runs as quickly as the shop's capacity allows. Normally, all of a shop's tasks share that capacity on a first-come, first-served basis — which means a single task generating tens of thousands of runs (say, a backfill tagging every historical order) can leave your time-sensitive automations waiting in line behind it. + +Two advanced task settings, found in the "Runtime" tab of the advanced task editor, let you decide how a task shares your queue: + +* **Run this task in the background** – Background runs wait until your other tasks are caught up. A background task's runs only ever receive spare capacity: whenever any of your other work is waiting, that work goes first, and incoming events are always processed ahead of background runs. Use this for bulk jobs and backfills that aren't time-sensitive. +* **Maximum concurrent runs** – Caps how many of this task's runs may be running at the same time, leaving room in your queue for your other tasks. Leave this blank for no limit. + +## How background tasks behave + +* The background setting applies to the task's runs — both its task runs and the action runs they generate. Event processing is unaffected: events are always received and interpreted at normal speed, so your run history fills in promptly even when the resulting work waits. +* Changing the setting takes effect immediately, in both directions, including for runs that are already waiting in the queue. If a bulk task is slowing down your store right now, marking it as a background task moves its queued work to the back of the line right away — and un-checking the setting brings a background task's waiting runs back to normal scheduling just as quickly. +* A busy shop may pause background work entirely for a while — that's the setting working as intended, not a stuck task. Mechanic still guarantees slow progress: a background run that has been waiting for several hours is given the next available slot, so background work can be paused, but never stopped for good. +* A background task's backlog has a limit. Once a background task has 10,000 runs waiting, new runs for that task are cancelled on arrival (each with an error message explaining why), until the backlog drains or the task leaves the background. This keeps a paused background task from accumulating unbounded work. + +{% hint style="warning" %} +Background tasks are not recommended for tasks that use [bulk operations](../../../platform/graphql/basics/bulk-operations.md). Shopify's bulk operation results are only available for a limited time, and a background task may wait long enough for its results to expire. +{% endhint %} + +## Maximum concurrent runs + +A concurrency limit is useful when a task's work should keep moving, but shouldn't be allowed to fill your whole queue. A task with a limit of 2 will never have more than 2 of its runs (task runs plus action runs) executing at once, no matter how much of its work is waiting — the rest of your queue stays available for your other tasks. + +Compared to the background setting, a concurrency limit is the gentler tool: a background task yields *completely* to your other work, while a concurrency-limited task keeps a steady, bounded pace regardless of what else is happening. The two settings can be combined. + +Retrying a run manually always runs it immediately, regardless of these settings. From 94dada387de712e3f5bda2d3444aca00858100b4 Mon Sep 17 00:00:00 2001 From: Matt Sodomsky Date: Sun, 5 Jul 2026 13:08:33 -0500 Subject: [PATCH 2/4] Background tasks never drop work The backlog cap was removed from the implementation (lightward/mechanic-api#2290): background runs queue without limit, like everything else in Mechanic. Say so explicitly. Co-Authored-By: Claude Fable 5 --- core/tasks/advanced-settings/background-tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/core/tasks/advanced-settings/background-tasks.md b/core/tasks/advanced-settings/background-tasks.md index 3cbec39..e7caf17 100644 --- a/core/tasks/advanced-settings/background-tasks.md +++ b/core/tasks/advanced-settings/background-tasks.md @@ -12,7 +12,7 @@ Two advanced task settings, found in the "Runtime" tab of the advanced task edit * The background setting applies to the task's runs — both its task runs and the action runs they generate. Event processing is unaffected: events are always received and interpreted at normal speed, so your run history fills in promptly even when the resulting work waits. * Changing the setting takes effect immediately, in both directions, including for runs that are already waiting in the queue. If a bulk task is slowing down your store right now, marking it as a background task moves its queued work to the back of the line right away — and un-checking the setting brings a background task's waiting runs back to normal scheduling just as quickly. * A busy shop may pause background work entirely for a while — that's the setting working as intended, not a stuck task. Mechanic still guarantees slow progress: a background run that has been waiting for several hours is given the next available slot, so background work can be paused, but never stopped for good. -* A background task's backlog has a limit. Once a background task has 10,000 runs waiting, new runs for that task are cancelled on arrival (each with an error message explaining why), until the backlog drains or the task leaves the background. This keeps a paused background task from accumulating unbounded work. +* Nothing is ever dropped. A background task's runs queue up without limit, exactly like any other task's — the background setting only changes *when* they run, never *whether* they run. {% hint style="warning" %} Background tasks are not recommended for tasks that use [bulk operations](../../../platform/graphql/basics/bulk-operations.md). Shopify's bulk operation results are only available for a limited time, and a background task may wait long enough for its results to expire. From 04a61b0677119c405ff418db24524bfa90a12d25 Mon Sep 17 00:00:00 2001 From: Matt Sodomsky Date: Sun, 5 Jul 2026 14:08:48 -0500 Subject: [PATCH 3/4] Background waiting is always visible --- core/tasks/advanced-settings/background-tasks.md | 1 + 1 file changed, 1 insertion(+) diff --git a/core/tasks/advanced-settings/background-tasks.md b/core/tasks/advanced-settings/background-tasks.md index e7caf17..3afd7b2 100644 --- a/core/tasks/advanced-settings/background-tasks.md +++ b/core/tasks/advanced-settings/background-tasks.md @@ -13,6 +13,7 @@ Two advanced task settings, found in the "Runtime" tab of the advanced task edit * Changing the setting takes effect immediately, in both directions, including for runs that are already waiting in the queue. If a bulk task is slowing down your store right now, marking it as a background task moves its queued work to the back of the line right away — and un-checking the setting brings a background task's waiting runs back to normal scheduling just as quickly. * A busy shop may pause background work entirely for a while — that's the setting working as intended, not a stuck task. Mechanic still guarantees slow progress: a background run that has been waiting for several hours is given the next available slot, so background work can be paused, but never stopped for good. * Nothing is ever dropped. A background task's runs queue up without limit, exactly like any other task's — the background setting only changes *when* they run, never *whether* they run. +* Background waiting is always visible. The Mechanic home page shows how much background work is waiting and for how long, and each waiting run shows how many runs from your other tasks are ahead of it. Background work doesn't count toward your queue's lag — a deliberately-waiting backfill isn't your queue running behind. {% hint style="warning" %} Background tasks are not recommended for tasks that use [bulk operations](../../../platform/graphql/basics/bulk-operations.md). Shopify's bulk operation results are only available for a limited time, and a background task may wait long enough for its results to expire. From af81bab15736ba040ffcf0f74422ebb171962ac2 Mon Sep 17 00:00:00 2001 From: Matt Sodomsky Date: Sun, 5 Jul 2026 23:17:25 -0500 Subject: [PATCH 4/4] Accuracy fixes from adversarial review Fix the dead bulk-operations link; the valve grants periodic (not per-slot) progress; manual retries re-enter the queue with the task's current settings rather than bypassing them; large-backlog lane changes take moments, not zero time. --- core/tasks/advanced-settings/background-tasks.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/core/tasks/advanced-settings/background-tasks.md b/core/tasks/advanced-settings/background-tasks.md index 3afd7b2..43177bd 100644 --- a/core/tasks/advanced-settings/background-tasks.md +++ b/core/tasks/advanced-settings/background-tasks.md @@ -10,13 +10,13 @@ Two advanced task settings, found in the "Runtime" tab of the advanced task edit ## How background tasks behave * The background setting applies to the task's runs — both its task runs and the action runs they generate. Event processing is unaffected: events are always received and interpreted at normal speed, so your run history fills in promptly even when the resulting work waits. -* Changing the setting takes effect immediately, in both directions, including for runs that are already waiting in the queue. If a bulk task is slowing down your store right now, marking it as a background task moves its queued work to the back of the line right away — and un-checking the setting brings a background task's waiting runs back to normal scheduling just as quickly. -* A busy shop may pause background work entirely for a while — that's the setting working as intended, not a stuck task. Mechanic still guarantees slow progress: a background run that has been waiting for several hours is given the next available slot, so background work can be paused, but never stopped for good. +* Changing the setting takes effect right away, in both directions, including for runs that are already waiting in the queue (a very large backlog may take a few moments to finish moving over). If a bulk task is slowing down your store right now, marking it as a background task moves its queued work to the back of the line right away — and un-checking the setting brings a background task's waiting runs back to normal scheduling just as quickly. +* A busy shop may pause background work entirely for a while — that's the setting working as intended, not a stuck task. Mechanic still guarantees slow progress: a background run that has been waiting for several hours is periodically given a slot even while your queue is busy, so background work can be paused, but never stopped for good. * Nothing is ever dropped. A background task's runs queue up without limit, exactly like any other task's — the background setting only changes *when* they run, never *whether* they run. * Background waiting is always visible. The Mechanic home page shows how much background work is waiting and for how long, and each waiting run shows how many runs from your other tasks are ahead of it. Background work doesn't count toward your queue's lag — a deliberately-waiting backfill isn't your queue running behind. {% hint style="warning" %} -Background tasks are not recommended for tasks that use [bulk operations](../../../platform/graphql/basics/bulk-operations.md). Shopify's bulk operation results are only available for a limited time, and a background task may wait long enough for its results to expire. +Background tasks are not recommended for tasks that use [bulk operations](../../../platform/graphql/bulk-operations.md). Shopify's bulk operation results are only available for a limited time, and a background task may wait long enough for its results to expire. {% endhint %} ## Maximum concurrent runs @@ -25,4 +25,4 @@ A concurrency limit is useful when a task's work should keep moving, but shouldn Compared to the background setting, a concurrency limit is the gentler tool: a background task yields *completely* to your other work, while a concurrency-limited task keeps a steady, bounded pace regardless of what else is happening. The two settings can be combined. -Retrying a run manually always runs it immediately, regardless of these settings. +A manually retried run re-enters the queue like any other run, using its task's current settings — retrying a run that failed while its task was a background task, after the task has left the background, runs it at normal priority.