feat: implement queue clean up support#5
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a comprehensive queue cleanup feature for the standalone app, enabling users to efficiently remove old terminal tasks or events from queues through a new UI modal dialog and supporting backend infrastructure.
Key Changes:
- Adds a new "Clean Up" button and modal dialog to the queues page with options to clean tasks and/or events by age
- Implements backend cleanup operations with batched SQL deletions for improved performance on large datasets
- Adds comprehensive benchmarks and tests to validate cleanup performance and correctness
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
standalone/src/routes/queues/+page.svelte |
Adds cleanup modal UI with target selection (tasks/events), age filters (7d/30d/all), and validation |
standalone/src/lib/providers/absurdData.ts |
Defines cleanup types and integrates cleanupQueue method across all providers (Tauri, tRPC, mock) |
standalone/src-tauri/src/worker.rs |
Implements exponential backoff for worker restarts with crash tracking and improved error messaging |
standalone/src-tauri/src/lib.rs |
Registers new cleanup_queue Tauri command |
standalone/src-tauri/src/dev_api.rs |
Adds cleanupQueue tRPC procedure with input type mapping |
standalone/src-tauri/src/db_commands.rs |
Implements cleanup_queue command with batched deletion loop and SQL function wrappers |
docs/specs/queue-cleanup-ui.md |
Documents UI design, backend API, and implementation approach for the cleanup feature |
absurd-sqlite-extension/src/run.rs |
Refactors cleanup functions to use single batched DELETE statements with rowid IN subqueries |
absurd-sqlite-extension/src/lib.rs |
Adds large-batch cleanup test (12,000 tasks and events) and imports params! macro |
absurd-sqlite-extension/benches/cleanup.rs |
Adds Criterion benchmarks for cleanup operations across various batch sizes |
absurd-sqlite-extension/Cargo.toml |
Adds rlib crate type and Criterion benchmark configuration |
Cargo.lock |
Adds Criterion and its dependencies for benchmarking support |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "7d")} | ||
| disabled={!cleanupTargets.tasks} | ||
| > | ||
| > 7d |
There was a problem hiding this comment.
The HTML entity ">" is displayed literally as text, showing "> 7d" to the user. Instead of using the HTML entity in the button text, use the actual greater-than character ">" or write it as text like "Older than 7d" for better clarity.
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "30d")} | ||
| disabled={!cleanupTargets.tasks} | ||
| > | ||
| > 30d |
There was a problem hiding this comment.
The HTML entity ">" is displayed literally as text, showing "> 30d" to the user. Instead of using the HTML entity in the button text, use the actual greater-than character ">" or write it as text like "Older than 30d" for better clarity.
| > 7d | ||
| </button> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupEventAge === "30d" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.events ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.events && (cleanupEventAge = "30d")} | ||
| disabled={!cleanupTargets.events} | ||
| > | ||
| > 30d | ||
| </button> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupEventAge === "all" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.events ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.events && (cleanupEventAge = "all")} | ||
| disabled={!cleanupTargets.events} | ||
| > | ||
| All |
There was a problem hiding this comment.
The HTML entity ">" is displayed literally as text for all event age buttons. Instead of using the HTML entity in the button text, use the actual greater-than character ">" or write it as text like "Older than 7d/30d" for better clarity.
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.tasks ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "7d")} |
There was a problem hiding this comment.
The onclick handler uses a conditional expression that could fail to execute when the checkbox is checked quickly. The expression "cleanupTargets.tasks && (cleanupTaskAge = "7d")" will not execute the assignment if cleanupTargets.tasks is false, but this button is already disabled when tasks is false, making the conditional check redundant. Consider simplifying to just "cleanupTaskAge = '7d'" since the disabled attribute already prevents clicks when the checkbox is unchecked.
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.tasks ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "30d")} |
There was a problem hiding this comment.
The onclick handler uses a conditional expression that could fail to execute when the checkbox is checked quickly. The expression "cleanupTargets.tasks && (cleanupTaskAge = "30d")" will not execute the assignment if cleanupTargets.tasks is false, but this button is already disabled when tasks is false, making the conditional check redundant. Consider simplifying to just "cleanupTaskAge = '30d'" since the disabled attribute already prevents clicks when the checkbox is unchecked.
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.tasks ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "all")} |
There was a problem hiding this comment.
The onclick handler uses a conditional expression that could fail to execute when the checkbox is checked quickly. The expression "cleanupTargets.tasks && (cleanupTaskAge = "all")" will not execute the assignment if cleanupTargets.tasks is false, but this button is already disabled when tasks is false, making the conditional check redundant. Consider simplifying to just "cleanupTaskAge = 'all'" since the disabled attribute already prevents clicks when the checkbox is unchecked.
| {#if isCleanupOpen} | ||
| <div | ||
| class="fixed inset-0 z-10 flex items-center justify-center bg-slate-900/30 p-4" | ||
| role="dialog" | ||
| aria-modal="true" | ||
| aria-label="Clean up queue" | ||
| > | ||
| <form class="w-full max-w-3xl rounded-xl bg-white p-6 shadow-xl" onsubmit={handleCleanup}> | ||
| <h2 class="text-lg font-semibold text-slate-900"> | ||
| Clean up {cleanupQueue?.name} | ||
| </h2> | ||
| <p class="mt-1 text-sm text-slate-500"> | ||
| Remove old terminal tasks or events from this queue. | ||
| </p> | ||
|
|
||
| <div class="mt-5 space-y-4"> | ||
| <div class="rounded-lg border border-black/10 bg-white p-4"> | ||
| <div class="space-y-4"> | ||
| <label class="flex items-center gap-2 text-sm font-medium text-slate-700 cursor-pointer"> | ||
| <input type="checkbox" bind:checked={cleanupTargets.tasks} class="cursor-pointer" /> | ||
| Tasks | ||
| </label> | ||
| <div class="flex flex-col gap-2 text-sm font-medium text-slate-700"> | ||
| Age | ||
| <div class="flex flex-wrap gap-2"> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupTaskAge === "7d" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.tasks ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "7d")} | ||
| disabled={!cleanupTargets.tasks} | ||
| > | ||
| > 7d | ||
| </button> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupTaskAge === "30d" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.tasks ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "30d")} | ||
| disabled={!cleanupTargets.tasks} | ||
| > | ||
| > 30d | ||
| </button> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupTaskAge === "all" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.tasks ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.tasks && (cleanupTaskAge = "all")} | ||
| disabled={!cleanupTargets.tasks} | ||
| > | ||
| All | ||
| </button> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| <p class="mt-3 text-xs text-slate-500"> | ||
| Pending, running, and sleeping tasks will be preserved. | ||
| </p> | ||
| </div> | ||
|
|
||
| <div class="rounded-lg border border-black/10 bg-white p-4"> | ||
| <div class="space-y-4"> | ||
| <label class="flex items-center gap-2 text-sm font-medium text-slate-700 cursor-pointer"> | ||
| <input type="checkbox" bind:checked={cleanupTargets.events} class="cursor-pointer" /> | ||
| Events | ||
| </label> | ||
| <div class="flex flex-col gap-2 text-sm font-medium text-slate-700"> | ||
| Age | ||
| <div class="flex flex-wrap gap-2"> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupEventAge === "7d" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.events ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.events && (cleanupEventAge = "7d")} | ||
| disabled={!cleanupTargets.events} | ||
| > | ||
| > 7d | ||
| </button> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupEventAge === "30d" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.events ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.events && (cleanupEventAge = "30d")} | ||
| disabled={!cleanupTargets.events} | ||
| > | ||
| > 30d | ||
| </button> | ||
| <button | ||
| type="button" | ||
| class={`rounded-md border px-3 py-1.5 text-sm cursor-pointer disabled:cursor-not-allowed ${ | ||
| cleanupEventAge === "all" | ||
| ? "border-slate-900 bg-slate-900 text-white" | ||
| : "border-black/10 bg-white text-slate-700" | ||
| } ${!cleanupTargets.events ? "opacity-50" : ""}`} | ||
| onclick={() => cleanupTargets.events && (cleanupEventAge = "all")} | ||
| disabled={!cleanupTargets.events} | ||
| > | ||
| All | ||
| </button> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| </div> | ||
|
|
||
| {#if cleanupError} | ||
| <p class="mt-3 text-sm text-rose-600">{cleanupError}</p> | ||
| {/if} | ||
|
|
||
| <div class="mt-6 flex items-center justify-end gap-3"> | ||
| <Button | ||
| type="button" | ||
| class="rounded-md border border-black/10 bg-white px-4 py-2 text-sm text-slate-700" | ||
| onclick={closeCleanup} | ||
| disabled={isCleaning} | ||
| > | ||
| Cancel | ||
| </Button> | ||
| <Button | ||
| type="submit" | ||
| class="rounded-md bg-slate-900 px-4 py-2 text-sm font-semibold text-white shadow-sm hover:bg-slate-800 disabled:cursor-not-allowed disabled:opacity-60" | ||
| disabled={isCleaning} | ||
| > | ||
| {isCleaning ? "Cleaning..." : "Clean up"} | ||
| </Button> | ||
| </div> | ||
| </form> | ||
| </div> | ||
| {/if} |
There was a problem hiding this comment.
The modal implementation is missing keyboard escape handling. Users should be able to press the Escape key to close the modal, which is a standard accessibility requirement for modal dialogs. Add an onkeydown handler to close the modal when Escape is pressed and isCleaning is false.
|
|
||
| ### Modal UI | ||
| - Title: `Clean up {queue.name}` | ||
| - Target selector: `Tasks` | `Events` (default: `Tasks`). |
There was a problem hiding this comment.
The documentation states "Target selector: Tasks | Events (default: Tasks)" but the actual implementation allows both targets to be selected simultaneously with independent checkboxes. The documentation should clarify that users can select one or both targets, not just a single exclusive selection.
| - Target selector: `Tasks` | `Events` (default: `Tasks`). | |
| - Target selector: independent checkboxes for `Tasks` and `Events` (both can be selected; default: `Tasks` checked, `Events` unchecked). |
| const BATCH_LIMIT: i64 = 1_000; | ||
| const BENCH_SIZES: [usize; 5] = [100, 500, 1_000, 5_000, 10_000]; |
There was a problem hiding this comment.
The benchmark sizes include 10,000 items but the cleanup limit is set to 1,000. This could lead to a very small number of iterations (10) which may not provide statistically significant benchmark results. Consider adding larger benchmark sizes like 50,000 or 100,000 to better test the batching behavior under more realistic high-volume scenarios.
| cleanupQueue = null; | ||
| } catch (error) { | ||
| console.error("Failed to clean up queue", error); | ||
| cleanupError = "Failed to clean up queue."; |
There was a problem hiding this comment.
The error message "Failed to clean up queue." is generic and doesn't provide users with actionable information about what went wrong. The actual error is only logged to the console. Consider including more specific error information in the displayed message, such as "Failed to clean up queue: [error details]" or at least different messages for different failure scenarios (e.g., network issues vs. invalid queue name).
| cleanupError = "Failed to clean up queue."; | |
| const errorMessage = | |
| error instanceof Error | |
| ? error.message | |
| : typeof error === "string" | |
| ? error | |
| : "Unknown error"; | |
| cleanupError = `Failed to clean up queue: ${errorMessage}`; |
This pull request introduces a queue cleanup feature for the standalone app, enabling users to efficiently remove old terminal tasks or events from a queue. It adds a backend API and command to support this functionality, optimizes the cleanup implementation for better performance, and includes new benchmarks and tests to ensure correctness and scalability.
Backend/API and Integration:
cleanup_queueTauri command and API endpoint, including theCleanupQueueOptionsandCleanupResulttypes, allowing the frontend to trigger cleanup operations for tasks or events in a queue, with age-based filtering and batching. [1] [2] [3] [4] [5] [6] [7]queue-cleanup-ui.md).Performance and Implementation Improvements:
absurd_cleanup_tasksandabsurd_cleanup_eventsto perform batched deletions using a single SQL statement per batch, significantly improving performance for large queues. [1] [2] [3]Testing and Benchmarking:
benches/cleanup.rs) using Criterion to measure cleanup performance across different batch sizes and data volumes. [1] [2]Build and Dependency Updates:
rlibcrate type and adds dependencies for benchmarking. [1] [2]Minor Improvements:
params!macro in tests.