From 09e469ca036e05ec392819b17ccb082879d0b42f Mon Sep 17 00:00:00 2001 From: "H. Furkan Bozkurt" Date: Sun, 12 Jul 2026 05:02:55 +0000 Subject: [PATCH 1/8] feat(bench): reshape tasks to api-first framework-focus grading --- tasks/async-word-counter/PROMPT.md | 49 +-- tasks/async-word-counter/test.spec.ts | 344 ++++++------------- tasks/auth-notes/PROMPT.md | 32 +- tasks/auth-notes/test.spec.ts | 242 +++++++------ tasks/cognito-profile/PROMPT.md | 29 +- tasks/cognito-profile/test.spec.ts | 275 +++++---------- tasks/collab-presence-board/PROMPT.md | 30 +- tasks/collab-presence-board/test.spec.ts | 279 +++++---------- tasks/file-gallery/PROMPT.md | 29 +- tasks/file-gallery/test.spec.ts | 411 ++++++++--------------- tasks/kb-chat-agent/PROMPT.md | 36 +- tasks/kb-chat-agent/test.spec.ts | 267 +++++---------- tasks/oidc-dsql-notes/PROMPT.md | 41 ++- tasks/oidc-dsql-notes/test.spec.ts | 297 +++++++--------- tasks/sql-kb-catalog/PROMPT.md | 40 ++- tasks/sql-kb-catalog/test.spec.ts | 151 ++++++--- 16 files changed, 1020 insertions(+), 1532 deletions(-) diff --git a/tasks/async-word-counter/PROMPT.md b/tasks/async-word-counter/PROMPT.md index 4b0a5d3dc..c6c2d92c2 100644 --- a/tasks/async-word-counter/PROMPT.md +++ b/tasks/async-word-counter/PROMPT.md @@ -1,41 +1,50 @@ # Task: Async Word Counter -Build an async word counter in this AWS Blocks app. A user submits some text; the counting happens in a background job. The row for that submission starts as "processing" and flips to "done" with the word count once the job finishes. Results survive a page reload. +Build an async word counter in this AWS Blocks app. A user submits some text; the counting happens in a **background job**. A submission starts as `"processing"` and flips to `"done"` with its word count once the job finishes. Results are persisted and survive a reload. + +The core of this task is **using the framework**: submissions are enqueued and read through an **`api` namespace** whose methods run on the server, do the counting in a **background job block**, and persist each job in a **key/value block** keyed by job id. The page is a thin client that polls that API. ## Setup (do this first) -The workspace has already been scaffolded. Begin by reading README.md, then do all your edits in this workspace. +The workspace has already been scaffolded. Begin by reading README.md (and any AGENTS.md), then do all your edits in this workspace. ## Requirements -1. A user types text into an input and clicks submit. -2. Submitting enqueues a background job (do the counting in the job — not inline in the request handler) and immediately adds a row for it whose `data-status` is `"processing"`. -3. The background job counts the words and stores the result in a key/value block keyed by the job id. **Count by whitespace runs only:** trim the text, then split on any run of whitespace (spaces, tabs) — so `one two three four five` is 5, and `" a b "` (leading/trailing + double spaces) is 2. A naive `split(' ')` that counts empty gaps is wrong. - - **Punctuation is part of a word, not a separator:** only whitespace separates words, so `hello,world foo.bar-baz!` is **3** words, not 5. - - **Unicode and emoji tokens each count as one word:** `café 日本語 🙂 naïve` is **4** words. Count each maximal run of non-whitespace as one word — do **not** use `\w+` / `\W+`, which miscount punctuation and non-ASCII/emoji characters. -4. **Polling:** the frontend polls for the result; when it's ready the row's `data-status` becomes `"done"` and the row shows the word count. -5. **Persistence (including in-flight jobs):** persist each submission to the key/value block **when you enqueue it** (status `"processing"`, keyed by job id) — not only when it finishes. On load, **restore the whole list from the store** (every job, including still-`processing` ones) and resume polling — not just the most recent submission. A row reloaded **while still processing** must reappear and still resolve to `"done"` with its count. (An app that tracks the job list only in client memory loses an in-flight row on reload — that fails.) -6. **Multiple jobs, keyed by job id:** submitting several times enqueues several independent jobs; each gets its own row and its own correct count. Results must be keyed by **job id**, never by the input text — so submitting the **same text twice** produces **two** independent rows, each with its own result that doesn't bleed into the other. -7. **Input validation:** disable `[data-testid=wc-submit]` whenever the input is empty or whitespace-only (trim before the check); re-enable it once real text is present. Don't enqueue a job for blank input. +### The `api` namespace (primary surface) + +Expose an **`api` namespace** (a `POST /aws-blocks/api` JSON-RPC endpoint) with these methods: + +1. **`api.enqueue(text)`** — enqueues a background word-count job for `text` and returns the new job's id as `{ "id": }` (a non-empty job id). It must: + - do the counting in a **background job** (the AsyncJob block) — **not** inline in the request handler; + - **persist the job at enqueue time** with status `"processing"`, keyed by job id, in the key/value block (not only when it finishes); + - **validate the input** — `text` that is empty or whitespace-only (trim first) is rejected with a JSON-RPC **error** envelope and enqueues **no** job. +2. **`api.getJob(id)`** — returns the job as exactly `{ "id": , "text": , "status": "processing" | "done", "count": }`. `count` is the word count once `status === "done"` (and may be `null`/absent while processing). An **unknown** id returns a JSON-RPC **error** envelope (not a bogus success). +3. **`api.listJobs()`** — returns an array of every job (each the same `{ id, text, status, count }` shape), **restored from the store** — including still-`processing` ones. This is what lets the page rebuild its list on load. + +**Word count — count by whitespace runs only.** Trim the text, then split on any run of whitespace (spaces, tabs, newlines): `one two three four five` is `5`, and `" a b "` is `2` (a naive `split(' ')` that counts empty gaps is wrong). **Punctuation is part of a word, not a separator** — `hello,world foo.bar-baz!` is `3`. **Unicode and emoji tokens each count as one word** — `café 日本語 🙂 naïve` is `4`. Count each maximal run of non-whitespace as one word; do **not** use `\w+` / `\W+` (they miscount punctuation and non-ASCII/emoji). + +**Keyed by job id, never by text.** Submitting the **same text twice** produces **two** independent jobs, each with its own id and its own result — results must never bleed between jobs or collapse into one. -A single shared list — no login. +### The page (thin client / light smoke) + +1. A text input (`[data-testid=wc-input]`) and a submit button (`[data-testid=wc-submit]`). Disable submit whenever the input is empty or whitespace-only (trim before checking); re-enable it once real text is present. +2. Submitting calls `api.enqueue`, then the page polls (`api.getJob` / `api.listJobs`) and renders a row per job inside `[data-testid=wc-list]`. +3. On load, the list is rebuilt from `api.listJobs` (every job, including still-processing ones) and polling resumes — so a row reloaded while still processing reappears and still resolves to `"done"`. ## Selector contract -The Playwright test grades your work using `data-testid` hooks and one data attribute. Implement them exactly. +The page smoke test uses these hooks. Implement them exactly. | Selector | Element | Purpose | |---|---|---| | `[data-testid=wc-input]` | `` (or `