feat: Code Mode — experimental run-code + get-code-docs tools

[MQ37]

Code Mode for the Apify MCP Server

Status: Draft — pending approval before implementation.
Owner: TBD
Origin: Finalized from the design spec in PR #794; closes #788.

Summary

The existing experimental tool category in mcp.apify.com gains two opt-in tools that let the caller LLM submit a TypeScript program that orchestrates Apify resources through typed bindings. The program runs in a workerd V8 isolate hosted in a new Apify Actor (apify/code-runtime) — a normal per-run Actor (not Standby) — using the Worker Loader API. The Actor runs with limited permissions (scoped token + sandboxed isolate — see Security) and pushes the program's captured output as a single dataset item { stdout, stderr }.

Two new tools in the experimental category: run-code (submit code; waits up to waitSecs, else returns a runId) and get-code-docs (binding reference + usage examples). Output retrieval reuses the existing get-actor-run + get-dataset-items tools — no code-specific retrieval tool. The async model mirrors call-actor + get-actor-run so callers stay under MCP clients' fixed tool-call timeouts. Existing tools unchanged. Discovery (search-actors, fetch-actor-details, search-apify-docs) stays on existing MCP tools.

Problem

Some Actors return more data than fits in caller LLM context. Some tasks need programmatic processing (filter, project, chain) the caller cannot do locally. The current MCP tool set forces the caller to round-trip every intermediate result through the model.

Architecture

External MCP client
  | tools/call run-code({ code, waitSecs })        -> run result, or { runId } if still running
  | tools/call get-actor-run({ runId, waitSecs })  (existing) poll until terminal
  | tools/call get-dataset-items({ datasetId })    (existing) read { stdout, stderr }
  v
mcp.apify.com (apify-mcp-server, experimental category opt-in)
  | client.actor('apify/code-runtime').call({ code }, { waitSecs })  (existing call-actor plumbing)
  v
apify/code-runtime (normal per-run Apify Actor, limited permissions)
  +-------------------------------------------------+
  | entrypoint.sh: boot workerd (loopback),         |
  | POST /run once via curl, then exit              |
  | +---------------------------------------------+ |
  | | parent worker (POST /run)                   | |
  | |  - reads Actor INPUT { code } via REST       | |
  | |  - substitutes into inner.js template       | |
  | |  - LOADER.get(uuid, () => ({ modules,env }))| |
  | |  - pushes { stdout, stderr } to dataset      | |
  | +---------------------------------------------+ |
  | +---------------------------------------------+ |
  | | inner V8 isolate (one per run)              | |
  | |  - apify global = fetch-based REST shim     | |
  | |  - runs runUserCode(apify, console)         | |
  | |  - captures stdout and stderr separately    | |
  | |  - scoped APIFY_TOKEN visible only here     | |
  | +---------------------------------------------+ |
  +-------------------------------------------------+

MCP server changes

Both tools are added to the existing experimental tool category, opt-in only — not in toolCategoriesEnabledByDefault (src/tools/categories.ts). No new category is introduced.

Async execution model

User code can be long-running (it may start Actors and wait on them), and several MCP clients impose a fixed tool-call timeout — often 60 s, sometimes as low as 10 s. To stay under that ceiling, run-code is a thin wrapper over call-actor for apify/code-runtime and reuses the existing run lifecycle:

• run-code starts the run and waits up to waitSecs for it to finish. If it finishes in time, it returns the run result (status + defaultDatasetId + nextStep). If not, it returns the runId while the run keeps going.
• The caller polls with the existing get-actor-run (again up to waitSecs) until the run is terminal, then reads the single { stdout, stderr } item with the existing get-dataset-items.

waitSecs is 0–45, default 30 — the same bounds as call-actor (CALL_ACTOR_WAIT_SECS_DEFAULT = 30, WAIT_SECS_MAX = 45); the 45 s cap deliberately stays under the 60 s tool-call ceiling.

Tool: run-code

Field	Value
Input	{ code: string, waitSecs?: number } — waitSecs 0–45, default 30
Output (finished within waitSecs)	Run result like call-actor: structuredContent: { runId, status, defaultDatasetId, summary, nextStep }; nextStep = "Use get-dataset-items with datasetId=… to read the { stdout, stderr } output."
Output (still running after waitSecs)	isError: false; structuredContent: { runId, status: "RUNNING", nextStep }; nextStep = "Use get-actor-run with runId=… and waitSecs=… to poll."
Error	isError: true; diagnostic in text; structuredContent: { status, failureCategory, runId? }
paymentRequired	true
readOnlyHint	false
taskSupport	optional

The captured stdout and stderr are kept strictly separated — distinct keys (stdout, stderr) in the single dataset item, never concatenated.

Output retrieval — existing tools

• get-actor-run ({ runId, waitSecs }) — poll the run to terminal status; returns defaultDatasetId + a nextStep to get-dataset-items.
• get-dataset-items ({ datasetId }) — returns the single item { stdout, stderr }.

Tool: get-code-docs

Field	Value
Input	{ topic: enum }
Output	Markdown docs (~300–500 tokens): binding/function reference + usage examples, TypeScript fenced
paymentRequired	false
readOnlyHint	true

Topics: running-actors, datasets, key-value-stores.

Runtime Actor (apify/code-runtime, new repo)

Container

• Two-stage Docker build:
  • Stage 1 (node:24-bookworm-slim): install workerd@1.20260402.x (via pnpm) to extract the binary.
  • Stage 2 (debian:bookworm-slim): minimal runtime; the workerd binary + ca-certificates + curl (the loopback client the entrypoint uses to trigger the single run).
• workerd is statically linked except for libc + libm (verified via ldd); no Node.js, no Apify SDK, no apify-client.
• ENTRYPOINT: entrypoint.sh — boots workerd serve --experimental on loopback, waits for /health, triggers POST /run once via curl, then exits.
• Image: ~210 MB (workerd binary dominates; curl + ca-certificates add a few MB; no Node runtime). PoC-validated: Worker Loader works in self-hosted workerd with --experimental.

Permissions

The apify/code-runtime Actor runs with limited permissions: it is granted only a scoped Apify token sufficient for the operations the bindings expose, outbound network is restricted to *.apify.com, and the inner isolate has no filesystem and no module resolution. User code cannot escalate beyond the bindings.

Run configuration (normal Actor)

Setting	Value
usesStandbyMode	false
memoryMbytes	1024 (initial)
timeoutSecs	per run (caller may pass timeout through call-actor)

Each run is a fresh container: the platform starts the Actor, it executes one program, pushes the output, and exits. The small no-Node image keeps per-run cold start low. (The earlier Standby PoC traded warm latency for a long-lived container; per-run async is simpler and keeps each call within MCP client timeouts via run-code + get-actor-run.)

Per-run lifecycle

1. run-code → call-actor starts apify/code-runtime with input { code }.
2. The container boots; entrypoint.sh starts workerd on loopback and waits for /health.
3. entrypoint.sh sends POST /run once via curl.
4. The parent worker reads the Actor INPUT ({ code }) from the default key-value store via REST and substitutes the code into the inner.js template.
5. Parent spawns a fresh isolate: env.LOADER.get(uuid, () => ({ mainModule: 'main.js', modules: { 'main.js': moduleCode }, env: { APIFY_TOKEN } })).
6. Inner isolate runs runUserCode(apify, console), capturing stdout and stderr into separate buffers; returns { exitCode, stdout, stderr, durationMs }.
7. Parent pushes a single dataset item { stdout, stderr } to the run's default dataset via REST.
8. A non-2xx /run response fails the run (curl -f); otherwise the run SUCCEEDS and the container exits.

Run status (SUCCEEDED / FAILED / TIMED-OUT) is the polling signal for get-actor-run; the { stdout, stderr } item is the output read with get-dataset-items.

Sandbox properties

• Network: outbound restricted to *.apify.com via globalOutbound.
• Filesystem: none (workerd has no FS APIs).
• Imports: none — only apify global + standard JS built-ins.
• State: none — a fresh isolate (and a fresh container) per run.

Bindings exposed to the script

The apify global is a simplified subset of apify-client that calls the Apify REST API directly via fetch(). apify-client cannot run inside a workerd isolate at all (it requires a Node runtime workerd does not provide — PoC-validated), so the hand-rolled fetch shim is the path of least resistance, not just a size optimization. It also presents an LLM-ergonomic surface.

apify.actor.search({ query, limit?, category? })
apify.actor.getDetails({ actorId })
apify.actor.run({ actorId, input?, memoryMbytes?, timeoutSecs?, waitForFinishSecs?, maxTotalChargeUsd?, maxItems? })
apify.actor.start({ actorId, input?, ...same opts })
apify.actor.runAndGetItems({ actorId, input?, fields?, limit?, ...opts })

apify.run.get({ runId })
apify.run.wait({ runId, waitForFinishSecs? })
apify.run.abort({ runId })
apify.run.getLog({ runId, limit? })

apify.dataset.listItems({ datasetId, fields?, omit?, limit?, offset?, clean?, desc? })
apify.dataset.iterate({ datasetId, fields?, ... })
apify.dataset.getSchema({ datasetId, sample? })
apify.dataset.create({ name? })
apify.dataset.pushItems({ datasetId, items })

apify.kvs.get({ storeId, key })
apify.kvs.set({ storeId, key, value, contentType? })
apify.kvs.list({ storeId, limit?, exclusiveStartKey? })
apify.kvs.create({ name? })

23 methods. Each is a fetch() wrapper to api.apify.com. Implementation lives in apify/code-runtime.

Possible v2 additions: bindings for schedules, tasks, webhooks, request queues, builds. Discovery (search-actors, fetch-actor-details, search-apify-docs) intentionally stays on the existing MCP tools.

Binding design principles

1. Object arguments only — no positional parameters.
2. Direct returns — return what the caller wants; no wrapper objects.
3. Units in names — memoryMbytes, timeoutSecs, waitForFinishSecs.
4. One method per common task — convenience aggregations (runAndGetItems) over composing two calls.
5. Consistent shapes — same patterns across all namespaces.

Security

• The apify/code-runtime Actor runs with limited permissions — a scoped Apify token, no filesystem, no module resolution, and outbound network restricted to *.apify.com via globalOutbound.
• The bindings hit api.apify.com only and cannot reach mcp.apify.com, so a sub-agent cannot recursively invoke run-code.
• The caller's APIFY_TOKEN lives in container env and is passed only to the inner isolate; it is never exposed back to the caller.
• workerd alone is not a hardened sandbox; the Apify container is the outer boundary. Each run is its own container hosting a single discarded V8 isolate.

Follow-up

• Implementation lands in separate PRs.
• apify/code-runtime will be a separate repo.