Skip to content

(feat) Add resource bounds to the mounts archive, gitignore, and recency-descent paths #5415

Description

@mmabrouk

Several code paths added for the drive/mounts surfaces have no resource ceilings, so a large or pathological (but agent-constructable) mount can drive unbounded memory, connection, or time cost. None of these is a correctness bug; each is a missing bound.

Note: streaming very large individual files is explicitly out of scope for "download all" (product decision, 2026-07-20). This issue is about bounding breadth and duplication, not redesigning per-file reads into a streaming API.

The goal is to put ceilings on the new work so a big drive degrades predictably instead of exhausting the worker.

  • Archive endpoint: cap and deduplicate the request's mounts list (each entry is a DB fetch plus a full recursive listing; today the list is unbounded and duplicable).
  • .gitignore handling: add a per-file size ceiling (~1 MB) before parsing, truncate a level's batch to the remaining _MAX_GITIGNORE_FILES budget instead of only checking the cap between batches, and put the batch reads under the existing _LIST_CONCURRENCY semaphore.
  • Promote the concurrency knobs _LIST_CONCURRENCY and _ARCHIVE_READ_CONCURRENCY to env.py, matching the repo's env-config convention (precedent: AGENTA_RUNNER_CONCURRENCY_LIMIT). The two algorithmic backstops can stay module constants.
  • Add an overall time/count budget to the non-count-only recency descent — the one genuinely unbounded case is a huge tree with no .gitignore, which the recency path fully enumerates.
  • Consider moving the synchronous per-file regex matching / rollup off the event loop (to_thread or memoized per-directory verdicts) so big trees don't stall the worker.

Follow-up from the #5400 backend review. Cited findings: 3.4 (the archive concurrency knob bounds file count, not bytes; the gitignore cap is soft; promote knobs to env.py), 4.5 (unbounded/duplicable archive mounts list), 4.6 (unbounded .gitignore size, unbounded per-batch concurrency, soft cap), 4.7 (synchronous regex/rollup on the event loop). The review report is not in the repo, so this summary stands alone.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions