Deep-research agent: chat-triggered long-running corpus research with citations

[JSv4]

Summary

Adds a "deep research" agent: while chatting with a corpus, a user can say "please research X, Y, and Z" and a long-lived (5–30 min) autonomous job kicks off in the background. The job runs a PydanticAI corpus agent with a strictly read-only retrieval surface plus two job-bound scratchpad tools, produces a markdown report with footnote-style citations, and notifies the user when complete.

The user comes back later, opens the notification (or the chat — a system message lands there too), and reads the report.

Architecture

New app opencontractserver/research/:

• ResearchReport model — sibling of Analysis/Extract. Tracks corpus FK, prompt, status (JobStatus + new CANCELLED), start/complete/last-progress timestamps, cancel_requested flag, step budget, rendered content markdown, structured findings/citations/tool_call_log/model_usage/warnings JSON sidecars, M2Ms to cited annotations & touched documents, and FKs back to the originating conversation/message.
• ResearchReportService — canonical entry point per CLAUDE.md rule 7. Lifecycle transitions, scratchpad writes, terminal finalize (runs the citation post-processor that converts <cite ids="…">claim</cite> placeholders into [^n] markers + a ## Sources block), concurrency soft-guard (refuses a second QUEUED/RUNNING report on the same corpus within an hour).

Closed-citation-graph invariant — the heart of the no-hallucinated-citations guarantee. record_finding validates every supporting_source_id against PydanticAIDependencies.retrieved_annotation_ids (the existing accumulator that retrieval tools populate). Annotation IDs not surfaced by a retrieval tool in this run are rejected with an error string the model self-corrects on.

Celery loop (opencontractserver/tasks/research_tasks.py) mirrors run_agent_corpus_action. Builds the agent via agents.for_corpus(...), runs with UsageLimits(request_limit=max_steps, request_tokens_limit=…), has a salvage path that composes findings into markdown if the budget exhausts without an explicit finalize_report. Notifications + an inserted system ChatMessage land on every terminal state.

Chat kickoff — start_deep_research tool auto-attached to the corpus agent for authenticated users. corpus_id / user_id / conversation_id are auto-injected from PydanticAIDependencies (not LLM-controlled), closing the prompt-injection vector. The tool calls ResearchReportService.start, enqueues the Celery task on commit, and returns a short confirmation so the chat turn ends gracefully.

Plumbing changes (small, additive — both already worked for the document agent path):

• restrict_tool_names plumbed through UnifiedAgentFactory.create_corpus_agent + PydanticAICorpusAgent.create.
• PydanticAIDependencies.conversation_id (new optional field) + build_inject_params_for_context now threads conversation_id for tools that declare it.

GraphQL: ResearchReportType, researchReport(id) / researchReports(corpusId?, status?) queries, startResearchReport(corpusId, prompt, title?, maxSteps?) / cancelResearchReport(id) mutations. All read paths go through BaseService.get_or_none / BaseService.filter_visible (clean against the opencontracts.E001 invariant).

v1 scope decisions (confirmed before implementation)

• Creator-only visibility — no sharing, no is_public semantics. Sidesteps the IDOR-on-shared-report problem entirely.
• Corpus-wide — no document-subset selector. The agent crawls everything in the named corpus.
• Cooperative cancel — cancel_requested flag polled between tool calls; partial findings are preserved.

Test plan

• docker compose -f test.yml run django pytest opencontractserver/tests/research/ -n 4 --dist loadscope (model + service + kickoff-tool tests)
• docker compose -f test.yml run django pytest opencontractserver/tests/architecture/ -n 4 --dist loadscope (confirm the service-layer invariant test still passes — none of the new graphql code uses forbidden Tier-0 identifiers)
• docker compose -f test.yml run django python manage.py migrate --plan (verify the two new migrations apply cleanly)
• Manual end-to-end against a real corpus:
  1. Open a corpus chat as an authenticated user
  2. Ask the assistant: "Please do a deep research investigation on the indemnification clauses in this corpus." — confirm the LLM invokes start_deep_research and returns a confirmation string
  3. Watch the ResearchReport row transition QUEUED → RUNNING → COMPLETED
  4. Verify report.content has footnote markers + a ## Sources block, and that report.citations matches
  5. Verify a RESEARCH_REPORT_COMPLETE notification fires and an inserted system ChatMessage shows up in the originating conversation
  6. Try cancelResearchReport mid-loop — confirm CANCELLED status + preserved partial findings
  7. Try startResearchReport twice in a row on the same corpus — second call should return a friendly "already in progress" error

Out of scope for v1 (tracked for v2)

Sharing / is_public on reports; viewer-aware citation redaction for shared reports; document-subset scoping; resume-from-partial-findings; watchdog cleanup task for crashed workers (the last_progress_at column is in place for when it's added); per-user notification preferences; AgentConfiguration integration for saved research presets; frontend ResearchReportView.

See the CHANGELOG entry for the full rationale and field-by-field documentation.

---

🤖 Generated with Claude Code

---

Generated by Claude Code

[claude]

Code Review

This is a substantial and well-architected feature — the PR description is thorough, the service-layer discipline is sound, and the closed-citation-graph invariant is a clever design. Below are findings organised by severity.

---

Must-Fix (Blocking)

1. ask_document doesn't populate retrieved_annotation_ids — citation invariant is broken for sub-agent sources

DEEP_RESEARCH_READ_ONLY_TOOLS includes ask_document, and ask_document_tool returns DocAnswer.sources with real annotation IDs. However, it never appends those IDs to ctx.deps.retrieved_annotation_ids. This means any annotation the LLM sees via ask_document is silently rejected when it tries to cite it via record_finding (the closure at task line ~2038 validates against retrieved_annotation_ids). Either wire the sub-agent's captured annotation IDs back into the accumulator, or remove ask_document from DEEP_RESEARCH_READ_ONLY_TOOLS until it's wired correctly.

2. Missing migration for Analysis.status after CANCELLED added to JobStatus

Adding CANCELLED to opencontractserver/types/enums.py changes the choices list for Analysis.status in opencontractserver/analyzer/models.py (the choices are derived dynamically from JobStatus). Django will detect this as an AlterField and require a migration. No such migration is in this PR. CI will likely flag unapplied migrations.

3. SoftTimeLimitExceeded handled as FAILED instead of CANCELLED

The Celery soft_time_limit (30 min) raises celery.exceptions.SoftTimeLimitExceeded inside asyncio.run(...). It propagates to the generic except Exception as exc: handler, which calls mark_failed() and sends a RESEARCH_REPORT_FAILED notification — discarding partial findings with a misleading error. The correct path is to catch SoftTimeLimitExceeded explicitly, call mark_cancelled(), preserve partial findings, and send RESEARCH_REPORT_CANCELLED.

4. AnnotatePermissionsForReadMixin used without guardian permission model classes

ResearchReportType inherits AnnotatePermissionsForReadMixin, which calls getattr(self, "{model_name}userobjectpermission_set"). ResearchReport has no ResearchReportUserObjectPermission / ResearchReportGroupObjectPermission model classes and no corresponding migration (compare every other guarded model in the codebase). The mixin wraps the failure in a broad except Exception, so it silently returns myPermissions: [] for the creator — they see no permissions on their own reports. Either add the guardian permission model classes + migration, or remove AnnotatePermissionsForReadMixin and implement a simple creator-only resolve_my_permissions.

5. from_global_id() / int() not wrapped in try/except in GraphQL layer

research_queries.py line ~22: django_pk = int(from_global_id(kwargs["id"])[1]) — a malformed base64 input or non-numeric decoded value raises ValueError / UnicodeDecodeError, surfacing as a 500. The same pattern appears in research_mutations.py for both cancel and start mutations. Compare search_queries.py which properly wraps in try/except (ValueError, TypeError, UnicodeDecodeError).

---

Should-Fix (Non-Blocking but Important)

6. Race condition in concurrency guard

In ResearchReportService.start(), the .exists() check and .create() are not in the same transaction.atomic() + SELECT ... FOR UPDATE block. Two simultaneous requests from the same user on the same corpus can both pass the guard before either creates the row. Either add a unique DB constraint on (creator, corpus, status__in=[QUEUED, RUNNING]) or wrap both operations in a single atomic block with select_for_update().

7. No ceiling on user-supplied max_steps

ResearchReportService.start() accepts the user's max_steps without a maximum cap. A caller could pass max_steps=100000 via the GraphQL mutation and burn an enormous LLM budget. Add a hard ceiling (e.g., min(user_value, settings.DEEP_RESEARCH_MAX_STEPS_CEILING)).

8. Double-write in salvage path

In research_tasks.py, when the budget exhausts: ResearchReportService.finalize() is called (sets status = COMPLETED, saves), and then ResearchReportService.mark_completed() is called again immediately (sets status = COMPLETED again with a new completed_at, wasting a DB round-trip). Since finalize() already handles status, remove the subsequent mark_completed() call.

9. Missing @login_required on resolve_research_report

The list resolver resolve_research_reports has @login_required, but the single-object resolver resolve_research_report does not. While BaseService.get_or_none() with visible_to_user(None) will return None for anonymous users (no data leak), the inconsistency is worth closing. Add @login_required.

10. Unlimited prompt length

Neither the GraphQL mutation nor ResearchReportService.start() validates prompt length. A caller can submit an arbitrarily large prompt, causing excessive token use or DB bloat. Add a reasonable max (e.g., 10,000 chars).

11. Status filter not validated in resolve_research_reports

research_queries.py line ~43: qs = qs.filter(status=status) accepts an arbitrary string from the query variable. Filtering by a non-existent status silently returns an empty queryset with no error. Either validate against JobStatus values or use status__in.

---

Test Coverage Gaps

• No tests for the run_deep_research Celery task itself (only the kickoff tool is tested)
• No tests for the salvage path (budget exhausted before finalize_report)
• No test for SoftTimeLimitExceeded handling
• No GraphQL layer tests (startResearchReport, cancelResearchReport, researchReports, researchReport node)
• No test for myPermissions returning correct values (would catch finding Bump responses from 0.21.0 to 0.22.0 #4)
• No test confirming ask_document citation accumulation (would catch finding Bump postgres from 14.5 to 15.0 in /compose/production/postgres #1)

---

Minor / Cleanup

• Dead setting: DEEP_RESEARCH_STUCK_THRESHOLD_SECONDS is defined in settings/base.py but never referenced anywhere. Remove it or add a # reserved for v2 watchdog comment.
• Misleading restrict union: research_tasks.py adds SCRATCHPAD_TOOL_NAMES to restrict_tool_names, but the agent explicitly states caller-supplied closure tools bypass restrict_tool_names. Including them in restrict is harmless but misleading — add a comment or remove.
• Magic number 60: models.py uses getattr(settings, "DEEP_RESEARCH_DEFAULT_MAX_STEPS", 60) — the fallback 60 should reference a named constant from constants.py per CLAUDE.md rule 4.
• re imported inside function body: _render_citations imports re inside the function — move to module level.
• CREATED status is unreachable: Reports are created at QUEUED, skipping CREATED. The dead enum choice adds noise; document this intentional divergence from the Analyzer pattern.

---

What's Working Well

The design of the closed-citation-graph invariant (enforced at both record_finding and finalize) is well-thought-out. corpus_id/user_id/conversation_id are all injected from PydanticAIDependencies rather than LLM-controlled — this correctly closes the prompt-injection vector. The service layer is clean and consistent with the project's architecture. Prompt content is fenced with fence_user_content(). The cooperative cancel design (polled cancel_requested flag between tool calls) is appropriate for this use case.

[claude]

Code Review: Deep-Research Agent (#1814)

Overall: This is a substantial, well-thought-out feature. The architecture is sound — the service layer, closed-citation-graph invariant, and TOCTOU-safe concurrency guard are all well-executed. The issues below are mostly medium/low severity with one real bug, one CLAUDE.md policy violation, and a few design inconsistencies.

---

CLAUDE.md Policy Violation

PR body credits Claude Code — CLAUDE.md rule: "Never credit Claude or Claude Code in commit messages, PR messages, comments, or any other artifacts." The PR description contains both:

🤖 Generated with Claude Code
_Generated by Claude Code (session_01Ns...)_

These should be removed before merging.

---

Bug: Broken Chat-Message Link

opencontractserver/tasks/research_tasks.py — _insert_completion_chat_message:

body = (
    f"Deep research **{status_label}**: *{report.title}*.\n\n"
    f"[Open report](/research/{report.slug})"
)

The PR explicitly lists "frontend ResearchReportView" as out of scope for v1. Every completion message will drop a broken link into the originating chat. Either omit the link for now, or substitute the corpus URL / a placeholder that users can act on.

---

DRY Violation: Duplicated _decode_global_pk

The helper is copy-pasted verbatim into both config/graphql/research_mutations.py and config/graphql/research_queries.py. Per CLAUDE.md rule 2, extract it to a shared location — the natural home is config/graphql/utils.py or the closest existing utility file in config/graphql/.

---

Permission Inconsistency: start_deep_research_tool

opencontractserver/llms/tools/research_tools.py declares requires_write_permission=True, but ResearchReportService.start() only checks PermissionTypes.READ. A user with READ-but-not-WRITE hits the approval gate, then is allowed through by the service anyway. If READ is sufficient to kick off research, change the CoreTool declaration accordingly; if WRITE is genuinely required, align the service check.

---

Data Inconsistency Window in finalize()

opencontractserver/research/services/research_reports.py:

report.save(update_fields=update_fields)   # status = COMPLETED here

if citations:
    report.source_annotations.set(...)    # outside the save — crash window
    report.source_documents.set(...)

A worker killed between the save() and the M2M writes leaves the report COMPLETED with empty provenance M2Ms. Wrap the whole block in with transaction.atomic(): to close this.

---

Missing Tests: Celery Task Paths

run_deep_research has no direct tests. Uncovered paths include:

• SoftTimeLimitExceeded → CANCELLED + notification
• Generic exception → FAILED + notification
• Salvage path (_compose_salvage_body + second finalize when budget exhausts)
• Notification + chat message insertion on every terminal state

These are exactly the paths that fail silently in production. A minimal task-layer test with mocked agents.for_corpus would catch regressions.

---

Dead Code: RESEARCH_REPORT_PROGRESS Notification Type

Defined in notifications/models.py and migrated in 0005_…, but ResearchReportService.mark_progress() never emits it. Either remove the type (and simplify the migration) or emit it with appropriate throttling. If it's reserved for a future polling endpoint, a short comment to that effect prevents the next reader from treating it as accidental dead code.

---

Minor: status Filter Should Be a GraphQL Enum

config/graphql/research_queries.py declares status=graphene.String(required=False), forcing clients to guess valid values. Using graphene.Enum.from_enum(JobStatus) surfaces valid values in schema introspection and eliminates the manual validation block below it. Other query files in this codebase use GraphQL enums for status fields.

---

Minor: deps_ref Relies on an Internal Attribute

deps_ref["deps"] = getattr(agent, "agent_deps", None)

When agent_deps is None (e.g. after a PydanticAI wrapper rename), record_finding silently rejects every source ID and the agent records no findings. It fails safely but invisibly. An explicit assertion or raised exception right after the getattr would surface the misconfiguration immediately.

---

Nit: _SOFT_TIME_LIMIT / _HARD_TIME_LIMIT Evaluated at Import Time

Celery task decoration happens at module import, so @override_settings(DEEP_RESEARCH_SOFT_TIME_LIMIT=…) in tests will not change the task's registered limits. Standard Celery behaviour, but worth a comment so future test authors don't chase a confusing no-op.

---

Positive Highlights

• Closed-citation-graph invariant — the two-layer check (tool-time rejection in record_finding + finalize-time intersection) is exactly right.
• SELECT FOR UPDATE concurrency guard in ResearchReportService.start() correctly closes the TOCTOU window on the (creator, corpus) pair.
• Salvage path gracefully recovers partial findings when the budget exhausts without an explicit finalize_report call.
• prompt_sanitization fencing in build_deep_research_system_prompt correctly treats corpus metadata and the user's task as untrusted content.
• Creator-only visibility sidesteps the IDOR-on-shared-report problem cleanly; the manager shape is designed for easy widening in v2.
• Service-layer test coverage (lifecycle transitions, concurrency guard, citation post-processing, deleted-annotation edge cases) is excellent.

[claude]

Code Review — PR #1814: Deep-research agent

This is a well-architected feature with thoughtful security design (closed-citation-graph invariant, prompt injection fencing, IDOR-safe mutations). The following items range from genuine bugs to minor style concerns.

---

Bugs / Correctness

similarity_score documented but never populated (research/services/research_reports.py)
The citations JSONField help text and the PR description both reference similarity_score as a citation table field, but _render_citations never sets it. Every citation entry emitted will be missing this key. If the frontend or any consumer tries to read it, they'll get a KeyError / undefined. Either populate it (by fetching from the retrieval result metadata) or drop it from the documented schema.

Time-limit constants evaluated at import time (tasks/research_tasks.py:2114)

_SOFT_TIME_LIMIT, _HARD_TIME_LIMIT = _resolve_time_limits()

@shared_task(soft_time_limit=_SOFT_TIME_LIMIT, ...)
def run_deep_research(...):

Because @shared_task arguments are evaluated at decoration time (module import), override_settings(DEEP_RESEARCH_SOFT_TIME_LIMIT=...) in tests has no effect on the actual task limits. Any test that wants to verify timeout behaviour will silently run with production limits. Document this limitation prominently, or use a different approach (e.g. read settings inside the task body and use a shorter constant for decorator args).

---

DRY Violation

_decode_global_pk defined twice (config/graphql/research_mutations.py:100 and config/graphql/research_queries.py:215)
These are byte-for-byte identical. Per CLAUDE.md rule 2, extract it to a shared location (e.g. a config/graphql/utils.py or import it from whichever file defines it first). Several other query files already use their own copies — this would be a good time to consolidate all of them.

---

Architecture / Conventions

_load_user_and_corpus bypasses the service layer (llms/tools/research_tools.py:702-707)

def _load_user_and_corpus(user_id: int, corpus_id: int):
    user = get_user_model().objects.filter(pk=user_id).first()
    corpus = Corpus.objects.filter(pk=corpus_id).first()  # direct ORM
    return user, corpus

CLAUDE.md rule 7 requires code with a user context to go through services/. The corpus_id here comes from PydanticAIDependencies (trusted), so there's no IDOR risk in practice — but the pattern still bypasses the invariant enforced by opencontracts.E001. The same applies to _load_conversation_context. Use CorpusService / ConversationService (or BaseService.get_or_none) here.

getattr(agent, "agent_deps", None) is fragile (tasks/research_tasks.py:2305)

deps_ref["deps"] = getattr(agent, "agent_deps", None)

agent_deps appears to be an internal attribute of PydanticAICorpusAgent. If the attribute is renamed or the implementation changes, deps_ref["deps"] silently becomes None, the retrieved set becomes empty, and record_finding rejects every annotation ID — completely breaking citation recording without any exception. A more robust solution would be for the agent factory to expose deps through a public method or return value.

Crashed-worker RUNNING jobs block the concurrency guard indefinitely
A worker crash mid-research leaves the report stuck in RUNNING with no cancel_requested. The concurrency guard (status__in=[QUEUED, RUNNING]) will block new research on that (user, corpus) until the guard window expires (1 hour default). The last_progress_at column is in place (good), but the watchdog is deferred to v2. Consider adding a note in the start() method that explains why the guard window is bounded (i.e. the guard is a soft block, not a hard serialiser).

requires_write_permission=True on start_deep_research_tool (llms/tools/research_tools.py:759)
The research task is strictly read-only on the corpus. Requiring corpus WRITE permission just to kick off a read-only investigation is more restrictive than necessary and could surprise users who have read-only corpus access. Consider whether corpus READ is the right gate here (matching what ResearchReportService.start() checks internally).

---

Performance

append_tool_call does a refresh_from_db on every call (research/services/research_reports.py:1735)
The comment says "Cheap; does not bump progress" but each call still does a SELECT + UPDATE round-trip. For a 60-step run that logs every tool call, that's 60 unnecessary refreshes. The refresh_from_db in append_finding is justified (it reads cancel_requested), but append_tool_call doesn't need to check that flag. Consider removing the refresh there, or at least not reading fields that aren't used.

resolve_full_source_annotation_list / resolve_full_source_document_list can cause N+1
Both resolvers call .all() on M2M relations with no prefetch hint. If researchReports is ever paginated in a list view, each node will fire two additional queries. Add a prefetch_related_fields or document that these fields should only be requested on single-object queries.

---

Test Coverage Gaps

• No GraphQL-level tests for startResearchReport / cancelResearchReport / researchReports. The service and model are tested well, but the API surface (auth enforcement, relay ID decoding, error paths) is untested.
• No test for the Celery task itself (run_deep_research), including the salvage path (_compose_salvage_body) and the SoftTimeLimitExceeded branch.
• No test for build_deep_research_system_prompt with a None corpus description or an oversized prompt.
• The kickoff tool tests correctly use TransactionTestCase — good.

---

Minor / Nits

• is_public field exists but is silently ignored by the manager in v1. test_visible_to_user_anonymous_sees_nothing creates a report with is_public=True to confirm it's still hidden — good defensive test. But any future reader who sees is_public on the model and assumes it works like on other models will be confused. A # v1: is_public has no effect comment on the manager would help.
• The _SOFT_TIME_LIMIT, _HARD_TIME_LIMIT module-level tuple unpack is unusual style for a settings-derived constant. A plain getattr(settings, ..., default) at the top of the task body would be more familiar and sidestep the import-time evaluation issue.
• status field max_length=20 in the model but max_length=24 in the Analysis migration 0022_alter_analysis_status — not a bug (both fit CANCELLED), but the inconsistency is worth noting.

---

Summary

The feature is solid end-to-end: the closed-citation-graph invariant is the right design, the cooperative-cancel pattern is clean, and the service-layer structure follows project conventions. The main items to address before this sees heavy production traffic are the similarity_score omission, the import-time time-limit evaluation, the duplicated _decode_global_pk, and the service-layer bypass in _load_user_and_corpus.