Service layer centralization — Phase 2B & 2C: CorpusService + caller migration (#1716)#1745
Conversation
…monolith (#1716) Split the ~2,900-line corpus_objs_service.py monolith into a segmented corpuses/services/ package, one cohesive module per responsibility, each inheriting the Phase 1 BaseService: - paths.py CorpusPathService DocumentPath disambiguation - corpus_documents.py CorpusDocumentService doc-in-corpus R/W + membership - lifecycle.py DocumentLifecycleService soft-delete / restore / trash - folders.py FolderService folder CRUD + doc-in-folder Pure, behaviour-preserving relocation: all 40 methods move byte-for-byte (verified against the original); the only code change is 14 cross-module call sites rewritten from the monolith's cls.<helper> to an explicit owning-service reference. corpus_objs_service.py becomes a thin shim whose CorpusObjsService facade multiply-inherits the four services, so every existing call site and the test_corpus_objs_service.py regression suite keep working — the suite's only edits are 9 @patch target relocations following moved symbols. Adds test_corpus_services_package.py covering the structural contract (package layout, BaseService inheritance, facade MRO, standalone service operation, cross-service delegation) and the Phase 2 implementation plan. Phase 2 of the service-layer centralization roadmap; Phases 2B (corpus_service.py for Corpus-row CRUD) and 2C (caller migration + shim deletion) follow.
Narrow CorpusFolder | None return values from FolderService.create_folder with explicit asserts before attribute access, resolving the 10 mypy errors that failed the linter check.
…ings - Add a runtime DeprecationWarning to the corpus_objs_service shim so Phase 2C call-site discovery is automatic, consistent with the existing corpus.get_documents() deprecation pattern. - Update stale CorpusObjsService references in service docstrings to the owning segmented service (CorpusDocumentService, FolderService).
TestPermission_SuperuserBypassesAllChecks created a superuser named "admin", which collides with the superuser seeded into every test database by migration 0003_create_initial_superuser (DJANGO_SUPERUSER_* env vars in .envs/.test/.django). The duplicate username raised a UniqueViolation in setUp, failing all three tests in the class. Rename to a unique username, matching the convention used by other admin tests.
…-UK5mC # Conflicts: # CHANGELOG.md
…t rename - Fix shim warnings.warn stacklevel 2->1 so the warning points at the deprecated module itself rather than import machinery internals. - Update plan doc Section 6 to document the runtime DeprecationWarning decision (previously the plan said no warning, contradicting the shim). - Add test_shim_import_emits_deprecation_warning pinning the warning. - Document why the superuser fixture username is su_bypass_admin (a generic "admin" collides across files sharing a worker DB under pytest -n --dist loadscope).
- Wrap the corpus_objs_service shim import in test_corpus_services_package.py in a warnings filter so DeprecationWarning does not surface at pytest collection time (safe if filterwarnings=error is ever added). - Add a TODO note to folders.py documenting its above-guideline size and the candidate Phase 2C split. - Reword the shim's "for one release" language to point at issue #1716 Phase 2C and align "Phase C" -> "Phase 2C". - Add a comment to test_shim_import_emits_deprecation_warning explaining why importlib.reload is needed and why it is safe here. - Trim the Phase 2A CHANGELOG entry to the essential facts.
Address review: the ~1,300-line folders.py exceeded the ~800-line service module guideline. Split it along its natural seam: - folders.py — FolderCRUDService (~800 lines): folder CRUD, the folder tree, search, and bulk structure creation (10 methods). - folder_documents.py — FolderDocumentService (~560 lines): document-in-folder placement, listing, counts, and lookup (6 methods). The two classes have no interdependency, so the split adds no cross-service coupling. Method bodies are relocated verbatim. The CorpusObjsService facade now multiply-inherits all five segmented services; __init__.py, the plan doc, and the CHANGELOG are updated accordingly, and test_corpus_services_package.py gains a standalone test for FolderDocumentService. Also raise the shim's DeprecationWarning stacklevel from 1 to 2 so the warning names the importing file, making each deprecated call site actionable for the Phase 2C migration.
… CRUD (#1716) Add the sixth segmented corpus service — CorpusService(BaseService) — owning the write surface of the Corpus row itself, closing the service-layer design doc §3 Problem 3 gap (corpus contents had a service; the corpus row did not). CorpusService provides: - delete_corpus personal-corpus / user-lock / DELETE-permission gates - set_visibility PERMISSION-gated public/private switch + cascade task - update_description creator-only markdown-description versioning - assert_embedder_change_allowed issue #437 embedder guard - grant_creator_permissions CRUD + PUBLISH + PERMISSION The corpus-row GraphQL mutations delegate their corpus-specific logic to the service: SetCorpusVisibility, UpdateCorpusDescription, and DeleteCorpusMutation become thin wrappers; CreateCorpusMutation / UpdateCorpusMutation keep their shared DRFMutation create/update machinery but call CorpusService for the creator-permission grant and the embedder guard. No GraphQL contract change. Covered by new test_corpus_service.py (16 tests). The make_corpus_public_task @patch target in test_corpus_visibility.py moved to permissioning_tasks following the relocated task dispatch. Phases 2A (shipped) and 2C (caller migration + shim deletion) bracket this work; the 2B/2C plan is in docs/refactor_plans/.
…val (#1716) Migrate every caller of the deprecated CorpusObjsService facade onto the segmented services it shipped behind, then delete the corpus_objs_service.py re-export shim and the CorpusObjsService facade class (CLAUDE.md no-dead-code rule). Phase 2A kept the shim for one release; this completes its removal. - ~23 production caller files (config/graphql/, mcp/, llms/, tasks/, discovery/, utils/) rewritten: the facade import becomes an import of the specific owning service(s) from opencontractserver.corpuses.services, and each CorpusObjsService.<method> call is repointed at its single owning service. The mapping is deterministic — the segmented services have disjoint method names. - test_corpus_objs_service.py (~290-test behavioural regression suite) keeps every scenario and assertion unchanged; only the call targets are repointed. - test_corpus_services_package.py drops the obsolete shim/facade structural tests and adds CorpusService to its package-structure coverage; test_document_service.py's service-split tests now assert against the segmented services. - Docstring/comment references in corpuses/models.py, documents/apps.py, documents/document_service.py, utils/permissioning.py, CLAUDE.md rule 7, and the consolidated permissioning guide repointed at the services package. - schema.graphql description lines regenerated to match the updated resolver docstrings. Pure relocation of import paths and call targets — no permission semantics, transaction boundary, or query shape changes. Verified by the unchanged test_corpus_objs_service.py suite plus the migrated callers' test modules.
Code Review — Phase 2B & 2C: CorpusService + Caller MigrationOverviewThis PR completes the service-layer centralization for issue #1716, adding Phase 2B — CorpusServiceStrengths:
Issues / questions: 1. Permission bootstrap in Both test classes create corpora via Suggestion: Add a brief comment in the test 2. Old code passed 3. All other 4. Local import in The Phase 2C — Caller Migration + Shim DeletionStrengths:
Nits:
Test Coverage
Optional enhancement: A test asserting SecurityNo permission semantics changed. IDOR-safe patterns are maintained throughout:
SummaryWell-executed structural refactor. The migration is mechanical and auditable, the shim is cleanly deleted, and the test suite is updated at the right level of granularity. The main item to address before merge is point 1 — the permission bootstrap question in the new |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
- Document why CorpusService unit tests' creators pass require_permission without an explicit grant (creator short-circuit in _default_user_can) - Note that assert_embedder_change_allowed deliberately returns a plain str rather than ServiceResult - Explain the local make_corpus_public_task import is circular-import avoidance - Add a structural test asserting the deleted corpus_objs_service shim raises ModuleNotFoundError
Code Review — PR #1745: Service Layer Phase 2B & 2COverviewThis PR completes the service-layer centralization roadmap (issue #1716) across two phases:
The approach is sound: pure relocation of call sites, no semantic changes, backed by a 290-test regression suite and 16 new service-level tests. The architecture is significantly cleaner after this. Positives
Issues and Suggestions1.
|
…-phase2bc-1716 # Conflicts: # CHANGELOG.md # CLAUDE.md # docs/refactor_plans/2026-05-21-service-layer-phase2-corpus-services-plan.md # opencontractserver/corpuses/corpus_objs_service.py # opencontractserver/corpuses/services/__init__.py # opencontractserver/corpuses/services/paths.py # opencontractserver/tests/test_corpus_objs_service.py # opencontractserver/tests/test_corpus_services_package.py
Code Review - Service Layer Phase 2B & 2COverviewThis PR completes the CorpusObjsService monolith decomposition (issue #1716):
The diff is large (49 files, +1255/-1523) but the vast majority is mechanical import substitution. The code is clean, well-tested, and follows established patterns throughout. What Works WellService design is consistent.
Permission semantics are preserved exactly.
Mock patch target fix in Test coverage for the new service is solid. 16 tests covering success, creator denial (description), personal-corpus reject, user-lock reject, permission denial, visibility no-op, embedder guard (same/different/blocked), and explicit permission grant. The test docstrings correctly explain the creator short-circuit to prevent assertions from being vacuous. Phase 2C is purely mechanical. No permission semantics, transaction boundary, or query shape changed. Every scenario and assertion in the ~290-test regression suite is preserved - only call targets are repointed. One Gap to Address
Every other mutating method in
The Suggested addition at the end of cls.log_action("Granted creator permissions on", corpus, user)Minor Notes (Non-blocking)
Local vs. top-level imports in SecurityNo concerns. Every entry point that accepts a user-supplied corpus ID continues to run SummaryThe |
Add cls.log_action("Granted creator permissions on", corpus, user) at
the end of CorpusService.grant_creator_permissions so the highest-privilege
mutation in the service is auditable alongside the other write methods.
Code Review — Phase 2B & 2C: CorpusService + Caller Migration (#1716)OverviewThis PR adds Strengths
Observations1.
|
Signed-off-by: JSIV <5049984+JSv4@users.noreply.github.com>
Code Review - PR #1745: Service Layer Phase 2B & 2C
OverviewThis PR completes issue #1716 (service-layer centralization) in two phases:
The migration is pure relocation of import paths and call targets — no permission semantics, transaction boundaries, or query shapes change. StrengthsArchitecture is clean and principled. Zero semantic changes in Phase 2C. Each
Patch target change for
Issues / SuggestionsMinor: Bare
|
| Area | Assessment |
|---|---|
test_corpus_service.py (16 new tests) |
All 5 methods: success, permission-denied, no-op, personal/lock guards |
test_corpus_objs_service.py (~290 tests) |
Same scenarios, all call targets repointed — regression coverage maintained |
test_corpus_services_package.py |
Shim/facade tests correctly dropped; CorpusService added; deletion pinned with ModuleNotFoundError |
test_corpus_visibility.py |
Patch targets and corpus_id assertion updated |
| PR author's test run | 438 + 444 passed across the relevant test files |
One gap: no integration-level test verifying assert_embedder_change_allowed fires end-to-end through UpdateCorpusMutation. The service-unit tests in TestCorpusServiceEmbedderGuard are solid, but a mutation-level test (similar to test_corpus_visibility.py) would fully close the loop.
Verdict
Clean, disciplined refactor with no semantic changes to production behaviour. Architecture is sound, migration is complete, documentation is thorough, test coverage is solid. The two nits (bare assert style, missing corpus-creation context in the class docstring) are cosmetic. The acknowledged corpus_id type change is low-risk and properly tested.
…ce.set_visibility (#1763) Phase 2B (#1745) carried over the pre-existing inline `make_corpus_public_task.si(corpus_id=corpus.pk).apply_async()` from `SetCorpusVisibility.mutate` byte-for-byte. Under `ATOMIC_REQUESTS = True` the GraphQL mutation runs in a request-wrapping transaction, so an unwrapped `apply_async` could race a Celery worker against uncommitted state and fire the cascade after the request transaction rolls back on a mid-handler exception -- publicising every child object of a corpus whose own `is_public` flag never persisted. Wraps the dispatch in `transaction.on_commit`, matching every other Celery dispatch site in `config/graphql/corpus_mutations.py` (`StartCorpusFork`, `ReEmbedCorpus`, ...). `corpus.pk` is captured into a local before the closure so the lambda body doesn't depend on the `corpus` instance still being attached at commit time. Tests: - New regression test `test_make_public_task_not_dispatched_on_rollback` in `test_corpus_service.py` pins the safety invariant by forcing a rollback inside a `transaction.atomic()` and asserting the task is never dispatched. This test fails against the pre-fix inline code. - Existing positive test `test_make_public_dispatches_cascade_task` is wrapped in `self.captureOnCommitCallbacks(execute=True)` so the deferred callback fires synchronously under `TestCase`. - GraphQL-level `test_owner_can_make_corpus_public` in `test_corpus_visibility.py` gets the same `captureOnCommitCallbacks` wrapper so its `mock_task.si.assert_called_once_with` assertion still pins task arguments. Pure correctness fix -- no GraphQL contract change, no permission semantics change. Closes #1761 Signed-off-by: JSIV <5049984+JSv4@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
Summary
Phases 2B and 2C of the service-layer centralization roadmap (issue #1716), stacked on Phase 2A (PR #1737). Targets the 2A branch so this diff is exactly 2B + 2C; GitHub will auto-retarget to
mainonce #1737 merges.Phase 2A split the
corpus_objs_service.pymonolith into the segmentedcorpuses/services/package behind a deprecatedCorpusObjsServicere-export shim. This PR adds the sixth service and removes the shim.Phase 2B —
CorpusServicefor Corpus-row CRUDNew
CorpusService(BaseService)(corpuses/services/corpus_service.py) owning the write surface of theCorpusrow itself, closing design-doc §3 Problem 3 (corpus contents had a service; the corpus row did not):delete_corpus— personal-corpus / user-lock / DELETE-permission gatesset_visibility— PERMISSION-gated public/private switch +make_corpus_public_taskcascadeupdate_description— creator-only markdown-description versioningassert_embedder_change_allowed— issue Embedder Consistency Issue: PREVENT Need for Re-embedding Support #437 embedder guardgrant_creator_permissions— CRUD + PUBLISH + PERMISSIONSetCorpusVisibility,UpdateCorpusDescription, andDeleteCorpusMutationbecome thin wrappers.CreateCorpusMutation/UpdateCorpusMutationkeep their sharedDRFMutationcreate/update machinery (deliberately — gutting it for one model would be a needless frontend-contract risk) but delegate their corpus-specific logic toCorpusService. No GraphQL contract change.Phase 2C — caller migration + shim removal
Every caller of the
CorpusObjsServicefacade is migrated onto the segmented services, andcorpus_objs_service.py+ theCorpusObjsServicefacade class are deleted (CLAUDE.md no-dead-code rule):test_corpus_objs_service.py(~290-test regression suite) keeps every scenario/assertion unchanged — only call targets repointedtest_corpus_services_package.pydrops the obsolete shim/facade structural tests, addsCorpusServiceschema.graphqldescriptions updatedPure relocation of import paths and call targets — no permission semantics, transaction boundary, or query shape changes.
Test plan
pytest test_corpus_objs_service.py test_corpus_services_package.py test_corpus_service.py test_base_service.py test_document_service.py test_permanent_deletion.py test_zip_import_integration.py test_corpus_get_documents_deprecation.py test_corpus_folder_mutations.py permissioning/test_permission_optimizer.py permissioning/test_corpus_visibility.py— 438 passedpytest test_llms_typing_coverage.py test_move_document_tool.py test_document_stats.py test_import_utils.py test_mcp_extended.py test_og_metadata_queries.py test_extract_mutations.py test_document_relationship_mutations.py test_llm_tools.py test_caml_review_tools.py— 444 passed (1 unrelated env error — parallel-worker PDF-storage collision — confirmed passing in isolation)black/isort/flake8/mypyclean on all changed filestest_corpus_service.py— 16 tests covering everyCorpusServicemethodPlan:
docs/refactor_plans/2026-05-22-service-layer-phase2bc-corpus-service-and-caller-migration.mdCloses the remainder of #1716 (Phases 2B + 2C; Phase 2A is PR #1737).