feat(BA-6555): AppConfig merge engine + resolve service (service layer) (#12359)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: jopemachine

changes/12359.feature.md

@@ -0,0 +1 @@
+Add the app_config read service: resolve a user's merged AppConfig for one or many config names by rank-ordering the visible public / domain / user fragments and deep-merging them.

proposals/BEP-1052-scoped-app-config-redesign.md

@@ -21,7 +21,9 @@ each managed independently.
 `config_name` is the **deep merge of every fragment that applies to
 them**, taken from `app_config_fragments` ordered by the `rank` of each
 fragment's allow-list entry — one indexed join, **no permission
-lookup**.
+lookup**. Nor is one needed to keep callers apart: a read names only the
+domain, and the user half of the scope is injected from the session, so
+a caller has no way to ask for anyone else's config.
 
 **Write authorization is set up ahead of time.** Every config name is
 **explicitly registered** in `app_config_definitions`, and `app_config_allow_list`
@@ -71,11 +73,14 @@ Three scopes cover the use cases (`public` for the pre-login shell):
   fragment requires an entry, and the entry requires a registered name — so
   no direct fragment FK is needed). No fragment may exist for an
   unregistered `config_name`.
-- **Reads are unconditional.** The merge **must** join
+- **Reads cost no permission query.** The merge **must** join
   `app_config_fragments` to `app_config_allow_list` — `rank` lives only on
   the allow-list entry, so the ordering cannot be computed without it (an
   indexed `(config_name, scope_type)` join). The join is for `rank` alone;
-  no permission or policy is evaluated at read time.
+  no RBAC or policy is evaluated at read time. Tenancy is structural rather
+  than checked: the read takes a `domain_id` and nothing else, and the
+  service fills the user half of the scope from the session, so there is no
+  field in which to name another user.
 - **Allow-list = the write gate and the merge order.** `app_config_allow_list`
   holds **one record per `(config_name, scope_type)`**; a fragment at
   that scope may be created **only if** the record exists — through the
@@ -192,10 +197,11 @@ Two kinds of mutation:
 
 `create` errors if the natural key already exists; `update` errors if it
 does not; `purge` removes the row (and thus its contribution to the
-merge). A caller "clears" a config without deleting it by `update`-ing
-with `{}`, which reads back as `null` (null projection, §3). `update`
-replaces the stored JSON wholesale — no partial/deep update at the write
-boundary.
+merge). `update` replaces the stored JSON wholesale — no partial/deep
+update at the write boundary. A caller "clears" a config without deleting
+it by `update`-ing with `{}`; the fragment stays and still counts as a
+contribution, so the merge succeeds and yields whatever the other
+fragments hold. Removing the contribution entirely is `purge`.
 
 **Overridability is a write-grant decision:**
 
@@ -260,18 +266,35 @@ those whose scope applies to them:
 - the user's `user` fragment (`scope_id = the user's id`).
 
 A single `app_config_fragments` query selects exactly those rows (the
-user's domain is known from the session — no permission check), joins
+user's domain is known from the session — no RBAC lookup), joins
 each to its allow-list entry for the `rank`, orders by it (low → high),
 and deep-merges: nested objects recurse, scalars and lists are
 wholesale-replaced, and the higher `rank` wins on conflict.
 
-**Null projection.** A stored `config` of `{}` reads back as `null`, and
-a merged `config` that is empty after combining every fragment is
-likewise `null` — clients fall back to their built-in defaults.
+**Nothing to merge is a 404.** A `config_name` no visible fragment
+contributes to raises `AppConfigFragmentNotFound` rather than resolving
+to an empty or null value — the name is unregistered, or nothing is
+visible at the caller's scopes. Clients fall back to their built-in
+defaults on the 404.
+
+The merged `config` is therefore always an object, never null. It is
+empty only when every contributing fragment's own `config` was `{}`: the
+merge adds and replaces keys but never drops one, so no combination of
+non-empty fragments can reduce to `{}`.
 
 ### Read variants
 
-- **Single** — resolve one `(user, config_name)` to its `AppConfig`.
+- **Resolve** — the only read. Takes a list of `config_name`s and
+  returns one `AppConfig` per requested name, in request order; a
+  repeated name is repeated in the output. A single name is a
+  one-element request — there is no separate single-name variant, since
+  a client bootstrapping its shell asks for several configs at once and
+  two entry points would only differ in how they report a missing name.
+  **All-or-nothing:** one requested name nothing contributes to fails
+  the whole call. A partial result would have to mark the absent names
+  somehow, and every way of doing that pushes the caller into branching
+  on a second, quieter kind of failure. The cost is that a client cannot
+  batch optional config names together with required ones.
 - **Search (self)** — paginate the user's own `AppConfig`s, grouped by
   `(user_id, config_name)`; each name's merge is evaluated
   independently.

src/ai/backend/manager/data/app_config/__init__.py

@@ -0,0 +1,20 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any
+
+from ai.backend.manager.data.app_config_fragment.types import AppConfigFragmentData
+
+
+@dataclass(frozen=True)
+class AppConfigData:
+    """Merged per-user view of one ``config_name``.
+
+    The contributing ``fragments`` (rank low -> high) plus their deep-merged ``merged_config``.
+    At least one fragment always contributes, so an empty ``merged_config`` means the
+    fragments themselves were empty — not that none were found.
+    """
+
+    config_name: str
+    fragments: list[AppConfigFragmentData]
+    merged_config: dict[str, Any]

src/ai/backend/manager/data/app_config/types.py

@@ -34,7 +34,7 @@
     AppConfigFragmentPurgerSpec,
 )
 from ai.backend.manager.repositories.app_config_fragment.types import (
-    AppConfigScopeArguments,
+    ResolvedAppConfigScope,
 )
 from ai.backend.manager.repositories.base import (
     BatchQuerier,
@@ -210,24 +210,22 @@ async def scoped_search(
 
     @app_config_fragment_db_source_resilience.apply()
     async def list_visible_fragments_bulk(
-        self, config_names: list[str], scope: AppConfigScopeArguments
+        self, config_names: list[str], scope: ResolvedAppConfigScope | None = None
     ) -> list[AppConfigFragmentData]:
-        """Visible fragments for several ``config_names`` at once, in a single query.
-
-        Selects the requested names AND any one of the principal's visible scopes (public,
-        its domain, or its own user). The scope filter is name-independent, so it is a single
-        OR group AND-combined with the name membership. Merge priority (``rank``) lives on the
-        joined allow-list entry; the result is always ordered by ascending ``rank`` so the
-        caller can group by name (each name's subset stays rank-ordered) and deep-merge each
-        name's fragments in order.
+        """Visible fragments for several ``config_names`` in one query, ordered by ascending ``rank``.
+
+        ``public`` always contributes; a ``scope`` additionally admits its domain and user
+        overlay, while ``scope=None`` (anonymous) sees only ``public``. Rank-ordered so the
+        caller can group by name and deep-merge each name's fragments in order.
         """
         if not config_names:
             return []
-        scope_visibility = [
-            AppConfigFragmentConditions.by_public_visibility(),
-            AppConfigFragmentConditions.by_domain_visibility(str(scope.domain_id)),
-            AppConfigFragmentConditions.by_user_visibility(str(scope.user_id)),
-        ]
+        scope_visibility = [AppConfigFragmentConditions.by_public_visibility()]
+        if scope is not None:
+            scope_visibility += [
+                AppConfigFragmentConditions.by_domain_visibility(str(scope.domain_id)),
+                AppConfigFragmentConditions.by_user_visibility(str(scope.user_id)),
+            ]
         querier = BatchQuerier(
             pagination=NoPagination(),
             conditions=[

src/ai/backend/manager/repositories/app_config_fragment/db_source/db_source.py

@@ -25,7 +25,7 @@
     AppConfigFragmentPurgerSpec,
 )
 from ai.backend.manager.repositories.app_config_fragment.types import (
-    AppConfigScopeArguments,
+    ResolvedAppConfigScope,
 )
 from ai.backend.manager.repositories.base import (
     BatchQuerier,
@@ -105,6 +105,6 @@ async def bulk_purge(
 
     @app_config_fragment_repository_resilience.apply()
     async def list_visible_fragments_bulk(
-        self, config_names: list[str], scope: AppConfigScopeArguments
+        self, config_names: list[str], scope: ResolvedAppConfigScope | None = None
     ) -> list[AppConfigFragmentData]:
         return await self._db_source.list_visible_fragments_bulk(config_names, scope)

src/ai/backend/manager/repositories/app_config_fragment/repository.py

@@ -17,17 +17,27 @@
 
 __all__ = (
     "AppConfigScopeArguments",
+    "ResolvedAppConfigScope",
     "DomainAppConfigFragmentSearchScope",
     "UserAppConfigFragmentSearchScope",
 )
 
 
 @dataclass(frozen=True)
 class AppConfigScopeArguments:
+    """The scope arguments a caller supplies for a resolve — the domain, never the user.
+
+    Add new caller-supplied scope dimensions here rather than growing method signatures.
+    """
+
+    domain_id: DomainID
+
+
+@dataclass(frozen=True)
+class ResolvedAppConfigScope:
     """The principal an ``AppConfig`` is resolved for: the resolving user and its domain.
 
-    Bundles the scope-identifying arguments so they travel together (add new principal
-    dimensions here rather than growing method signatures). Plain value object — not a
+    :class:`AppConfigScopeArguments` plus the session user. Plain value object — not a
     :class:`SearchScope`.
     """
 

src/ai/backend/manager/repositories/app_config_fragment/types.py

@@ -8,6 +8,9 @@
 from ai.backend.manager.repositories.app_config_definition.repositories import (
     AppConfigDefinitionRepositories,
 )
+from ai.backend.manager.repositories.app_config_fragment.repositories import (
+    AppConfigFragmentRepositories,
+)
 from ai.backend.manager.repositories.artifact.repositories import ArtifactRepositories
 from ai.backend.manager.repositories.artifact_registry.repositories import (
     ArtifactRegistryRepositories,
@@ -98,6 +101,7 @@ class Repositories:
     agent: AgentRepositories
     app_config_allow_list: AppConfigAllowListRepositories
     app_config_definition: AppConfigDefinitionRepositories
+    app_config_fragment: AppConfigFragmentRepositories
     auth: AuthRepositories
     container_registry: ContainerRegistryRepositories
     deployment: DeploymentRepositories
@@ -154,6 +158,7 @@ def create(cls, args: RepositoryArgs) -> Self:
         agent_repositories = AgentRepositories.create(args)
         app_config_allow_list_repositories = AppConfigAllowListRepositories.create(args)
         app_config_definition_repositories = AppConfigDefinitionRepositories.create(args)
+        app_config_fragment_repositories = AppConfigFragmentRepositories.create(args)
         auth_repositories = AuthRepositories.create(args)
         container_registry_repositories = ContainerRegistryRepositories.create(args)
         deployment_repositories = DeploymentRepositories.create(args)
@@ -211,6 +216,7 @@ def create(cls, args: RepositoryArgs) -> Self:
             agent=agent_repositories,
             app_config_allow_list=app_config_allow_list_repositories,
             app_config_definition=app_config_definition_repositories,
+            app_config_fragment=app_config_fragment_repositories,
             auth=auth_repositories,
             container_registry=container_registry_repositories,
             deployment=deployment_repositories,