Merge branch 'main' into chore/phase1-a1

Author: rocketstack-matt

calm-hub/PERMISSIONS.md

@@ -1,37 +1,110 @@
-# Structure of CalmHub permissions system
+# CalmHub Permissions — Developer Reference
 
-CalmHub drives its permission system from the in-memory database.
-Entitlements are stored as `UserAccess` records.
+For user-facing documentation, see [`docs/docs/working-with-calm/calm-hub-entitlements.md`](../docs/docs/working-with-calm/calm-hub-entitlements.md).
 
-## Structure of entitlements model
+---
 
-Entitlements are applied at a per-namespace level, at domain level for control requirements and configurations.
+## Storage model
 
-The available actions are the following. 
+Entitlements are stored as `UserAccess` records (see `domain/UserAccess.java`). Each record has:
+- `username` — a real username from the OIDC provider, or `*` for the public-access wildcard
+- `permission` — `read`, `write`, or `admin`
+- `namespace` — the namespace this grant applies to (mutually exclusive with `domain`)
+- `domain` — the control domain this grant applies to (mutually exclusive with `namespace`)
 
-| Action  | Description                                                                                                                                                                       |
-|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `read`  | Can read any documents of that type in the namespace.                                                                                                                             |
-| `write` | Can write any documents of that type in the namespace. This includes deleting them. Note that by default resources in CalmHub are immutable, so this usually means 'create' only. |
-| `admin` | Can do anything to all resource types, and also grant entitlements to other users in the namespace.                                                                               |
+The `*` wildcard is a valid username value (per `USERNAME_REGEX` in `ResourceValidationConstants`). It represents any user and is evaluated by the same logic as named user grants.
 
-For example, `read` means the user can read all resources in that NS.
+---
 
-Please note that each entitlement implies all previous levels - i.e. `write` implies `read`.
-`admin` implies `read` and `write` on all resource types.
+## Permission levels
 
-## Global admin
+| Permission | Implies |
+|------------|---------|
+| `read`     | — |
+| `write`    | `read` |
+| `admin`    | `read`, `write` |
 
-Some resources aren't tied to any one namespace.
-Creating namespaces and managing core schemas requires the `admin` role, with the namespace `GLOBAL` in the database.
+---
 
-**NOTE**: Global admin also gives you the right to manage domains. 
-There's only one notion of global admin.
+## Hierarchical namespace rules
 
-## Global READ mode
+Namespaces use `.` as a separator (`org`, `org.ab`, `org.ab.cd`). The ancestor chain for a namespace is the list of all prefixes, inclusive of the namespace itself.
 
-It's possible to configure CalmHub to grant `read` to all users by default.
+### READ — AND across ancestor chain
 
-To do this, set the property `calm.auth.allow-public-read=true`.
-By default this property is `false`.
+```
+canRead(username, namespace):
+    grants = store.getGrantsForUser(username)   // user grants + * grants in one query
+    ancestors = ancestorChain(namespace)         // ["org", "org.ab", "org.ab.cd"] for "org.ab.cd"
+    return ancestors.ALL(level →
+        grants.ANY(g → g.namespace == level && permissionSufficient(g, READ)))
+```
 
+**Every** level must have a matching grant. This allows a parent to be public while a child is restricted.
+
+### WRITE / ADMIN — OR across ancestor chain
+
+```
+canWrite(username, namespace):
+    grants = store.getGrantsForUser(username)
+    ancestors = ancestorChain(namespace)
+    return ancestors.ANY(level →
+        grants.ANY(g → g.namespace == level && permissionSufficient(g, WRITE)))
+```
+
+**Any** ancestor (including the namespace itself) with a sufficient grant is enough. Grants at a parent cascade to all descendants for write/admin purposes.
+
+### `permissionSufficient`
+
+| Requested | Sufficient grant |
+|-----------|-----------------|
+| READ      | `read`, `write`, or `admin` |
+| WRITE     | `write` or `admin` |
+| ADMIN     | `admin` only |
+
+---
+
+## Key components
+
+| Class | Role |
+|-------|------|
+| `CalmHubPermissionChecker` | Implements the AND/OR hierarchical logic; calls `getGrantsForUser` |
+| `UserAccessStore` (interface) | `getGrantsForUser(username)` returns user + `*` grants in one query |
+| `MongoUserAccessStore` | Mongo implementation of `getGrantsForUser` |
+| `NitriteUserAccessStore` | Nitrite implementation of `getGrantsForUser` |
+| `UserAccessValidator` | `getReadableNamespaces(username)` — exact ancestor-chain filter used by search |
+| `NamespaceService` | Orchestrates namespace creation + automatic `* read` grant insertion |
+| `NamespaceMigrationService` | StartupEvent observer; backfills `* read` grants on pre-existing namespaces |
+
+---
+
+## Special bypasses
+
+### `allow-public-read` config flag
+
+`calm.auth.allow-public-read=true` is a global bypass in `CalmHubPermissionChecker.canRead`. It short-circuits all namespace checks. Intended for fully-open deployments; default is `false`.
+
+### GLOBAL admin
+
+A user with `admin` on `GLOBAL` bypasses all namespace-level permission checks via `hasGlobalAdmin`. Domain access is also granted. `GLOBAL` is not part of the namespace hierarchy.
+
+---
+
+## Default-open namespace behaviour
+
+When a namespace is created via `NamespaceService`, a `UserAccess("*", read, namespace)` record is inserted automatically. This keeps the hub open by default.
+
+To restrict a namespace: delete the `* read` record. The AND rule for child namespaces means restriction cascades automatically.
+
+---
+
+## `getGrantsForUser` vs `getUserAccessForUsername`
+
+- **`getGrantsForUser(username)`** — returns grants where `username = :username OR username = '*'`. Used by the permission checker (single round-trip per request).
+- **`getUserAccessForUsername(username)`** — returns grants for exactly one username. Used by admin/management endpoints (`UserAccessResource`, `UserAccessValidator`) that need to inspect a single user's grants without `*` mixed in.
+
+---
+
+## Domain-scoped access
+
+`hasDomainAccess`, `canReadByDomain`, etc. are **not** hierarchical — domain access remains flat. `UserAccess` records have either `namespace` or `domain` set, never both.

calm-hub/mongo/init-mongo.js

@@ -84,9 +84,9 @@ if (db.counters.countDocuments({ _id: "flowStoreCounter" }) === 0) {
 if (db.counters.countDocuments({ _id: "userAccessStoreCounter" }) === 0) {
     db.counters.insertOne({
         _id: "userAccessStoreCounter",
-        sequence_value: 6
+        sequence_value: 11
     });
-    logSuccess("Initialized userAccessStoreCounter with sequence_value 6");
+    logSuccess("Initialized userAccessStoreCounter with sequence_value 11");
 } else {
     logSkip("userAccessStoreCounter already exists, no initialization needed");
 }
@@ -2532,9 +2532,33 @@ if (db.userAccess.countDocuments() === 0) {
             "username": "demo",
             "permission": "read",
             "namespace": "workshop"
+        },
+        {
+            "userAccessId": NumberInt(7),
+            "username": "*",
+            "permission": "read",
+            "namespace": "finos"
+        },
+        {
+            "userAccessId": NumberInt(8),
+            "username": "*",
+            "permission": "read",
+            "namespace": "workshop"
+        },
+        {
+            "userAccessId": NumberInt(9),
+            "username": "*",
+            "permission": "read",
+            "namespace": "traderx"
+        },
+        {
+            "userAccessId": NumberInt(10),
+            "username": "*",
+            "permission": "read",
+            "namespace": "ai-governance-v2"
         }
     ]);
-    logSuccess("Initialized user access for demo_admin and demo users");
+    logSuccess("Initialized user access for demo_admin, demo, and * (public read) users");
 } else {
     logSkip("User access already initialized, skipping...");
 }

calm-hub/src/main/java/org/finos/calm/mcp/tools/SearchTools.java

@@ -102,7 +102,7 @@ private Optional<Set<String>> resolveReadableNamespaces() {
         if (!authEnabled || !userAccessValidatorInstance.isResolvable()) {
             return Optional.empty();
         }
-        return Optional.of(userAccessValidatorInstance.get().getReadableNamespaces(identity.getPrincipal().getName()));
+        return userAccessValidatorInstance.get().getReadableNamespaces(identity.getPrincipal().getName());
     }
 
     private static Map<String, List<SearchResult>> toGroupMap(GroupedSearchResults results) {

calm-hub/src/main/java/org/finos/calm/resources/NamespaceResource.java

@@ -16,7 +16,7 @@
 import org.finos.calm.domain.namespaces.NamespaceInfo;
 import org.finos.calm.security.CalmHubPermissionChecker;
 import org.finos.calm.security.CalmHubScopes;
-import org.finos.calm.store.NamespaceStore;
+import org.finos.calm.services.NamespaceService;
 
 import java.net.URI;
 import java.net.URISyntaxException;
@@ -25,11 +25,11 @@
 @Path("/api/calm/namespaces")
 public class NamespaceResource {
 
-    private final NamespaceStore namespaceStore;
+    private final NamespaceService namespaceService;
 
     @Inject
-    public NamespaceResource(NamespaceStore store) {
-        this.namespaceStore = store;
+    public NamespaceResource(NamespaceService namespaceService) {
+        this.namespaceService = namespaceService;
     }
 
     @GET
@@ -39,7 +39,7 @@ public NamespaceResource(NamespaceStore store) {
     )
     @Authenticated
     public ValueWrapper<NamespaceInfo> namespaces() {
-        return new ValueWrapper<>(namespaceStore.getNamespaces());
+        return new ValueWrapper<>(namespaceService.getNamespaces());
     }
 
     @POST
@@ -62,7 +62,7 @@ public Response createNamespace(@Valid @NotNull(message = "Request must not be n
         }
 
         try {
-            namespaceStore.createNamespace(name, description);
+            namespaceService.createNamespace(name, description);
         } catch (NamespaceAlreadyExistsException e) {
             return Response.status(Response.Status.CONFLICT)
                     .entity("{\"error\":\"Namespace already exists\"}")

calm-hub/src/main/java/org/finos/calm/resources/ResourceValidationConstants.java

@@ -6,9 +6,9 @@
 public class ResourceValidationConstants {
     public static final String NAMESPACE_REGEX = "^[A-Za-z0-9-]+(\\.[A-Za-z0-9-]+)*$";
     public static final String NAMESPACE_MESSAGE = "namespace must match pattern '^[A-Za-z0-9-]+([.][A-Za-z0-9-]+)*$'";
-    // dashes, dots, usernames, upper/lowercase letters and numbers.
-    public static final String USERNAME_REGEX = "^[A-Za-z0-9._-]+$";
-    public static final String USERNAME_MESSAGE = "username must match pattern '^[A-Za-z0-9._-]+$'";
+    // dashes, dots, usernames, upper/lowercase letters, numbers, and * for the public-access wildcard.
+    public static final String USERNAME_REGEX = "^(\\*|[A-Za-z0-9._-]+)$";
+    public static final String USERNAME_MESSAGE = "username must match pattern '^(\\*|[A-Za-z0-9._-]+)$'";
     public static final String DOMAIN_REGEX = "^[A-Za-z0-9-]+$";
     public static final String DOMAIN_MESSAGE = "domain name must match pattern '^[A-Za-z0-9-]+$'";
     public static final String VERSION_REGEX = "^(0|[1-9][0-9]*)[-.]?(0|[1-9][0-9]*)[-.]?(0|[1-9][0-9]*)$";

calm-hub/src/main/java/org/finos/calm/resources/SearchResource.java

@@ -81,7 +81,7 @@ private Optional<Set<String>> resolveReadableNamespaces() {
         if (!authEnabled || !userAccessValidatorInstance.isResolvable()) {
             return Optional.empty();
         }
-        return Optional.of(userAccessValidatorInstance.get().getReadableNamespaces(identity.getPrincipal().getName()));
+        return userAccessValidatorInstance.get().getReadableNamespaces(identity.getPrincipal().getName());
     }
 
     /**

calm-hub/src/main/java/org/finos/calm/security/CalmHubPermissionChecker.java

@@ -12,6 +12,8 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import java.util.ArrayList;
+import java.util.List;
 import java.util.function.Predicate;
 
 
@@ -102,9 +104,9 @@ public boolean hasGlobalAdmin(SecurityIdentity identity) {
                                     && permissionSufficient(grant, UserAction.ADMIN));
 
             if (granted) {
-                logger.info("User [{}] AUTHORIZED for GLOBAL admin privileges", username);
+                logger.debug("User [{}] AUTHORIZED for GLOBAL admin privileges", username);
             } else {
-                logger.warn("User [{}] DENIED for GLOBAL admin privileges", username);
+                logger.debug("User [{}] DENIED for GLOBAL admin privileges", username);
             }
             return granted;
         } catch (UserAccessNotFoundException e) {
@@ -114,8 +116,45 @@ public boolean hasGlobalAdmin(SecurityIdentity identity) {
     }
 
     private boolean hasNamespaceAccess(SecurityIdentity identity, String namespace, UserAction action) {
-        return hasAccess(identity, "namespace", namespace, action,
-                grant -> namespace != null && namespace.equals(grant.getNamespace()));
+        if (namespace == null) return false;
+        String username = identity.getPrincipal().getName();
+        logger.debug("Checking namespace access for user [{}] on namespace [{}] action=[{}]", username, namespace, action);
+
+        List<UserAccess> grants = userAccessStore.getGrantsForUser(username);
+        List<String> ancestors = ancestorChain(namespace);
+
+        boolean result = action == UserAction.READ
+                ? allAncestorsHaveGrant(ancestors, grants, action)
+                : anyAncestorHasGrant(ancestors, grants, action);
+
+        if (result) {
+            logger.debug("User [{}] AUTHORIZED for [{}] in namespace [{}]", username, action, namespace);
+        } else {
+            logger.debug("User [{}] DENIED for [{}] in namespace [{}]", username, action, namespace);
+        }
+        return result;
+    }
+
+    private boolean allAncestorsHaveGrant(List<String> ancestors, List<UserAccess> grants, UserAction action) {
+        return ancestors.stream().allMatch(level ->
+                grants.stream().anyMatch(g -> level.equals(g.getNamespace()) && permissionSufficient(g, action)));
+    }
+
+    private boolean anyAncestorHasGrant(List<String> ancestors, List<UserAccess> grants, UserAction action) {
+        return ancestors.stream().anyMatch(level ->
+                grants.stream().anyMatch(g -> level.equals(g.getNamespace()) && permissionSufficient(g, action)));
+    }
+
+    static List<String> ancestorChain(String namespace) {
+        String[] parts = namespace.split("\\.");
+        List<String> chain = new ArrayList<>(parts.length);
+        StringBuilder sb = new StringBuilder();
+        for (String part : parts) {
+            if (!sb.isEmpty()) sb.append('.');
+            sb.append(part);
+            chain.add(sb.toString());
+        }
+        return chain;
     }
 
     private boolean hasDomainAccess(SecurityIdentity identity, String domain, UserAction action) {
@@ -132,9 +171,9 @@ private boolean hasAccess(SecurityIdentity identity, String scopeType, String sc
             boolean result = userAccessStore.getUserAccessForUsername(username).stream()
                     .anyMatch(grant -> grantMatcher.test(grant) && permissionSufficient(grant, action));
             if (result) {
-                logger.info("User [{}] AUTHORIZED for [{}] in {} [{}]", username, action, scopeType, scopeValue);
+                logger.debug("User [{}] AUTHORIZED for [{}] in {} [{}]", username, action, scopeType, scopeValue);
             } else {
-                logger.warn("User [{}] DENIED for [{}] in {} [{}]", username, action, scopeType, scopeValue);
+                logger.debug("User [{}] DENIED for [{}] in {} [{}]", username, action, scopeType, scopeValue);
             }
             return result;
         } catch (UserAccessNotFoundException e) {