[#11103] feat(core): Add the hierarchy convention layer#11074
Conversation
- RelationalSchemaNamingBridge and JDBCBackend entity/relation conversions - SupportsRelationOperations.batchInsertRelations; RelationalEntityStore cache invalidation - RelationalBackend; backend-focused tests Co-authored-by: Cursor <cursoragent@cursor.com>
- Replace Preconditions.checkNotNull with Objects.requireNonNull in JDBCBackend.batchInsertRelations for consistency with the interface - Add @SuppressWarnings explanatory comments in unchecked-cast methods - Document FILESET/TOPIC/MODEL/MODEL_VERSION passthrough in default switch branches of nameIdentifierForStorage/nameIdentifierForApi - Expand Javadoc on embeddedNamespaceForStorage/Api to warn about the index-2 layout assumption - Guard statisticEntityForApi/Storage with Preconditions.checkArgument to reject non-TableStatisticEntity subtypes at the call site - Add tests: roleEntityForStorage/Api round-trips, batchInsertRelations cache invalidation in RelationalEntityStore, and non-OWNER_REL rejection in JDBCBackend.batchInsertRelations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extract mapRoleSecurableObjects helper to eliminate the duplicate
builder block shared by roleEntityForStorage and roleEntityForApi
- Extract statisticEntityWithNamespace helper to consolidate the type
guard and builder shared by statisticEntityForApi and
statisticEntityForStorage
- Replace repeated Lists.newArrayList(Privileges.UseSchema.allow()) and
SecurableObjects.ofCatalog("catalog", ...) constructions in tests with
private static final constants USE_SCHEMA_PRIVS and CATALOG_OBJ
- Remove inline WHAT-comments that restate what the assertions already
express
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…yStore batchDelete and batchPut were the only write paths that bypassed the cache — callers that relied on them would silently observe stale reads. Apply the same pattern as their single-item counterparts: - batchDelete: invalidate each (ident, entityType) pair after backend - batchPut: put each entity into cache after backend Add unit tests covering the backend-then-cache ordering for both methods. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nalEntityStore" This reverts commit cb21732.
…Identifier-based approach Use the existing nameIdentifierForStorage/Api infrastructure via a sentinel metalake prefix instead of bespoke string-splitting logic. Removes the package-private convertMetadataObjectDottedFullName and convertSchemaSegmentAt helpers; updates securableObjectForStorage/Api and genericEntityMetadataFullNameForApi accordingly. Tests are replaced with equivalent coverage through the public securableObject API. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dataObject entity types Entity types without a MetadataObject.Type equivalent (e.g. TABLE_STATISTIC, MODEL_VERSION) must not be schema-converted. Re-add the MetadataObject.Type.valueOf guard that was dropped in the previous refactor. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Code Coverage Report
Files
|
Throw NoSuchEntityException with the correct entity type by checking catalogId/schemaId on the joined row, so callers see "catalog not found" or "schema not found" instead of a misleading parent error. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use the same pattern as Table: when the join returns no row or a placeholder row, throw NoSuchEntityException with the correct entity type by inspecting catalogId/schemaId on the result. Also switch the function-by-full-name select from INNER JOIN to LEFT JOIN on the version table, matching the list query and the Table/View shape. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Keep the INNER JOIN on the version table for the function-by-full-name select. Revert the Java getPOByFullName checks accordingly so the caller continues to handle the null PO case. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
@jerryshao @yuqi1129 Could u help me review this pull request? |
|
Is it ready to review? |
Yes, it is ready to review. |
|
The overall direction here is good — extracting per-entity PO logic into dedicated classes and using a decorator for hierarchical name translation is a clean separation. A few observations on the design that might be worth considering. The core tension
Mixing these in the base class causes the An alternative: expose two explicit SQL methods, route in the service // BasePOStorageOps: pure SQL delegation, no routing logic
abstract class BasePOStorageOps<PO, Mapper> {
// ID-based (cache-enabled path)
public PO getPOByParentId(Mapper mapper, Long parentId, String name) { ... }
public List<PO> listPOsByParentId(Mapper mapper, Long parentId) { ... }
// Name-based (full join, no-cache path)
public PO getPOByFullName(Mapper mapper, NameIdentifier ident) { ... }
public List<PO> listPOsByNSFullName(Mapper mapper, Namespace ns) { ... }
}Routing moves back to the service as a single private helper per operation — not repeated per-method: // SchemaMetaService
private SchemaPO getSchemaPO(NameIdentifier ident) {
return SessionUtils.getWithoutCommit(SchemaMetaMapper.class, mapper ->
cacheEnabled()
? ops.getPOByParentId(mapper, getCatalogId(ident), ident.name())
: ops.getPOByFullName(mapper, ident));
}Since This eliminates the What stays the same
|
| private final UnaryOperator<PO> readRewriter; | ||
| private final UnaryOperator<PO> writeRewriter; |
There was a problem hiding this comment.
These two variable names are confusing. What is the meaning of writeWriter? You'd better pick a better name.
There was a problem hiding this comment.
OK, I will rename them to LogicalToPhyiscalRewriter and PhysicalToLogicalRewriter.
| return HierarchicalSchemaUtil.logicalToPhysical(name, sep); | ||
| } | ||
|
|
||
| private static NameIdentifier apiIdentifierToStorage(NameIdentifier apiIdentifier) { |
There was a problem hiding this comment.
What is apiIdentifier, can you figure out a better name ?
There was a problem hiding this comment.
Maybe I should use the keywords physical and logical, too. Thanks for input.
| cache.invalidate(ident, srcType, relType); | ||
| } | ||
| cache.invalidate(dstIdentifier, dstType, relType); | ||
| } |
There was a problem hiding this comment.
Why do we need to add this?
There was a problem hiding this comment.
I will set owners for multiple metadata objects. So I need to insert multiple relations. Although we already have the update multiple relations, it can't cover the cases when we need to insert multiple records.
I got your point. |
| for (NameIdentifier ident : srcIdentifiers) { | ||
| cache.invalidate(ident, srcType, relType); | ||
| } | ||
| cache.invalidate(dstIdentifier, dstType, relType); |
There was a problem hiding this comment.
Is it more proper to invalidate the cache first and then operate the database?
There was a problem hiding this comment.
I take a look. Other relation ops follows this pattern, too. Do we need to modify them together?
There was a problem hiding this comment.
@diqiu50
Please take a look, and I recall that you have modified this point.
|
|
||
| @Override | ||
| public SchemaPO getPOByFullName(SchemaMetaMapper mapper, NameIdentifier identifier) { | ||
| Namespace namespace = identifier.namespace(); |
There was a problem hiding this comment.
There will be performance degradation if you are using the full qualified name when cache is enabled, we can utilize the entity ID to retrieve entities.
There was a problem hiding this comment.
I have added POStorageReadRouting to handle this issue.
| "listPOs by namespace and names is not supported by " + getClass().getSimpleName()); | ||
| } | ||
|
|
||
| public List<PO> listPOs(Mapper mapper, List<Long> uuids) { |
There was a problem hiding this comment.
Yes, I should use entityIds. It may be readable.
|
|
||
| static final Map<MetadataObject.Type, BasePOStorageOps<?, ?>> TYPE_TO_STORAGE_OPS_MAP = | ||
| ImmutableMap.<MetadataObject.Type, BasePOStorageOps<?, ?>>builder() | ||
| .put(MetadataObject.Type.SCHEMA, new SchemaPOStorageOps()) |
There was a problem hiding this comment.
The StorageOps only cover the schema related POs.
SchemaMetaService.insertSchema previously split the entity name on the physical separator (HierarchicalSchemaUtil.physicalSeparator()), so the ancestor-row creation path was only reachable when the caller already handed a storage-form name. Production callers (SchemaOperationDispatcher .importSchema and ManagedSchemaOperations.createSchema) pass the external API form, so the leaf was inserted with no ancestor rows. Split on the logical separator and let HierarchicalConventionPOStorageOps .batchInsertPOs apply its write rewriter to translate each PO to storage form before SQL execution. For the direct batchSelectSchemaByIdentifier lookup that bypasses the rewriter, convert logical -> physical at the call site. Update TestSchemaMetaService accordingly so both hierarchical tests build SchemaEntities with logical-form names. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
What changes were proposed in this pull request?
Add abstract class BasePOStorageOps to let all the PO logic to one class
Add SchemaPOStorageOps,TablePOStorageOps,FunctionPOStorageOps,ViewPOStorageOps.
Add Hierachical convention class
Why are the changes needed?
Fix: #11103
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Added tests.