Skip to content

[#11103] feat(core): Add the hierarchy convention layer#11074

Open
roryqi wants to merge 37 commits into
apache:mainfrom
qqqttt123:split/pr-nested-ns-backend
Open

[#11103] feat(core): Add the hierarchy convention layer#11074
roryqi wants to merge 37 commits into
apache:mainfrom
qqqttt123:split/pr-nested-ns-backend

Conversation

@roryqi
Copy link
Copy Markdown
Contributor

@roryqi roryqi commented May 13, 2026

What changes were proposed in this pull request?

Add abstract class BasePOStorageOps to let all the PO logic to one class

Add SchemaPOStorageOps,TablePOStorageOps,FunctionPOStorageOps,ViewPOStorageOps.

Add Hierachical convention class

Why are the changes needed?

Fix: #11103

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Added tests.

- RelationalSchemaNamingBridge and JDBCBackend entity/relation conversions
- SupportsRelationOperations.batchInsertRelations; RelationalEntityStore cache invalidation
- RelationalBackend; backend-focused tests

Co-authored-by: Cursor <cursoragent@cursor.com>
@roryqi roryqi marked this pull request as draft May 13, 2026 08:14
roryqi and others added 7 commits May 13, 2026 08:37
- Replace Preconditions.checkNotNull with Objects.requireNonNull in
  JDBCBackend.batchInsertRelations for consistency with the interface
- Add @SuppressWarnings explanatory comments in unchecked-cast methods
- Document FILESET/TOPIC/MODEL/MODEL_VERSION passthrough in default
  switch branches of nameIdentifierForStorage/nameIdentifierForApi
- Expand Javadoc on embeddedNamespaceForStorage/Api to warn about the
  index-2 layout assumption
- Guard statisticEntityForApi/Storage with Preconditions.checkArgument
  to reject non-TableStatisticEntity subtypes at the call site
- Add tests: roleEntityForStorage/Api round-trips, batchInsertRelations
  cache invalidation in RelationalEntityStore, and non-OWNER_REL
  rejection in JDBCBackend.batchInsertRelations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extract mapRoleSecurableObjects helper to eliminate the duplicate
  builder block shared by roleEntityForStorage and roleEntityForApi
- Extract statisticEntityWithNamespace helper to consolidate the type
  guard and builder shared by statisticEntityForApi and
  statisticEntityForStorage
- Replace repeated Lists.newArrayList(Privileges.UseSchema.allow()) and
  SecurableObjects.ofCatalog("catalog", ...) constructions in tests with
  private static final constants USE_SCHEMA_PRIVS and CATALOG_OBJ
- Remove inline WHAT-comments that restate what the assertions already
  express

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…yStore

batchDelete and batchPut were the only write paths that bypassed the
cache — callers that relied on them would silently observe stale reads.
Apply the same pattern as their single-item counterparts:
- batchDelete: invalidate each (ident, entityType) pair after backend
- batchPut: put each entity into cache after backend

Add unit tests covering the backend-then-cache ordering for both methods.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Identifier-based approach

Use the existing nameIdentifierForStorage/Api infrastructure via a sentinel
metalake prefix instead of bespoke string-splitting logic. Removes the
package-private convertMetadataObjectDottedFullName and convertSchemaSegmentAt
helpers; updates securableObjectForStorage/Api and
genericEntityMetadataFullNameForApi accordingly. Tests are replaced with
equivalent coverage through the public securableObject API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dataObject entity types

Entity types without a MetadataObject.Type equivalent (e.g. TABLE_STATISTIC,
MODEL_VERSION) must not be schema-converted. Re-add the MetadataObject.Type.valueOf
guard that was dropped in the previous refactor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@roryqi roryqi self-assigned this May 13, 2026
@roryqi roryqi requested a review from jerryshao May 13, 2026 12:27
@github-actions
Copy link
Copy Markdown

Code Coverage Report

Overall Project 66.03% +0.05% 🟢
Files changed 72.79% 🟢

Module Coverage
aliyun 1.72% 🔴
api 47.13% 🟢
authorization-common 85.96% 🟢
aws 1.08% 🔴
azure 2.47% 🔴
catalog-common 10.2% 🔴
catalog-fileset 80.02% 🟢
catalog-glue 83.41% 🟢
catalog-hive 81.83% 🟢
catalog-jdbc-clickhouse 79.18% 🟢
catalog-jdbc-common 43.93% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.05% 🟢
catalog-jdbc-starrocks 78.27% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 45.14% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 87.08% 🟢
catalog-lakehouse-paimon 76.85% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.96% 🟢
common 50.0% 🟢
core 82.14% -0.22% 🟢
filesystem-hadoop3 76.97% 🟢
flink 0.0% 🔴
flink-common 43.17% 🟢
flink-runtime 0.0% 🔴
gcp 14.12% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 46.83% 🟢
iceberg-common 55.46% 🟢
iceberg-rest-server 69.61% 🟢
idp-basic 94.68% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 19.95% 🔴
lance-rest-server 62.78% 🟢
lineage 53.02% 🟢
optimizer 82.95% 🟢
optimizer-api 21.95% 🔴
server 85.93% 🟢
server-common 71.21% 🟢
spark 32.79% 🔴
spark-common 39.09% 🔴
trino-connector 35.14% 🔴
Files
Module File Coverage
core JDBCBackend.java 78.66% 🟢
SupportsRelationOperations.java 61.54% 🟢
RelationalEntityStore.java 59.76% 🔴

roryqi and others added 6 commits May 14, 2026 16:52
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@roryqi roryqi removed the request for review from jerryshao May 14, 2026 10:39
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
roryqi and others added 4 commits May 14, 2026 10:59
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@roryqi roryqi changed the title feat(core): JDBC backend and entity store for nested namespace naming feat(core): Add the hierarchy convention layer May 14, 2026
@roryqi roryqi changed the title feat(core): Add the hierarchy convention layer [#11103] feat(core): Add the hierarchy convention layer May 14, 2026
@roryqi roryqi marked this pull request as ready for review May 14, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Subtask] Add the hierarchy convention layer

1 participant