fix(validation): reject reserved FQN characters in entity and column names#27521
fix(validation): reject reserved FQN characters in entity and column names#27521jaya6400 wants to merge 21 commits intoopen-metadata:mainfrom
Conversation
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
jaya6400
changed the title
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
jaya6400
force-pushed
the
fix/entity-name-validation-special-chars
branch
from
88f5ad3 to
596f26b
Compare
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Local pattern validation test: Bash command used: test-regex.txt
|
|
@shah-harshit @ShaileshParmar11 @harshach @shrabantipaul-collate @pmbrull requesting to add |
harshach
added
the
safe to test
jaya6400
force-pushed
the
fix/entity-name-validation-special-chars
branch
from
596f26b to
1fa38ef
Compare
|
The CI failures appear to be infrastructure-related (timeout during test environment setup), unrelated to the schema change. Could a maintainer please re-run the failed checks? |
|
Hi @jaya6400, Thanks for your contribution. I see that you have modified the regex pattern in a common place and in table.json to cover columns. Similar to this, we have many other asset children like tasks for pipelines, schema fields for topics, etc. Can you please verify if those are covered as well? |
There was a problem hiding this comment.
Pull request overview
This PR tightens OpenMetadata JSON Schema validation to prevent reserved characters and control characters from being used in entity names and table column names, addressing FQN construction/parsing issues reported in #23268.
Changes:
- Updated
entityNameandtestCaseEntityNameregex patterns inbasic.json. - Updated
columnNameregex pattern intable.jsonto reject reserved characters/control chars.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| openmetadata-spec/src/main/resources/json/schema/type/basic.json | Tightens entity name regex to block reserved FQN/entity-link breaking characters. |
| openmetadata-spec/src/main/resources/json/schema/entity/data/table.json | Tightens column name regex to block reserved FQN/entity-link breaking characters. |
🔴 Playwright Results — 58 failure(s), 7 flaky✅ 3482 passed · ❌ 58 failed · 🟡 7 flaky · ⏭️ 539 skipped
Genuine Failures (failed on all attempts)❌
|
|
…names Fixes open-metadata#23268 Updated regex patterns in JSON schema definitions to block reserved FQN separator characters (::, >, ") and newlines in entity names. Files changed: - openmetadata-spec/.../type/basic.json: entityName, testCaseEntityName - openmetadata-spec/.../entity/data/table.json: columnName
Updated pattern to use \x00-\x1f range to block all control characters including \r, \n, \t, \0 in addition to reserved FQN characters. This also restores the \r restriction that was present in the original dot metacharacter but lost in the character class rewrite.
…hema fields and UI - Added < and | to blocked characters (consistent with entityLink pattern) - Updated searchIndexFieldName pattern in searchIndex.json - Added pattern validation to tableConstraint columns and partitionColumnDetails in table.json - Updated UI ENTITY_NAME_REGEX to match backend validation - Added unit tests for new blocked characters in regex.constants.test.ts Addresses review feedback on open-metadata#27521
Addresses inconsistency flagged in code review - columnName definition was missing < and | from blocked characters unlike other patterns.
…e in ENTITY_NAME_REGEX
…erved characters Added Java integration tests to verify that entity names containing reserved FQN characters (::, >, <, ", |) and control characters are rejected by the backend validation. Files changed: - TableResourceIT.java: added column name validation tests - TopicResourceIT.java: added schema field name validation tests - PipelineResourceIT.java: added task name validation tests - SearchIndexResourceIT.java: added search index field name validation tests - TestSuiteBootstrap.java: updated bootstrap for test suite - pipeline.json: updated pattern for task names - schema.json: updated pattern for schema field names
…less formatting - Added minLength: 1 to pipeline task name to prevent empty strings - Applied mvn spotless:apply formatting fixes
- Add integration test in PipelineResourceIT.java to validate that creating a pipeline with empty task name returns 4xx error - Add Playwright E2E test in PipelineValidation.spec.ts to validate empty task name rejection via API - Tests verify the existing JSON schema validation (minLength: 1 on task name field) works correctly
- Block :: (reserved FQN separator) in JSON schema and frontend regex - Keep single colon allowed - Restore :: rejection unit tests
Code Review ✅ Approved 8 resolved / 8 findingsEnforces reserved FQN character restrictions for entity and column names, while addressing regex inconsistencies, license headers, and missing test cases. Ensure constraint and partition column names are updated to reject empty strings. ✅ 8 resolved✅ Edge Case: Pattern does not block \r or other control characters
✅ Bug: Inconsistent regex: columnName definition missing
|
| Compact |
|
Was this helpful? React with 👍 / 👎 | Gitar
|
Describe your changes:
Fixes #23268
I updated the regex patterns in JSON schema definitions to block reserved
FQN separator characters (
::,>,") and newlines (\n) from beingused in entity and column names.
The backend was accepting entity names with special characters like double
quotes and newlines, which caused issues in FQN construction since these
are reserved separator characters in OpenMetadata's FQN system.
Changes made:
openmetadata-spec/src/main/resources/json/schema/type/basic.json— updated pattern forentityNameandtestCaseEntityNameopenmetadata-spec/src/main/resources/json/schema/entity/data/table.json— updated pattern forcolumnNamePattern changed from
^((?!::).)*$to^((?!::)[^><\"|\\x00-\\x1f])*$Unit Test: #27521 (comment)
Type of change:
Checklist:
Fixes #23268: <short explanation>Summary by Gitar
PipelineValidation.spec.tsto verify entity name restriction behavior via Playwright E2E tests.TestSuiteBootstrap.javato enforce UTC timezone and improve robustness ofprojectRootpath resolution.PipelineResourceIT.javato assert 400 errors when creating pipelines with invalid task names.regex.constants.test.tsto cover additional reserved symbols and control characters.pipeline.jsontask names andschema.jsonfield names to ensure consistency across entity definitions.This will update automatically on new commits.