Skip to content

fix: support parsing Doris REFRESH MATERIALIZED VIEW, CLEAN PROFILE, and CREATE EXTERNAL RESOURCE#38557

Closed
daguimu wants to merge 2 commits into
apache:masterfrom
daguimu:fix/doris-sql-parsing-31457
Closed

fix: support parsing Doris REFRESH MATERIALIZED VIEW, CLEAN PROFILE, and CREATE EXTERNAL RESOURCE#38557
daguimu wants to merge 2 commits into
apache:masterfrom
daguimu:fix/doris-sql-parsing-31457

Conversation

@daguimu

@daguimu daguimu commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Problem

ShardingSphere's Doris SQL parser does not support parsing the following SQL statements:

  • REFRESH MATERIALIZED VIEW mv1 AUTO
  • REFRESH MATERIALIZED VIEW mv1 COMPLETE
  • REFRESH MATERIALIZED VIEW mv1 PARTITIONS(p1, p2)
  • CLEAN PROFILE
  • CREATE EXTERNAL RESOURCE "name" PROPERTIES (...)

These are valid Doris SQL statements documented in the Doris official documentation.

Root Cause

The ANTLR grammar rules for these Doris-specific SQL statements were not defined in the parser.

Fix

Added complete parsing support including:

Grammar Changes

  • DorisKeyword.g4: Added EXTERNAL keyword
  • DDLStatement.g4: Added dorisRefreshMaterializedView rule with partitionSpec, COMPLETE, and AUTO variants
  • DALStatement.g4: Added cleanProfile and createExternalResource rules
  • DorisStatement.g4: Registered new rules in the execute block

Visitor & Statement Classes

  • DorisRefreshMaterializedViewStatement (DDL)
  • DorisCleanProfileStatement (DAL)
  • DorisCreateExternalResourceStatement (DAL)
  • Corresponding visitor methods in DorisDALStatementVisitor and DorisDDLStatementVisitor
  • SQLVisitorRule entries for all new statements

Tests Added

  • doris_refresh_materialized_view_auto - Tests REFRESH MATERIALIZED VIEW mv1 AUTO
  • doris_refresh_materialized_view_complete - Tests REFRESH MATERIALIZED VIEW mv1 COMPLETE
  • doris_refresh_materialized_view_partitions - Tests REFRESH MATERIALIZED VIEW mv1 PARTITIONS(p1, p2)
  • clean_profile - Tests CLEAN PROFILE
  • create_external_resource_spark - Tests CREATE EXTERNAL RESOURCE with Spark properties
  • create_external_resource_odbc - Tests CREATE EXTERNAL RESOURCE with ODBC properties
  • Test case classes, assert classes, and XML test definitions for all cases

Impact

Only affects Doris SQL parsing. No changes to other database dialects or core functionality.

Fixes #31457

daguimu added 2 commits March 25, 2026 21:02
@daguimu
fix: support parsing Doris REFRESH MATERIALIZED VIEW, CLEAN PROFILE, …
2d694b6 
…and CREATE EXTERNAL RESOURCE SQL

Add ANTLR grammar rules, visitor implementations, statement classes, and test
cases for the following Doris SQL statements:
- REFRESH MATERIALIZED VIEW with AUTO, COMPLETE, and PARTITIONS variants
- CLEAN PROFILE keyword
- CREATE EXTERNAL RESOURCE with PROPERTIES clause

Fixes apache#31457
@daguimu
Apply Spotless formatting for Doris SQL parsing support
3a81be9
@daguimu

daguimu commented Apr 23, 2026

Copy link
Copy Markdown
Contributor Author

Hi, I have just addressed the Spotless violations in a follow-up commit; CI should now pass. Would appreciate a review when someone has time. Thanks!

terrymanu
terrymanu requested changes May 28, 2026
View reviewed changes

@terrymanu terrymanu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please resolve conflicts and fix checkstyle first

@terrymanu

terrymanu commented Jun 18, 2026

Copy link
Copy Markdown
Member

Hi @daguimu, thanks again for working on this Doris parser issue.

I am going to close this PR because it has had no response or update for a long time after the change request to resolve conflicts and fix Checkstyle, and the current approach also needs to be revisited.

For #31457, CLEAN ALL PROFILE is already supported in the current codebase, and Doris official documentation only describes CLEAN ALL PROFILE. This PR adds support for CLEAN PROFILE, which does not match the linked issue or the official Doris syntax. The remaining Doris parser cases, such as REFRESH MATERIALIZED VIEW ... and CREATE EXTERNAL RESOURCE ..., should be handled in a fresh, rebased PR with verified official syntax and focused parser IT assertions.

The linked issue remains open, and you are very welcome to submit a new PR later with a smaller and clarified scope.

Thanks again for your contribution.

@terrymanu terrymanu closed this Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@terrymanu terrymanu terrymanu requested changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Support parsing Doris sql

2 participants

@daguimu @terrymanu