AGENTS.md - StarRocks Frontend (FE)

Frontend-specific guidelines for AI coding agents. The FE is written in Java and handles SQL parsing, query optimization, and metadata management.

Architecture Overview

Start with handbook/index.md for repo-wide routing and handbook/domains/frontend.md for FE commands, guardrails, and open harness gaps.

The Frontend is responsible for:

SQL Parsing: ANTLR-based parser for StarRocks SQL dialect
Query Optimization: Cost-based optimizer with rule-based transformations
Metadata Management: Catalog, database, table, and statistics management
Cluster Coordination: FE leader election, BE management, load balancing

Module Structure

fe/
├── fe-core/           # Core FE logic (optimizer, catalog, execution coordination)
├── fe-parser/         # SQL parser and AST definitions
├── fe-grammar/        # ANTLR grammar files (.g4)
├── fe-type/           # Type system definitions
├── fe-spi/            # Service Provider Interfaces
├── fe-utils/          # General-purpose utilities (no business logic)
├── fe-testing/        # Shared test utilities and mocks
├── connector/         # Data source connector implementations
├── plugin/            # Non-connector plugins (auth, audit, etc.)
└── spark-dpp/         # Spark DPP application

Key Packages (fe-core)

com.starrocks/
├── sql/
│   ├── parser/       # SQL parsing utilities
│   ├── ast/          # Abstract Syntax Tree nodes
│   ├── analyzer/     # Semantic analysis
│   └── optimizer/    # Query optimization
├── planner/          # Physical plan generation
├── catalog/          # Metadata (databases, tables, partitions)
├── qe/               # Query execution coordination
├── privilege/        # Authentication and authorization
├── scheduler/        # Task scheduling
├── load/             # Data loading
└── backup/           # Backup and restore

Build Commands

# Build FE only
./build.sh --fe

# Clean and build
./build.sh --fe --clean

# Build FE using Maven directly
cd fe && mvn package -DskipTests

# Build specific module
cd fe && mvn package -pl fe-core -am -DskipTests

Maven Commands

cd fe

# Compile
mvn compile

# Run tests
mvn test

# Package without tests
mvn package -DskipTests

# Check code style
mvn checkstyle:check

# Generate compile commands
mvn compile -Dmaven.compiler.showCompilationChanges=true

Code Style

Checkstyle

FE uses Checkstyle based on Google Java Style. Config: fe/checkstyle.xml

# Check style
cd fe && mvn checkstyle:check

# Check specific module
cd fe && mvn checkstyle:check -pl fe-core

Formatting Rules

Indent: 4 spaces
Line limit: 130 characters
Braces: Same line for control structures
Blank lines: One blank line between methods

Import Order

Checkstyle enforces this import order:

Third-party packages (com.google, org.apache, etc.)
Java standard packages (java., javax.)
Static imports

// Third-party
import com.google.common.collect.Lists;
import org.apache.logging.log4j.LogManager;

// Java standard
import java.util.ArrayList;
import java.util.List;

// Static imports
import static com.starrocks.common.util.Util.checkNotNull;

Naming Conventions

// Classes: PascalCase
public class QueryPlanner { }

// Interfaces: PascalCase (no "I" prefix)
public interface Catalog { }

// Methods: camelCase
public void analyzeStatement(StatementBase stmt) { }

// Variables: camelCase
private final String tableName;

// Constants: UPPER_SNAKE_CASE
public static final int DEFAULT_TIMEOUT = 300;

// Package names: lowercase
package com.starrocks.sql.optimizer;

Testing

Test Framework

Uses JUnit 5 (Jupiter) with Mockito for mocking.

# Run all FE tests
./run-fe-ut.sh

# Run specific test class
./run-fe-ut.sh --test com.starrocks.sql.plan.TPCHPlanTest

# Run with filter (skip certain tests)
./run-fe-ut.sh --filter "com.starrocks.sql.dump.QueryDumpRegressionTest"

# Parallel execution (default: 4)
./run-fe-ut.sh -j 8

Writing Tests

// File: fe/fe-core/src/test/java/com/starrocks/sql/plan/MyPlanTest.java
package com.starrocks.sql.plan;

import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

public class MyPlanTest extends PlanTestBase {

    @BeforeAll
    public static void beforeClass() throws Exception {
        PlanTestBase.beforeClass();
        // Additional setup
    }

    @Test
    public void testSimpleSelect() throws Exception {
        String sql = "SELECT * FROM t0 WHERE c1 > 10";
        String plan = getFragmentPlan(sql);
        assertContains(plan, "OlapScanNode");
    }

    @Test
    public void testJoinOrder() {
        runFileUnitTest("optimized-plan/join");  // Runs test from file
    }
}

Test Utilities

PlanTestBase: Base class for plan tests
StarRocksAssert: Utility for creating test objects
UtFrameUtils: Test environment setup

Location: fe/fe-testing/src/main/java/

Key Design Principles

AST Immutability

Critical: AST nodes must be immutable after parsing.

// DO NOT add setters to AST nodes
public class SelectStmt extends QueryStatement {
    private final List<SelectListItem> selectList;  // Final

    // NO: public void setSelectList(...) { }

    // YES: Create new node if modification needed
    public SelectStmt withSelectList(List<SelectListItem> newList) {
        return new SelectStmt(newList, this.fromClause, ...);
    }
}

Reasons:

AST should not be used as temporary storage between Parser and Analyzer
Immutability ensures thread safety
Facilitates caching and sharing

SPI Boundaries

Connectors and plugins should only depend on fe-spi, not FE internals.

// Good: Implement SPI interface
public class MyConnector implements Connector {
    // Only uses types from fe-spi
}

// Bad: Direct dependency on internal classes
import com.starrocks.catalog.OlapTable;  // Internal!

Utilities (fe-utils)

Contains only general-purpose helpers
No business logic or FE internals
Can be used by any module

Database and Table Locks

FE metadata access goes through com.starrocks.common.util.concurrent.lock.Locker. There are two lock-acquisition styles; new code should default to the intensive path.

// AVOID as the default choice: full DB-scope lock. WRITE blocks every op
// on every table in the DB; READ blocks every DDL in the DB and every
// concurrent ALTER on any table. Still the right answer for a few cases
// (see rubric below); the @Deprecated marker on Locker.java reflects
// "prefer the intensive path unless you have a specific reason".
locker.lockDatabase(dbId, LockType.READ /* or WRITE */);

// PREFERRED for table-scoped work: IS/IX on the DB + READ/WRITE on
// listed tables. Unrelated tables in the same DB proceed without
// blocking.
locker.lockTableWithIntensiveDbLock(dbId, tableId, LockType.READ);
locker.lockTablesWithIntensiveDbLock(dbId, List.of(tblId1, tblId2), LockType.WRITE);

// Or via try-with-resources:
try (AutoCloseableLock ignore =
        new AutoCloseableLock(new Locker(), dbId, List.of(tableId), LockType.READ)) {
    ...
}

// THIRD VARIANT: plain IS on the DB. Use when you only need the list of
// tables in the DB to be stable (IS blocks CREATE/DROP since they take
// DB WRITE) but want to allow concurrent ALTER on other tables (IS does
// not block IX). Typically paired with a per-table intensive READ for
// each table's internal state - see TabletScheduler.java:2114 for the
// canonical split-design example.
locker.lockDatabase(dbId, LockType.INTENTION_SHARED);
try {
    List<Table> snapshot = db.getTables();
    // ... then per-table:
    //   locker.lockTableWithIntensiveDbLock(dbId, t.getId(), LockType.READ);
} finally {
    locker.unLockDatabase(dbId, LockType.INTENTION_SHARED);
}

Decision rubric:

Critical section does...	Use
Mutates `Database.idToTable` / `nameToTable` / `fullQualifiedName` / `exist` / quotas	Plain `lockDatabase(WRITE)`
Iterates `db.getTables()` AND reads per-table internal state (partitions, schema, replicas) over an unbounded table set	Plain `lockDatabase(READ)` - blocks both DB-WRITE (CREATE/DROP TABLE) and IX (concurrent ALTER on individual tables) for the full walk
Reads / mutates only specific known tables	Intensive path (`lockTable[s]WithIntensiveDbLock`)
Snapshot a bounded table list then per-table work	Outer `lockDatabase(IS)` for the list snapshot (IS still blocks CREATE/DROP since they take DB WRITE) + inner per-table intensive READ for each table's state (see `TabletScheduler.java:2114` for the model)

Key conflict-matrix facts (from LockType.java): IS blocks DB-WRITE (so CREATE/DROP TABLE wait), but IS does NOT block IX (so concurrent ALTER on other tables can proceed). Plain READ blocks both. The choice between IS and READ for iteration is about whether you also need to freeze per-table internal state, not about whether create/drop is prevented.

Anti-pattern: lock acquisition inside try {}. If lockX() throws, the finally tries to release a never-acquired lock, which itself throws (e.g. "lock not held"); that secondary exception then suppresses the original cause, making the failure hard to diagnose. Always acquire before the try:

// BAD
Locker locker = new Locker();
try {
    locker.lockDatabase(dbId, LockType.READ);
    ...
} finally { locker.unLockDatabase(dbId, LockType.READ); }

// GOOD
Locker locker = new Locker();
locker.lockDatabase(dbId, LockType.READ);
try { ... } finally { locker.unLockDatabase(dbId, LockType.READ); }

try (AutoCloseableLock ...) is safe by design - the resource is never assigned if construction throws.

When evaluating an existing WRITE site, trace where the mutation actually lands. A site that looked like NECESSARY (WRITE) on first read (StarMgrMetaSyncer.syncTableColocationInfo) turned out to only need IS+table-READ because the real mutation went through ColocateTableIndex's own writeLock. "The comment says we need WRITE" is not the rubric.

References:

fe/fe-core/.../common/util/concurrent/lock/ - framework + LockType conflict matrix.
System view fe_locks (docs/en/sql-reference/sys/fe_locks.md) - inspect live holders.
Metric HISTO_SLOW_LOCK_WAIT_TIME_MS - track contention.
Test helpers in fe/fe-core/src/test/java/com/starrocks/common/lock/ (LockTestUtils, LockTask, LockThread, LockResult) - use these when writing concurrency tests for a relaxation.

Common Patterns

Status and Error Handling

Use StarRocksException hierarchy for errors:

// Throwing errors
throw new SemanticException("Column not found: " + columnName);
throw new AnalysisException("Invalid syntax");

// Catching and wrapping
try {
    analyze(stmt);
} catch (Exception e) {
    throw new DdlException("Failed to analyze: " + e.getMessage(), e);
}

Visitor Pattern

AST traversal uses the Visitor pattern:

public class MyVisitor extends AstVisitor<Void, Void> {
    @Override
    public Void visitSelectStatement(SelectStmt stmt, Void context) {
        // Process select
        visit(stmt.getFromClause(), context);
        return null;
    }

    @Override
    public Void visitJoinRelation(JoinRelation join, Void context) {
        // Process join
        return null;
    }
}

// Usage
new MyVisitor().visit(ast, null);

Configuration Properties

// Access session variables
ConnectContext ctx = ConnectContext.get();
long timeout = ctx.getSessionVariable().getQueryTimeoutS();

// Access global config
int maxConnections = Config.max_connections;

Extending FE

Adding New SQL Syntax

Update ANTLR grammar in fe/fe-grammar/
Add AST node in fe/fe-parser/src/main/java/com/starrocks/sql/ast/
Update parser visitor to create AST node
Add semantic analysis in analyzer
Add execution logic
Add tests

Adding New Connector

Implement interfaces in fe/fe-spi/
Create connector module in fe/connector/
Register via ServiceLoader or plugin manifest
Add integration tests

Adding New Function

Define function signature in FunctionSet
Implement evaluation in BE (for scalar functions)
Add tests

Key Files to Know

File	Purpose
`fe-core/.../sql/StatementPlanner.java`	Main entry for query planning
`fe-core/.../sql/optimizer/Optimizer.java`	Query optimizer
`fe-core/.../catalog/GlobalStateMgr.java`	Global metadata manager
`fe-core/.../qe/StmtExecutor.java`	Statement execution
`fe-parser/.../sql/ast/QueryStatement.java`	Query AST base
`fe-grammar/StarRocks.g4`	Main SQL grammar

RESTful API Guidelines

When adding HTTP endpoints, follow the standard in docs/en/developers/code-style-guides/restful-api-standard.md:

Format: /api/{version}/{target-object-path}/{action}

GET  /api/v2/catalogs/hive/databases           # List databases
GET  /api/v2/catalogs/hive/databases/tpch      # Get database details
POST /api/v2/catalogs/hive/databases           # Create database
DELETE /api/v2/catalogs/hive/databases/tpch    # Delete database

Performance Tips

Avoid creating objects in hot paths
Use Preconditions for early validation
Prefer ImmutableList/Map for readonly collections
Use appropriate collection types (ArrayList vs LinkedList)
Be careful with synchronized blocks

Optimizer Rule Contract

For optimizer rewrite rules, transform(...) must only return a non-empty list when it actually changes the input OptExpression.

If this transform don't change the input OptExpression, should return the empty list.
Do not return a structurally identical plan, or RewriteTreeTask may treat it as a change and keep rewriting.

Debugging

Logging

Use Log4j2:

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class MyClass {
    private static final Logger LOG = LogManager.getLogger(MyClass.class);

    public void doSomething() {
        LOG.info("Processing started");
        LOG.debug("Details: {}", details);  // Lazy evaluation
        LOG.warn("Slow query: {}", queryId);
    }
}

Remote Debugging

# Start FE with debug port
export JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
./fe/bin/start_fe.sh

Configuration and Metrics Guidelines

Adding/Modifying FE Configuration

FE configuration is defined in fe/fe-core/src/main/java/com/starrocks/common/Config.java.

// Example: Adding a new config parameter
@ConfField(mutable = true, description = "Description of my config")
public static int my_new_config = 100;  // mutable, default 100

@ConfField(description = "Description of static config")
public static long my_static_config = 50;  // immutable

@ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
public static boolean enable_experimental_feature = false;

When adding/modifying FE config, you MUST:

Add the config in Config.java with @ConfField annotation
Update documentation: docs/en/administration/management/FE_configuration.md
Document in the markdown:
- Parameter name
- Default value
- Type and valid range
- Description
- Whether it's mutable (mutable = true)
- Whether restart is required

Adding/Modifying FE Metrics

FE metrics are typically registered via metric registry classes.

// Example: Adding a new metric
public static final LongCounterMetric MY_COUNTER =
    new LongCounterMetric("my_counter", Metric.MetricUnit.REQUESTS, "description");

public static final GaugeMetric<Long> MY_GAUGE =
    new GaugeMetric<>("my_gauge", Metric.MetricUnit.BYTES, "description") {
        @Override
        public Long getValue() {
            return someValue;
        }
    };

When adding/modifying FE metrics, you MUST:

Register metric in appropriate class
Update documentation: docs/en/administration/management/monitoring/metrics.md
Document in the markdown:
- Metric name
- Type (Counter/Gauge/Histogram)
- Unit
- Description of what it measures
- Labels (if any)

PR Guidelines for FE Changes

PR Title

[Type] Brief description of FE change

Example: [Feature] Add support for PIVOT clause in SQL

PR Checklist for FE Changes

Code passes mvn checkstyle:check
Added/updated unit tests in fe/fe-core/src/test/
No new checkstyle violations
AST nodes remain immutable (no setters added)
If acquiring a metadata lock: justified against the rubric in "Database and Table Locks"; default to the intensive path; no new lockDatabase calls without explicit reason
If config changed: Updated docs/en/administration/management/FE_configuration.md
If metrics changed: Updated docs/en/administration/management/monitoring/metrics.md

Before Submitting

cd fe

# Check code style
mvn checkstyle:check

# Run relevant tests
mvn test -Dtest=YourTestClass

# Run all tests (takes longer)
cd .. && ./run-fe-ut.sh

SQL Syntax Changes

If your PR adds/modifies SQL syntax:

Update grammar in fe/fe-grammar/
Add AST nodes in fe/fe-parser/
Add analyzer/optimizer support
Add plan tests in fe/fe-core/src/test/java/com/starrocks/sql/plan/
Add SQL integration tests in test/sql/

For full PR guidelines, see the root AGENTS.md.

License Header

All Java files must have the Apache 2.0 license header:

// Copyright 2021-present StarRocks, Inc. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package com.starrocks.xxx;

Uh oh!

FilesExpand file tree

AGENTS.md

Latest commit

History