Skip to content

Commit 31f1b5f

Browse files
committed
merged main branch
Signed-off-by: Asif Bashar <asif.bashar@gmail.com>
2 parents 3b17563 + 82adaac commit 31f1b5f

801 files changed

Lines changed: 31613 additions & 20925 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.coderabbit.yaml

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
2+
3+
# CodeRabbit Configuration for OpenSearch SQL Project
4+
# This configuration uses .rules/REVIEW_GUIDELINES.md for code review standards
5+
6+
language: "en-US"
7+
early_access: false
8+
9+
reviews:
10+
profile: "chill"
11+
request_changes_workflow: false
12+
high_level_summary: true
13+
high_level_summary_in_walkthrough: true
14+
poem: false # Keep reviews professional and concise
15+
review_status: true
16+
collapse_walkthrough: true
17+
18+
auto_review:
19+
enabled: true
20+
auto_incremental_review: true
21+
drafts: false # Don't review draft PRs
22+
ignore_title_keywords:
23+
- "WIP"
24+
- "DO NOT MERGE"
25+
- "DRAFT"
26+
27+
# Path-specific review instructions
28+
path_instructions:
29+
- path: "**/*.java"
30+
instructions: |
31+
- Verify Java naming conventions (PascalCase for classes, camelCase for methods/variables)
32+
- Check for proper JavaDoc on public classes and methods
33+
- Flag redundant comments that restate obvious code
34+
- Ensure methods are under 20 lines with single responsibility
35+
- Verify proper error handling with specific exception types
36+
- Check for Optional<T> usage instead of null returns
37+
- Validate proper use of try-with-resources for resource management
38+
39+
- path: "**/test/**/*.java"
40+
instructions: |
41+
- Verify test coverage for new business logic
42+
- Check test naming follows conventions (*Test.java for unit, *IT.java for integration)
43+
- Ensure tests are independent and don't rely on execution order
44+
- Validate meaningful test data that reflects real-world scenarios
45+
- Check for proper cleanup of test resources
46+
47+
- path: "integ-test/**/*IT.java"
48+
instructions: |
49+
- Verify integration tests are in correct module (integ-test/)
50+
- Check tests can be run with ./gradlew :integ-test:integTest
51+
- Ensure proper test data setup and teardown
52+
- Validate end-to-end scenario coverage
53+
54+
- path: "**/ppl/**/*.java"
55+
instructions: |
56+
- For PPL parser changes, verify grammar tests with positive/negative cases
57+
- Check AST generation for new syntax
58+
- Ensure corresponding AST builder classes are updated
59+
- Validate edge cases and boundary conditions
60+
61+
- path: "**/calcite/**/*.java"
62+
instructions: |
63+
- Follow existing patterns in CalciteRelNodeVisitor and CalciteRexNodeVisitor
64+
- Verify SQL generation and optimization paths
65+
- Document any Calcite-specific workarounds
66+
- Test compatibility with Calcite version constraints
67+
68+
chat:
69+
auto_reply: false # require explicit tagging
70+
art: false # disable ASCII / Emoji art
71+
72+
# Knowledge base configuration
73+
knowledge_base:
74+
# Don't opt out - use knowledge base features
75+
opt_out: false
76+
77+
# Code guidelines - reference our custom review guidelines
78+
code_guidelines:
79+
enabled: true
80+
filePatterns:
81+
# Reference our custom review guidelines
82+
- ".rules/REVIEW_GUIDELINES.md"
83+
84+
# Enable web search for additional context
85+
web_search:
86+
enabled: true
87+
88+
# Use repository-specific learnings for this project
89+
learnings:
90+
scope: "local"
91+
92+
# Use repository-specific issues
93+
issues:
94+
scope: "local"
95+
96+
# Use repository-specific pull requests for context
97+
pull_requests:
98+
scope: "local"

.github/workflows/integ-tests-with-security.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ jobs:
2222
strategy:
2323
fail-fast: false
2424
matrix:
25-
java: [21, 24]
25+
java: [21, 25]
2626
runs-on: ubuntu-latest
2727
container:
2828
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution
@@ -63,7 +63,7 @@ jobs:
6363
fail-fast: false
6464
matrix:
6565
os: [ windows-latest, macos-14 ]
66-
java: [21, 24]
66+
java: [21, 25]
6767

6868
runs-on: ${{ matrix.os }}
6969

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
name: SQL CLI Integration Test
2+
3+
# This workflow tests sql-cli against the current SQL changes
4+
# to catch breaking changes before they're published
5+
6+
on:
7+
pull_request:
8+
paths:
9+
- '**/*.java'
10+
- '**/*.g4'
11+
- '!sql-jdbc/**'
12+
- '**gradle*'
13+
- '**lombok*'
14+
- 'integ-test/**'
15+
- '**/*.jar'
16+
- '**/*.pom'
17+
- '.github/workflows/sql-cli-integration-test.yml'
18+
push:
19+
branches:
20+
- main
21+
- '[0-9]+.[0-9]+'
22+
- '[0-9]+.x'
23+
paths:
24+
- '**/*.java'
25+
- '**/*.g4'
26+
- '!sql-jdbc/**'
27+
- '**gradle*'
28+
- '**lombok*'
29+
- 'integ-test/**'
30+
- '**/*.jar'
31+
- '**/*.pom'
32+
- '.github/workflows/sql-cli-integration-test.yml'
33+
workflow_dispatch:
34+
35+
jobs:
36+
test-sql-cli-integration:
37+
runs-on: ubuntu-latest
38+
strategy:
39+
fail-fast: false
40+
matrix:
41+
java: [21]
42+
43+
steps:
44+
- name: Checkout SQL CLI repository (latest main)
45+
uses: actions/checkout@v4
46+
with:
47+
repository: opensearch-project/sql-cli
48+
path: sql-cli
49+
ref: main
50+
51+
- name: Make a directory for the SQL repo
52+
working-directory: sql-cli
53+
run: mkdir remote
54+
55+
- name: Checkout SQL repository (current changes)
56+
uses: actions/checkout@v4
57+
with:
58+
path: sql-cli/remote/sql
59+
60+
- name: Set up JDK ${{ matrix.java }}
61+
uses: actions/setup-java@v4
62+
with:
63+
distribution: 'temurin'
64+
java-version: ${{ matrix.java }}
65+
66+
- name: Build and publish SQL modules to Maven Local
67+
working-directory: sql-cli/remote/sql
68+
run: |
69+
echo "Building SQL modules from current branch..."
70+
./gradlew publishToMavenLocal -x test -x integTest
71+
echo "SQL modules published to Maven Local"
72+
73+
- name: Run SQL CLI tests with local SQL modules
74+
working-directory: sql-cli
75+
run: |
76+
echo "Running SQL CLI tests against local SQL modules..."
77+
./gradlew test -PuseLocalSql=true -PskipSqlRepoPull=true
78+
79+
- name: Upload SQL CLI test reports
80+
if: always()
81+
uses: actions/upload-artifact@v4
82+
continue-on-error: true
83+
with:
84+
name: sql-cli-test-reports-java-${{ matrix.java }}
85+
path: |
86+
sql-cli/build/reports/**
87+
sql-cli/build/test-results/**
88+
89+
- name: Test Summary
90+
if: always()
91+
run: |
92+
echo "## SQL CLI Integration Test Results" >> $GITHUB_STEP_SUMMARY
93+
echo "" >> $GITHUB_STEP_SUMMARY
94+
echo "Tested SQL CLI against SQL changes from: \`${{ github.ref }}\`" >> $GITHUB_STEP_SUMMARY
95+
echo "SQL CLI version: main branch (latest)" >> $GITHUB_STEP_SUMMARY
96+
echo "Java version: ${{ matrix.java }}" >> $GITHUB_STEP_SUMMARY

.github/workflows/sql-pitest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
needs: Get-CI-Image-Tag
2121
strategy:
2222
matrix:
23-
java: [21, 24]
23+
java: [21, 25]
2424
runs-on: ubuntu-latest
2525
container:
2626
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution

.github/workflows/sql-test-and-build-workflow.yml

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
strategy:
3030
fail-fast: false
3131
matrix:
32-
java: [21, 24]
32+
java: [21, 25]
3333
test-type: ['unit', 'integration', 'doc']
3434
runs-on: ubuntu-latest
3535
container:
@@ -52,9 +52,9 @@ jobs:
5252
run: |
5353
chown -R 1000:1000 `pwd`
5454
if [ "${{ matrix.test-type }}" = "unit" ]; then
55-
su `id -un 1000` -c "./gradlew --continue build -x integTest -x doctest"
55+
su `id -un 1000` -c "./gradlew --continue build -x integTest -x yamlRestTest -x doctest"
5656
elif [ "${{ matrix.test-type }}" = "integration" ]; then
57-
su `id -un 1000` -c "./gradlew --continue integTest"
57+
su `id -un 1000` -c "./gradlew --continue integTest yamlRestTest"
5858
else
5959
su `id -un 1000` -c "./gradlew --continue doctest"
6060
fi
@@ -106,16 +106,16 @@ jobs:
106106
matrix:
107107
entry:
108108
- { os: windows-latest, java: 21, os_build_args: -PbuildPlatform=windows }
109-
- { os: windows-latest, java: 24, os_build_args: -PbuildPlatform=windows }
109+
- { os: windows-latest, java: 25, os_build_args: -PbuildPlatform=windows }
110110
- { os: macos-14, java: 21, os_build_args: '' }
111-
- { os: macos-14, java: 24, os_build_args: '' }
111+
- { os: macos-14, java: 25, os_build_args: '' }
112112
test-type: ['unit', 'integration', 'doc']
113113
exclude:
114114
# Exclude doctest for Windows
115115
- test-type: doc
116116
entry: { os: windows-latest, java: 21, os_build_args: -PbuildPlatform=windows }
117117
- test-type: doc
118-
entry: { os: windows-latest, java: 24, os_build_args: -PbuildPlatform=windows }
118+
entry: { os: windows-latest, java: 25, os_build_args: -PbuildPlatform=windows }
119119
runs-on: ${{ matrix.entry.os }}
120120

121121
steps:
@@ -130,9 +130,9 @@ jobs:
130130
- name: Build and Test
131131
run: |
132132
if [ "${{ matrix.test-type }}" = "unit" ]; then
133-
./gradlew --continue build -x integTest -x doctest ${{ matrix.entry.os_build_args }}
133+
./gradlew --continue build -x integTest -x yamlRestTest -x doctest ${{ matrix.entry.os_build_args }}
134134
elif [ "${{ matrix.test-type }}" = "integration" ]; then
135-
./gradlew --continue integTest ${{ matrix.entry.os_build_args }}
135+
./gradlew --continue integTest yamlRestTest ${{ matrix.entry.os_build_args }}
136136
else
137137
./gradlew --continue doctest ${{ matrix.entry.os_build_args }}
138138
fi
@@ -184,7 +184,7 @@ jobs:
184184
runs-on: ubuntu-latest
185185
strategy:
186186
matrix:
187-
java: [21, 24]
187+
java: [21, 25]
188188
container:
189189
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
190190
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}
@@ -230,7 +230,7 @@ jobs:
230230
runs-on: ubuntu-latest
231231
strategy:
232232
matrix:
233-
java: [21, 24]
233+
java: [21, 25]
234234
container:
235235
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
236236
options: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-start-options }}

.github/workflows/sql-test-workflow.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
needs: Get-CI-Image-Tag
2121
strategy:
2222
matrix:
23-
java: [21, 24]
23+
java: [21, 25]
2424
runs-on: ubuntu-latest
2525
container:
2626
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution

.rules/REVIEW_GUIDELINES.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Code Review Guidelines for OpenSearch SQL
2+
3+
This document provides guidelines for code reviews in the OpenSearch SQL project. These guidelines are used by CodeRabbit AI for automated code reviews and serve as a reference for human reviewers.
4+
5+
## Core Review Principles
6+
7+
### Code Quality
8+
- **Simplicity First**: Prefer simpler solutions unless there's significant functional or performance degradation
9+
- **Self-Documenting Code**: Code should be clear through naming and structure, not comments
10+
- **No Redundant Comments**: Avoid comments that merely restate what the code does
11+
- **Concise Implementation**: Keep code, docs, and notes short and focused on essentials
12+
13+
### Java Standards
14+
- **Naming Conventions**:
15+
- Classes: `PascalCase` (e.g., `QueryExecutor`)
16+
- Methods/Variables: `camelCase` (e.g., `executeQuery`)
17+
- Constants: `UPPER_SNAKE_CASE` (e.g., `MAX_RETRY_COUNT`)
18+
- **Method Size**: Keep methods under 20 lines with single responsibility
19+
- **JavaDoc Required**: All public classes and methods must have proper JavaDoc
20+
- **Error Handling**: Use specific exception types with meaningful messages
21+
- **Null Safety**: Prefer `Optional<T>` for nullable returns
22+
23+
### Testing Requirements
24+
- **Test Coverage**: All new business logic requires unit tests
25+
- **Integration Tests**: End-to-end scenarios need integration tests in `integ-test/` module
26+
- **Test Execution**: Verify changes with `./gradlew :integ-test:integTest`
27+
- **No Failing Tests**: All tests must pass before merge; fix or ask for guidance if blocked
28+
29+
### Code Organization
30+
- **Single Responsibility**: Each class should have one clear purpose
31+
- **Package Structure**: Follow existing module organization (core, ppl, sql, opensearch)
32+
- **Separation of Concerns**: Keep parsing, execution, and storage logic separate
33+
- **Composition Over Inheritance**: Prefer composition for code reuse
34+
35+
### Performance & Security
36+
- **Efficient Loops**: Avoid unnecessary object creation in loops
37+
- **String Handling**: Use `StringBuilder` for concatenation in loops
38+
- **Input Validation**: Validate all user inputs, especially queries
39+
- **Logging Safety**: Sanitize data before logging to prevent injection
40+
- **Resource Management**: Use try-with-resources for proper cleanup
41+
42+
## Review Focus Areas
43+
44+
### What to Check
45+
1. **Code Clarity**: Is the code self-explanatory?
46+
2. **Test Coverage**: Are there adequate tests?
47+
3. **Error Handling**: Are errors handled appropriately?
48+
4. **Documentation**: Is JavaDoc complete and accurate?
49+
5. **Performance**: Are there obvious performance issues?
50+
6. **Security**: Are inputs validated and sanitized?
51+
52+
### What to Flag
53+
- Redundant or obvious comments
54+
- Methods longer than 20 lines
55+
- Missing JavaDoc on public APIs
56+
- Generic exception handling
57+
- Unused imports or dead code
58+
- Hard-coded values that should be constants
59+
- Missing or inadequate test coverage
60+
61+
### What to Encourage
62+
- Clear, descriptive naming
63+
- Proper use of Java idioms
64+
- Comprehensive test coverage
65+
- Meaningful error messages
66+
- Efficient algorithms and data structures
67+
- Security-conscious coding practices
68+
69+
## Project-Specific Guidelines
70+
71+
### OpenSearch SQL Context
72+
- **JDK 21**: Required for development
73+
- **Java 11 Compatibility**: Maintain when possible for OpenSearch 2.x
74+
- **Module Structure**: Respect existing module boundaries
75+
- **Integration Tests**: Use `./gradlew :integ-test:integTest` for testing
76+
- **Test Naming**: `*IT.java` for integration tests, `*Test.java` for unit tests
77+
78+
### PPL Parser Changes
79+
- Test new grammar rules with positive and negative cases
80+
- Verify AST generation for new syntax
81+
- Include edge cases and boundary conditions
82+
- Update corresponding AST builder classes
83+
84+
### Calcite Integration
85+
- If the PR is for PPL command, refer docs/dev/ppl-commands.md and verify the PR satisfy the checklist.
86+
- Follow existing patterns in `CalciteRelNodeVisitor` and `CalciteRexNodeVisitor`
87+
- Test SQL generation and optimization paths
88+
- Document Calcite-specific workarounds

0 commit comments

Comments
 (0)