Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 132 additions & 11 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,145 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.14.0] - 2026-04-12

Headline themes: dialect-aware transforms, Snowflake at 100% of the QA corpus, ClickHouse significantly expanded (83% of the QA corpus, up from 53%), live schema introspection, SQL transpilation, and first-class integration sub-modules (OpenTelemetry and GORM). Drop-in upgrade from v1.13.0 — no breaking changes.

### Added
- **SQL Transpilation** (`pkg/transpiler`): New `Transpile(sql, from, to)` function converts SQL between dialects with a composable rewrite-rule pipeline

#### Dialect-aware formatting (closes #479)
- **`transform.FormatSQLWithDialect(stmt, keywords.SQLDialect)`**: renders an AST using dialect-specific row-limiting syntax — `SELECT TOP n` for SQL Server, `FETCH FIRST n ROWS ONLY` for Oracle, `LIMIT n` for PostgreSQL/MySQL/SQLite/Snowflake/ClickHouse/MariaDB
- **`transform.ParseSQLWithDialect(sql, keywords.SQLDialect)`**: dialect-aware parsing convenience wrapper
- `FormatOptions.Dialect` field threads dialect through the formatter pipeline
- `normalizeSelectForDialect()` converts `Limit`/`Offset` into `TopClause` (SQL Server) or `FetchClause` (Oracle) on a shallow copy — original AST never mutated
- Parsed `TopClause` now renders (previously silently dropped — a long-standing bug)
- SQL Server pagination: `OFFSET m ROWS FETCH NEXT n ROWS ONLY` when both limit and offset are set
- PR #507

#### Snowflake dialect (100% QA corpus — 87/87 queries, epic #483)
- **`MATCH_RECOGNIZE`** clause (SQL:2016 R010): PARTITION BY, ORDER BY, MEASURES, ONE/ALL ROWS PER MATCH, AFTER MATCH SKIP, PATTERN, DEFINE (PR #506)
- **`@stage` references** in FROM clause with optional path and file format (PR #505)
- **`MINUS` as EXCEPT synonym** for Snowflake/Oracle (PR #494); fix for MINUS-as-alias edge case (PR #504)
- **`SAMPLE` / `TABLESAMPLE`** clause with BERNOULLI/ROW/SYSTEM/BLOCK methods (PR #501)
- **`QUALIFY`** clause for window-function filtering (Snowflake/BigQuery) (PR #490)
- **VARIANT colon-path expressions** (`expr:field.sub[0]`) (PR #496)
- **Time-travel** `AT`/`BEFORE`/`CHANGES` on table references (PR #495)
- **`LATERAL FLATTEN`** table function + named arguments (`name => expr`) (PR #492)
- **`TRY_CAST`** + `IGNORE NULLS` / `RESPECT NULLS` for window functions (PR #486)
- **`LIKE ANY`/`LIKE ALL`** and `ILIKE ANY`/`ILIKE ALL` quantifiers (PR #500)
- **`USE [WAREHOUSE|DATABASE|SCHEMA|ROLE]`** and `DESCRIBE` with object-kind prefixes (PR #491)
- **`COPY INTO`**, `PUT`, `GET`, `LIST`, `REMOVE` parse-only stubs (PR #499)
- **`CREATE STAGE`/`STREAM`/`TASK`/`PIPE`/`FILE FORMAT`/`WAREHOUSE`/`DATABASE`/`SCHEMA`/`ROLE`/`SEQUENCE`/`FUNCTION`/`PROCEDURE`** parse-only stubs (PR #498)
- **`CLUSTER BY`**, `COPY GRANTS`, and CTAS (`CREATE TABLE ... AS SELECT`) for CREATE TABLE (PR #504)
- **`ILIKE`** + PIVOT/UNPIVOT support for Snowflake dialect (PR #484)

#### ClickHouse dialect (69/83 QA corpus — 83%, up from 53% in v1.13.0, epic #482)
- Remaining ClickHouse QA gaps (tracked for v1.15): ARRAY JOIN / LEFT ARRAY JOIN, LIMIT n,m BY, named window definitions, scalar CTE subqueries, CREATE MATERIALIZED VIEW / DISTRIBUTED TABLE, SAMPLE with decimal and OFFSET, standalone SELECT ... SETTINGS without preceding FROM/WHERE
- **Reserved-word identifiers**: `table`, `partition`, `tables`, `databases` now valid as column/identifier names (closes #480, PR #481)
- **Nested column types**: `Nullable(T)`, `Array(T)`, `Map(K,V)`, `LowCardinality(T)` in CREATE TABLE (PR #488)
- **Engine clauses**: `MergeTree()`, `ORDER BY`, `PARTITION BY`, `PRIMARY KEY`, `SAMPLE BY`, `TTL`, `SETTINGS` (PR #488)
- **Parametric aggregates**: `quantile(0.95)(duration)` double-paren syntax (PR #487)
- **Bare-bracket array literals**: `[1, 2, 3]` without ARRAY keyword (PR #485)
- **`ORDER BY ... WITH FILL`** with FROM/TO/STEP (PR #493)
- **`CODEC(...)`** column compression option (PR #497)
- **`WITH TOTALS`** in GROUP BY, **`LIMIT ... BY`** clause, **ANY/ALL JOIN** strictness prefix, `DEFAULT` as identifier (PR #503)
- **`SETTINGS`** clause on SELECT and in CREATE TABLE, **`TTL`** column clause, **`INSERT ... FORMAT`** (JSONEachRow, CSV, Parquet, etc.) (PR #489)

#### MariaDB dialect
- New `DialectMariaDB` extending MySQL (`mariadb`) (PR #431)
- **SEQUENCE DDL**: `CREATE/DROP/ALTER SEQUENCE` with full option set (START WITH, INCREMENT BY, MINVALUE/MAXVALUE, CYCLE, CACHE, NOCACHE)
- **Temporal tables**: `FOR SYSTEM_TIME`, `WITH SYSTEM VERSIONING`, `PERIOD FOR`
- **Hierarchical queries**: `CONNECT BY` with `PRIOR`, `START WITH`, `NOCYCLE`
- Added to playground and WASM dialect map (PR #432)

#### SQL Server enhancements
- **PIVOT / UNPIVOT** clause parsing (T-SQL) (PR #477)

#### SQL Transpilation (`pkg/transpiler`)
- **`Transpile(sql, from, to keywords.SQLDialect)`**: composable rewrite-rule pipeline for cross-dialect conversion (PR #449)
- MySQL → PostgreSQL: `AUTO_INCREMENT` → `SERIAL`/`BIGSERIAL`, `TINYINT(1)` → `BOOLEAN`
- PostgreSQL → MySQL: `SERIAL` → `INT AUTO_INCREMENT`, `ILIKE` → `LOWER() LIKE LOWER()`
- PostgreSQL → SQLite: `SERIAL`/`BIGSERIAL` → `INTEGER`, array types → `TEXT`
- `gosqlx.Transpile()` top-level convenience wrapper
- `gosqlx transpile --from <dialect> --to <dialect>` CLI subcommand
- **Live schema introspection** (`pkg/schema/db`): New `Loader` interface and `DatabaseSchema`/`Table`/`Column`/`Index`/`ForeignKey` types for querying live database metadata
- **PostgreSQL schema loader** (`pkg/schema/postgres`): Introspects tables, columns (with primary/unique flags), indexes, and foreign keys via `information_schema` and `pg_catalog`
- **MySQL schema loader** (`pkg/schema/mysql`): Introspects tables, columns, indexes, and foreign keys via `information_schema`
- **SQLite schema loader** (`pkg/schema/sqlite`): Introspects tables, columns, indexes, and foreign keys via PRAGMA commands (pure Go, no cgo required)
- **`gosqlx.LoadSchema()`** top-level convenience wrapper for dialect-agnostic schema loading
- **`gosqlx.Transpile()`** top-level convenience wrapper
- **`gosqlx transpile --from <dialect> --to <dialect>`** CLI subcommand

#### Live schema introspection (`pkg/schema/db`)
- **`Loader` interface** with `DatabaseSchema`/`Table`/`Column`/`Index`/`ForeignKey` types (PR #448)
- **PostgreSQL loader** (`pkg/schema/postgres`): introspects tables, columns (with primary/unique flags), indexes, and foreign keys via `information_schema` and `pg_catalog`
- **MySQL loader** (`pkg/schema/mysql`): introspects tables, columns, indexes, and foreign keys via `information_schema`
- **SQLite loader** (`pkg/schema/sqlite`): introspects via PRAGMA commands (pure Go, no cgo)
- **`gosqlx.LoadSchema()`** top-level dialect-agnostic convenience wrapper
- Integration tests using `testcontainers-go` v0.32.0 for PostgreSQL and MySQL loaders
- **MariaDB dialect** (`--dialect mariadb`): New SQL dialect extending MySQL with support for SEQUENCE DDL (`CREATE/DROP/ALTER SEQUENCE` with full option set), temporal tables (`FOR SYSTEM_TIME`, `WITH SYSTEM VERSIONING`, `PERIOD FOR`), and `CONNECT BY` hierarchical queries with `PRIOR`, `START WITH`, and `NOCYCLE`
- `integrations/opentelemetry/` sub-module: `InstrumentedParse()` wraps `gosqlx.Parse()` with OpenTelemetry spans including `db.system`, `db.statement.type`, `db.sql.tables`, `db.sql.columns` attributes
- `integrations/gorm/` sub-module: GORM plugin that records executed query metadata (tables, columns, statement type) via GoSQLX parsing with GORM SQL normalization (backtick identifiers, `?` placeholders); exposes `Stats()` and `Reset()` APIs

#### DML Transform API (`pkg/transform`)
- `AddSetClause`, `SetClause`, `RemoveSetClause`, `ReplaceSetClause` for UPDATE statements (PR #446)
- `AddReturning`, `RemoveReturning` for INSERT/UPDATE/DELETE (PR #446)

#### Linter — expanded from 10 to 30 rules (PR #445)
- New rule categories: **safety**, **performance**, **naming** (alongside existing style and whitespace)
- Rules cover SQL injection patterns, missing WHERE clauses on UPDATE/DELETE, implicit type conversions, inconsistent identifier casing, table aliasing conventions, and more
- See `docs/LINTING_RULES.md` for full reference

#### Optimization Advisor (`pkg/advisor`)
- 12 new optimization rules: **OPT-009 through OPT-020** (PR #464)
- `gosqlx optimize` CLI subcommand runs the full advisor pipeline

#### Fingerprinting (`pkg/fingerprint`)
- **`Normalize(sql)`** canonicalizes literals (`WHERE id = 123` → `WHERE id = ?`) (PR #463)
- **`Fingerprint(sql)`** returns SHA-256 hash of normalized query for deduplication and query caches
- Integration with advisor and linter for aggregated findings

#### Integrations (sub-modules)
- **`integrations/opentelemetry/`**: `InstrumentedParse()` wraps `gosqlx.Parse()` with OpenTelemetry spans including `db.system`, `db.statement.type`, `db.sql.tables`, `db.sql.columns` attributes (PR #451)
- **`integrations/gorm/`**: GORM plugin that records executed query metadata (tables, columns, statement type) via GoSQLX parsing with GORM SQL normalization (backtick identifiers, `?` placeholders); exposes `Stats()` and `Reset()` APIs (PR #452)
- CI workflow for integration sub-modules (`.github/workflows/integrations.yml`)

#### Parser / AST additions
- **DDL formatter**, **`CONNECT BY`** hierarchical queries, **`SAMPLE`** clause (PR #472)
- **JSON function parsing** improvements (PR #460)
- **`gosqlx stats`** CLI subcommand exposes object pool utilization (PR #459)
- **`gosqlx watch`** CLI subcommand for continuous validation (PR #458)
- **`gosqlx action`** GitHub Actions integration with inline annotations (PR #443)

#### C binding
- Coverage hardened from **18% to 93%** via comprehensive test suite (PR #447)

#### Docs, website, and CI
- "Who's Using GoSQLX" section in README (PR #475)
- OpenSSF Scorecard security analysis workflow (PR #443)
- Sentry → GitHub Issues automation (PR #438)
- Website: mobile responsiveness improvements (PR #441), comprehensive a11y/UX audit (PR #440)
- MariaDB + Snowflake added to playground dialect dropdown (PR #432)

### Changed
- **OpenTelemetry SDK** bumped from v1.42.0 to v1.43.0 to address **CVE-2026-39883** (PR #502)
- Linter rule count exposed in glama.json, docs, and CLI help (10 → 30)
- Docs: SQL compatibility matrix updated to reflect Snowflake and ClickHouse 100% pass rate

### Fixed
- **SQL Server TOP rendering**: parsed `TopClause` values now correctly render in formatted output (PR #507). Round-trippers that parse `SELECT TOP 10 * FROM users` and format again will see `TOP 10` preserved — previously it was silently dropped.
- **Snowflake ILIKE + PIVOT/UNPIVOT** parse-time fix (PR #484)
- **MINUS consumed as select-list alias**: `SELECT 1 MINUS SELECT 2` now correctly treats MINUS as a set operator (PR #494)
- **ClickHouse production playground WASM 404**: committed `gosqlx.wasm` to git, removed auto-commit step blocked by branch protection (PRs #423, #424)
- **Playground React error #310**: `useState`/`useCallback` now above early returns (PR #429)
- Website: Vercel Analytics, Speed Insights, and CSP fixes (PR #433); Sentry hydration mismatch (PRs #434, #437, #439)
- CI: `testcontainers` skip on Windows; `.trivyignore` entries for unfixable transitive CVEs; graceful handling of `glama-sync` on non-GA runners

### Deprecated
Carried over from v1.13.0 (no new deprecations in v1.14.0):
- `parser.Parse([]token.Token)` — use `ParseFromModelTokens` instead
- `ParseFromModelTokensWithPositions` — consolidated into `ParseFromModelTokens`
- `ConversionResult.PositionMapping` — always nil, will be removed in v2

### Security
- **CVE-2026-39883** (HIGH, `go.opentelemetry.io/otel/sdk`): resolved by upgrading to v1.43.0 (PR #502)
- **`.trivyignore`** entries audited and documented — all remaining ignores are transitive test-only (Docker via testcontainers) or npm-only (website build-time dependencies, not shipped in Go binaries)

### Companion releases
- **VS Code extension**: bumped to **1.14.0** — tracks library version, no behavioral changes
- **MCP server**: bumped to **1.14.0** — glama.json updated with MariaDB and ClickHouse in dialect list
- **`pygosqlx` Python bindings**: bumped to **0.2.0** — Python bindings follow an independent semver track (alpha) since they have not yet received the same QA sweep as the core library

## [1.13.0] - 2026-03-20

### Added
Expand Down
2 changes: 1 addition & 1 deletion cmd/gosqlx/cmd/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@
//
// Version information:
//
// Version = "1.13.0" - Current CLI version
// Version = "1.14.0" - Current CLI version
//
// # Dependencies
//
Expand Down
28 changes: 17 additions & 11 deletions cmd/gosqlx/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,22 @@ import (
// This version tracks feature releases and compatibility.
// Format: MAJOR.MINOR.PATCH (Semantic Versioning 2.0.0)
//
// Version 1.14.0 includes:
// - Dialect-aware SQL formatting (TOP/FETCH FIRST/LIMIT per dialect)
// - Snowflake dialect: 87/87 QA pass (MATCH_RECOGNIZE, @stage, SAMPLE, QUALIFY, VARIANT, time-travel, MINUS, LATERAL FLATTEN, TRY_CAST)
// - ClickHouse dialect: 69/83 QA pass, up from 53% (nested types, parametric aggregates, WITH FILL, CODEC, SETTINGS/TTL, INSERT FORMAT)
// - MariaDB dialect (SEQUENCE DDL, temporal tables, CONNECT BY)
// - SQL transpilation (MySQL↔PostgreSQL, PostgreSQL→SQLite) + CLI subcommand
// - Live schema introspection (Postgres/MySQL/SQLite loaders)
// - DML transform API (SET clause, RETURNING clause)
// - Linter expanded from 10 to 30 rules
// - Integrations: OpenTelemetry + GORM sub-modules
//
// Version 1.13.0 includes:
// - ClickHouse SQL dialect support (PREWHERE, FINAL, GLOBAL IN)
// - LSP semantic tokens + diagnostic debouncing
// - Parser API consolidation (ParseFromModelTokens canonical)
// Version 1.12.1 includes:
// - MCP Server: All GoSQLX SQL capabilities as Model Context Protocol tools over streamable HTTP
// - 7 MCP tools: validate_sql, format_sql, parse_sql, extract_metadata, security_scan, lint_sql, analyze_sql
// - Optional bearer token auth via GOSQLX_MCP_AUTH_TOKEN
// - Go minimum bumped to 1.23.0 (required by mark3labs/mcp-go)
var Version = "1.13.0"
// - ClickHouse SQL dialect support (PREWHERE, FINAL, GLOBAL IN)
// - LSP semantic tokens + diagnostic debouncing
// - Parser API consolidation (ParseFromModelTokens canonical)
var Version = "1.14.0"

var (
// verbose enables detailed output for debugging and troubleshooting.
Expand Down Expand Up @@ -120,12 +126,12 @@ Key features:
• High-performance formatting with intelligent indentation
• AST structure inspection and analysis
• Security vulnerability detection
• Multi-dialect SQL support (PostgreSQL, MySQL, SQL Server, Oracle, SQLite)
• Multi-dialect SQL support (PostgreSQL, MySQL, MariaDB, SQL Server, Oracle, SQLite, Snowflake, ClickHouse)
• Batch processing with directory/glob patterns
• CI/CD integration with proper exit codes

Performance: 1.5M+ operations/second sustained, 1.97M peak. 100-1000x faster than competitors.`,
Version: "1.13.0",
Version: "1.14.0",
}

// Execute adds all child commands to the root command and sets flags appropriately.
Expand Down
Loading
Loading