Skip to content

Commit b4d2225

Browse files
ajitpratap0Ajit Pratap Singhclaude
authored
feat(schema): live DB schema introspection for Postgres, MySQL, SQLite (#448)
* feat(schema): live DB schema introspection for Postgres, MySQL, SQLite (#448) Adds pkg/schema/db with Loader interface and DatabaseSchema/Table/Column/ Index/ForeignKey types for querying live database schema metadata. Dialect-specific implementations: - pkg/schema/postgres: information_schema + pg_catalog queries - pkg/schema/mysql: information_schema queries - pkg/schema/sqlite: PRAGMA commands (pure Go, no cgo) Also adds gosqlx.LoadSchema() top-level convenience wrapper and integration tests using testcontainers-go v0.32.0 for Postgres/MySQL. SQLite tests run without Docker and always pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(schema): resolve lint errcheck and bump vulnerable transitive deps Fix 12 errcheck lint errors by wrapping defer rows.Close() calls to explicitly discard the error. Bump testcontainers-go v0.32.0 -> v0.41.0, containerd v1.7.18 -> v1.7.30, docker v27.0.3 -> v28.5.2, x/crypto v0.22.0 -> v0.48.0, grpc v1.59.0 -> v1.67.0 to resolve CVE-2024-25621, CVE-2024-40635, CVE-2025-64329, CVE-2024-41110, CVE-2024-45337, CVE-2025-22869, CVE-2025-47914, CVE-2025-58181, CVE-2026-33186. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(schema): stabilize testcontainers tests and allow docker GHSA - Add testing.Short() guard to postgres/mysql loader tests so they skip in the Race Detector job (which runs with -short) - Replace wait.ForListeningPort with wait.ForLog for postgres container to wait until the database is fully ready (fixes EOF on create tables) - Add db.Ping retry loop after container start for both postgres and mysql to handle the window between port-listen and query-ready - Allow GHSA-x744-4wpc-v9h2 (docker AuthZ bypass) in dependency review; transitive dep via testcontainers-go with no upstream fix available Closes race-detector and macOS test failures on PR #471. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(schema): skip testcontainers tests when Docker is unavailable Add runtime Docker availability check (docker info) before attempting to create testcontainers. This prevents test failures on macOS CI runners that don't have Docker installed, without requiring -short flag. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Ajit Pratap Singh <ajitpratapsingh@Ajits-Mac-mini-2655.local> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent a7ce3aa commit b4d2225

File tree

14 files changed

+1605
-13
lines changed

14 files changed

+1605
-13
lines changed

.github/workflows/security.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,10 @@ jobs:
164164
uses: actions/dependency-review-action@v4
165165
with:
166166
fail-on-severity: high
167+
# GHSA-x744-4wpc-v9h2: docker/docker AuthZ plugin bypass with oversized
168+
# request bodies. Transitive dep via testcontainers-go (test-only).
169+
# No fixed version available upstream as of 2026-03-29.
170+
allow-ghsas: GHSA-x744-4wpc-v9h2
167171
# Include both the compound SPDX expression and individual components
168172
# to handle golang.org/x packages which report as compound license
169173
allow-licenses: >-

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1414
- PostgreSQL → SQLite: `SERIAL`/`BIGSERIAL``INTEGER`, array types → `TEXT`
1515
- `gosqlx.Transpile()` top-level convenience wrapper
1616
- `gosqlx transpile --from <dialect> --to <dialect>` CLI subcommand
17+
- **Live schema introspection** (`pkg/schema/db`): New `Loader` interface and `DatabaseSchema`/`Table`/`Column`/`Index`/`ForeignKey` types for querying live database metadata
18+
- **PostgreSQL schema loader** (`pkg/schema/postgres`): Introspects tables, columns (with primary/unique flags), indexes, and foreign keys via `information_schema` and `pg_catalog`
19+
- **MySQL schema loader** (`pkg/schema/mysql`): Introspects tables, columns, indexes, and foreign keys via `information_schema`
20+
- **SQLite schema loader** (`pkg/schema/sqlite`): Introspects tables, columns, indexes, and foreign keys via PRAGMA commands (pure Go, no cgo required)
21+
- **`gosqlx.LoadSchema()`** top-level convenience wrapper for dialect-agnostic schema loading
22+
- Integration tests using `testcontainers-go` v0.32.0 for PostgreSQL and MySQL loaders
1723
- **MariaDB dialect** (`--dialect mariadb`): New SQL dialect extending MySQL with support for SEQUENCE DDL (`CREATE/DROP/ALTER SEQUENCE` with full option set), temporal tables (`FOR SYSTEM_TIME`, `WITH SYSTEM VERSIONING`, `PERIOD FOR`), and `CONNECT BY` hierarchical queries with `PRIOR`, `START WITH`, and `NOCYCLE`
1824
- `integrations/opentelemetry/` sub-module: `InstrumentedParse()` wraps `gosqlx.Parse()` with OpenTelemetry spans including `db.system`, `db.statement.type`, `db.sql.tables`, `db.sql.columns` attributes
1925
- `integrations/gorm/` sub-module: GORM plugin that records executed query metadata (tables, columns, statement type) via GoSQLX parsing with GORM SQL normalization (backtick identifiers, `?` placeholders); exposes `Stats()` and `Reset()` APIs

go.mod

Lines changed: 73 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,93 @@ go 1.26.1
44

55
require (
66
github.com/fsnotify/fsnotify v1.9.0
7+
github.com/go-sql-driver/mysql v1.8.0
8+
github.com/lib/pq v1.10.9
79
github.com/mark3labs/mcp-go v0.45.0
810
github.com/spf13/cobra v1.10.1
911
github.com/spf13/pflag v1.0.9
10-
golang.org/x/term v0.20.0
12+
github.com/testcontainers/testcontainers-go v0.41.0
13+
golang.org/x/term v0.40.0
1114
gopkg.in/yaml.v3 v3.0.1
15+
modernc.org/sqlite v1.30.1
1216
)
1317

1418
require (
19+
dario.cat/mergo v1.0.2 // indirect
20+
filippo.io/edwards25519 v1.1.0 // indirect
21+
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
22+
github.com/Microsoft/go-winio v0.6.2 // indirect
1523
github.com/bahlo/generic-list-go v0.2.0 // indirect
1624
github.com/buger/jsonparser v1.1.1 // indirect
25+
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
26+
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
27+
github.com/cespare/xxhash/v2 v2.3.0 // indirect
28+
github.com/containerd/errdefs v1.0.0 // indirect
29+
github.com/containerd/errdefs/pkg v0.3.0 // indirect
30+
github.com/containerd/log v0.1.0 // indirect
31+
github.com/containerd/platforms v0.2.1 // indirect
32+
github.com/cpuguy83/dockercfg v0.3.2 // indirect
33+
github.com/davecgh/go-spew v1.1.1 // indirect
34+
github.com/distribution/reference v0.6.0 // indirect
35+
github.com/docker/docker v28.5.2+incompatible // indirect
36+
github.com/docker/go-connections v0.6.0 // indirect
37+
github.com/docker/go-units v0.5.0 // indirect
38+
github.com/dustin/go-humanize v1.0.1 // indirect
39+
github.com/ebitengine/purego v0.10.0 // indirect
40+
github.com/felixge/httpsnoop v1.0.4 // indirect
41+
github.com/go-logr/logr v1.4.3 // indirect
42+
github.com/go-logr/stdr v1.2.2 // indirect
43+
github.com/go-ole/go-ole v1.2.6 // indirect
1744
github.com/google/uuid v1.6.0 // indirect
45+
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
1846
github.com/inconshreveable/mousetrap v1.1.0 // indirect
1947
github.com/invopop/jsonschema v0.13.0 // indirect
48+
github.com/klauspost/compress v1.18.2 // indirect
49+
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
50+
github.com/magiconair/properties v1.8.10 // indirect
2051
github.com/mailru/easyjson v0.7.7 // indirect
52+
github.com/mattn/go-isatty v0.0.20 // indirect
53+
github.com/moby/docker-image-spec v1.3.1 // indirect
54+
github.com/moby/go-archive v0.2.0 // indirect
55+
github.com/moby/patternmatcher v0.6.0 // indirect
56+
github.com/moby/sys/sequential v0.6.0 // indirect
57+
github.com/moby/sys/user v0.4.0 // indirect
58+
github.com/moby/sys/userns v0.1.0 // indirect
59+
github.com/moby/term v0.5.2 // indirect
60+
github.com/morikuni/aec v1.0.0 // indirect
61+
github.com/ncruces/go-strftime v0.1.9 // indirect
62+
github.com/opencontainers/go-digest v1.0.0 // indirect
63+
github.com/opencontainers/image-spec v1.1.1 // indirect
64+
github.com/pkg/errors v0.9.1 // indirect
65+
github.com/pmezard/go-difflib v1.0.0 // indirect
66+
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
67+
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
68+
github.com/shirou/gopsutil/v4 v4.26.2 // indirect
69+
github.com/sirupsen/logrus v1.9.3 // indirect
2170
github.com/spf13/cast v1.7.1 // indirect
71+
github.com/stretchr/testify v1.11.1 // indirect
72+
github.com/tklauser/go-sysconf v0.3.16 // indirect
73+
github.com/tklauser/numcpus v0.11.0 // indirect
2274
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
2375
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
24-
golang.org/x/sys v0.20.0 // indirect
76+
github.com/yusufpapurcu/wmi v1.2.4 // indirect
77+
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
78+
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
79+
go.opentelemetry.io/otel v1.41.0 // indirect
80+
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0 // indirect
81+
go.opentelemetry.io/otel/metric v1.41.0 // indirect
82+
go.opentelemetry.io/otel/trace v1.41.0 // indirect
83+
go.opentelemetry.io/proto/otlp v1.0.0 // indirect
84+
golang.org/x/crypto v0.48.0 // indirect
85+
golang.org/x/sys v0.41.0 // indirect
86+
golang.org/x/time v0.12.0 // indirect
87+
golang.org/x/tools v0.41.0 // indirect
88+
google.golang.org/genproto/googleapis/api v0.0.0-20240814211410-ddb44dafa142 // indirect
89+
google.golang.org/protobuf v1.35.2 // indirect
90+
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
91+
modernc.org/libc v1.52.1 // indirect
92+
modernc.org/mathutil v1.6.0 // indirect
93+
modernc.org/memory v1.8.0 // indirect
94+
modernc.org/strutil v1.2.0 // indirect
95+
modernc.org/token v1.1.0 // indirect
2596
)

0 commit comments

Comments
 (0)