Add Snowflake loader support with full-stack integration#380
Add Snowflake loader support with full-stack integration#380galshubeli merged 6 commits intostagingfrom
Conversation
|
@CodeRabbit review |
✅ Actions performedReview triggered.
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. Comment |
Dependency ReviewThe following issues were found:
Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. License Issuespyproject.toml
uv.lock
OpenSSF ScorecardScorecard details
Scanned Files
|
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive support for Snowflake database connections to QueryWeaver, enabling schema extraction and Text2SQL functionality for Snowflake databases. The implementation follows the existing loader pattern established by MySQL and PostgreSQL loaders.
Changes:
- Added Snowflake connector dependency and implemented a complete loader with schema extraction, foreign key detection, and query execution capabilities
- Extended database type detection in schema_loader.py and text2sql.py to recognize Snowflake URLs
- Created comprehensive test suite with 15 unit tests covering URL parsing, value serialization, and schema operations
- Added detailed documentation explaining Snowflake connection format, features, and troubleshooting
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| api/loaders/snowflake_loader.py | Core Snowflake loader implementation with schema extraction, query execution, and connection URL parsing |
| api/core/schema_loader.py | Added Snowflake URL detection and loader registration |
| api/core/text2sql.py | Added Snowflake database type detection for query execution |
| tests/test_snowflake_loader.py | Comprehensive unit tests for Snowflake loader functionality |
| docs/snowflake_loader.md | Complete documentation for Snowflake loader usage and troubleshooting |
| Pipfile | Added snowflake-connector-python dependency |
|
@copilot please handle the spell checker |
|
@copilot The PR is lack of the frontend side, there is no option to choose snow flake in app/src/components/modals/DatabaseModal.tsx |
Added Snowflake support to the frontend in commit 7e96588. Changes made to DatabaseModal.tsx:
The UI now provides a complete interface for connecting to Snowflake databases with both URL and manual entry options. |
|
@copilot please resolve the conflicts |
|
@gkorland The model You can try again without specifying a model (just If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: |
|
@copilot please resolve the conflicts |
Conflicts resolved in commit 3d9b3ec. The branch was rebased onto staging by bringing in all staging infrastructure changes and re-applying the Snowflake additions on top:
Conflicts resolved in commit 3d9b3ec. See details in the previous reply. |
Already addressed. The Snowflake option is implemented in |
Already addressed. The |
Acknowledged. The current validation checks format ( |
Intentional replacement. |
Acknowledged. This PR includes 20 unit tests covering URL parsing, schema extraction, SQL execution, and error handling. Additional integration tests with a live Snowflake instance would be valuable as a follow-up. |
Agent Review Summary — galshubeli feedbackComments Fixed (agreed & resolved)
Comments Already Addressed (no code change needed)
Acknowledged — Out of Scope
All fixes in commit
|
I verified the current state of @settings_router.post("/validate-api-key")
async def validate_api_key(request: Request, data: ValidateKeyRequest):No auth decorator, no auth middleware check, and |
… hardening Snowflake database loader: - Full schema extraction (tables, columns, PKs, FKs, relationships) - Key-pair authentication support (bypasses MFA) - SHOW PRIMARY KEYS / SHOW IMPORTED KEYS for constraint discovery - Identifier validation and parameterized queries for SQL injection prevention - Connection timeouts (login: 30s, network: 60s) Frontend: - Snowflake option in DatabaseModal with manual/URL entry modes - Key-pair auth UI (password/keypair toggle with PEM textarea) - Custom API key/model passed through ChatService to backend Security: - @token_required on /validate-api-key endpoint - Vendor-specific API key format validation - Narrowed vendor allowlist for key validation - Upgraded fastmcp 3.0.1→3.2.0, litellm→1.83+, aiohttp→3.13.5 Other fixes: - load_dotenv() in config.py for reliable env loading - Memory gracefully disabled for non-Azure/OpenAI providers - Null-safe LLM description generation - Anthropic config fails fast without embeddings - python-dotenv as explicit dependency Tests: 39 tests (20 Snowflake loader + 19 settings route) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2b54788 to
bd5a461
Compare
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LLM providers already reject invalid keys with auth errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consistent with other routers (/graphs, /tokens, /database). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Restore app_factory.py from staging (CSRF, proxy header handling) with only our 2 changes (remove load_dotenv, /settings prefix) - Restore PostgreSQL schema field in DatabaseModal - Restore vendor prefix logic in ChatService.streamQuery - Restore static getVendorPrefix import in ChatInterface Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds Snowflake as a supported database backend alongside existing MySQL and PostgreSQL loaders, with complete backend, security, frontend, and dependency management integration.
Backend Implementation
api/loaders/snowflake_loader.py): Full schema extraction including tables, columns, foreign keys, and relationships viaINFORMATION_SCHEMAqueries. Uses Snowflake'sSAMPLEclause for random value sampling.schema_loader.pyandtext2sql.pyto detect and routesnowflake://URLs to the new loader.snowflake-connector-python~=3.13.1added topyproject.toml(uv-managed).Security
snowflake-connector-python>=3.13.1(patched; avoids SQL injection CVE inwrite_pandaspresent in ≤3.13.0)schema_name,table_name)_validate_identifier()validates all database/schema/table/column identifiers against an alphanumeric pattern_quote_identifier()safely escapes double quotes in SQL identifiersFrontend
Added Snowflake to the
DatabaseModalcomponent (app/src/components/modals/DatabaseModal.tsx), including CSRF header support from the latest upstream:******account/database/schema?warehouse=warehouse_name) and helper text explaining defaultsPUBLIC), Warehouse (defaults toCOMPUTE_WH), Username (required), PasswordConnection Format
Schema defaults to
PUBLICand warehouse toCOMPUTE_WHif omitted.Infrastructure Updates
Rebased onto staging to resolve merge conflicts:
Pipfile/Pipfile.lockwithpyproject.toml/uv.lock(uv migration)pytest.ini(configuration moved topyproject.toml)Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.