Skip to content

Add BigQuery adapter implementation (Fixes #117)#195

Merged
shiva-intugle merged 9 commits into
Intugle:mainfrom
ZAK1504:feature/bigquery-adapter
Dec 18, 2025
Merged

Add BigQuery adapter implementation (Fixes #117)#195
shiva-intugle merged 9 commits into
Intugle:mainfrom
ZAK1504:feature/bigquery-adapter

Conversation

@ZAK1504

@ZAK1504 ZAK1504 commented Dec 12, 2025

Copy link
Copy Markdown
Contributor

Description

This PR implements a new adapter for Google BigQuery, enabling Intugle to connect to BigQuery datasets for profiling, analysis, and data product generation.

Changes Made

  • Created BigQuery adapter (src/intugle/adapters/types/bigquery/bigquery.py) with full implementation of all abstract methods from the Adapter base class
  • Added Pydantic models (models.py) for BigQuery connection configuration and data configuration
  • Registered adapter in factory with conditional loading based on dependency availability
  • Added optional dependency google-cloud-bigquery>=3.11.0 to pyproject.toml
  • Created comprehensive unit tests with 20 test cases covering adapter contract, BigQuery-specific behavior, error handling, and model validation
  • Added documentation (README.md) with usage examples, configuration guide, and troubleshooting tips

Features Implemented

✅ Connect to BigQuery using service accounts or Application Default Credentials
✅ Profile BigQuery tables and columns with detailed statistics
✅ Execute Standard SQL queries
✅ Convert query results to pandas DataFrames
✅ Create views and tables from queries (materialization)
✅ Analyze relationships between tables (intersection analysis)
✅ Support for composite keys
✅ Graceful handling of missing dependencies

Technical Implementation

  • Follows established adapter pattern (PostgresAdapter as reference)
  • Uses google-cloud-bigquery Python client library
  • Implements singleton pattern for adapter instance
  • Supports Standard SQL dialect
  • Compatible with Intugle's semantic search and data product generation
  • Handles Google Cloud authentication (service accounts + ADC)

Testing

  • Unit Tests: 20 comprehensive test cases
  • Tests Status: Skip when BigQuery dependencies not installed (expected behavior)
  • Coverage: Adapter contract, BigQuery-specific methods, error handling, model validation

License Compliance

google-cloud-bigquery uses Apache License 2.0 (fully compatible with project's Apache 2.0 license)

Breaking Changes

None - this is a new feature addition

Related Issue

Closes #117 - [HELP WANTED] Implement a new adapter for Google BigQuery

@shiva-intugle shiva-intugle merged commit 707d742 into Intugle:main Dec 18, 2025
1 check passed
@raphael-intugle

Copy link
Copy Markdown
Collaborator

Superb work @ZAK1504 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[HELP WANTED] Implement a new adapter for Google BigQuery

3 participants