feat: add Polaris Catalog namespace implementation#204
Closed
jackye1995 wants to merge 7 commits into
Closed
Conversation
Add support for Polaris Catalog integration using the Generic Table API. This follows the same pattern as Unity Catalog implementation. - Implement PolarisNamespace with full namespace and table operations - Use RestClient from lance-namespace-core for HTTP communication - Support bearer token authentication - Map Polaris Generic Table API to Lance namespace interface - Add comprehensive documentation in docs/src/impls/polaris.md - Include unit tests with mocked RestClient The implementation allows Lance tables to be managed alongside other table formats (Iceberg, Delta) in Polaris Catalog. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add TestPolarisNamespaceIntegration for testing against running Polaris - Support OAuth and basic auth for Polaris authentication - Skip integration tests when Polaris is not available - Fix test mocking to include doc field in responses - Update README with testing instructions Integration tests run with: mvn test -Dtest.polaris.integration=true
- Remove need for -Dtest.polaris.integration=true flag - Automatically detect if Polaris is available at localhost:8182 - Skip integration tests if Polaris API is not responding - Check /api/catalog/v1/namespaces endpoint to verify Polaris is running - Update documentation to reflect automatic detection
- Remove module-specific README.md - Simplify polaris.md to focus on configuration, namespace mapping, and table definition - Remove programming language specific sections - Follow the concise style of Unity and Glue documentation
- Remove polaris. prefix from configuration properties - Update namespace mapping docs to reflect arbitrary nesting support - Remove created_at property from table definition - Set managed_by to 'storage' by default, matching Unity and Hive - Only set version property when managed_by=impl - Move API endpoints to top-level introduction - Remove limitations section from documentation
- Catalog is a user-creatable entity in Polaris, not a configuration - Catalog is now the first level of the namespace hierarchy - Update API URL to use /api/catalog/v1 path - Remove catalog from PolarisNamespaceConfig - Update documentation to reflect catalog as part of namespace mapping
- Change auth.token to auth_token - Change connect.timeout to connect_timeout - Change read.timeout to read_timeout - Change max.retries to max_retries - Follow consistent naming convention with underscore separators
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes
PolarisNamespaceclass with full namespace and table operationsPolarisNamespaceConfigfor configuration managementPolarisModelsfor Polaris API data transfer objectsRestClientfrom lance-namespace-core for HTTP communicationdocs/src/impls/polaris.mdImplementation Details
The implementation uses Polaris's Generic Table API endpoints:
/namespaces)/namespaces/{namespace}/generic-tables)Lance tables are stored as Generic Tables with
format: "lance"and appropriate properties for identification.Testing
Documentation
Comprehensive documentation added at
docs/src/impls/polaris.mdcovering:🤖 Generated with Claude Code