Skip to content

Commit 3e62c0d

Browse files
authored
spec: allow non-REST namespace implementations (#68)
Closes #67
1 parent dd1feb2 commit 3e62c0d

102 files changed

Lines changed: 5713 additions & 1168 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format
44
to standardize access to a collection of Lance tables (a.k.a. Lance datasets).
5-
It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc.
5+
It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc.
66
should store and use Lance tables, as well as how ML/AI tools and analytics compute engines should integrate with Lance tables.
77

88
## Repository structure
@@ -33,7 +33,7 @@ pip install -r requirements.txt
3333

3434
### Install Poetry for Python
3535

36-
If you want to build the entire project, or develop python specifically,
36+
If you want to build the entire project, or develop python specifically,
3737
you need to install [poetry](https://python-poetry.org/).
3838
To quickly install it in your virtual environment:
3939

@@ -51,7 +51,7 @@ There are 3 commands that is available at top level as well as inside each langu
5151

5252
You can also run `make <command>-<language>` to only run the command in the specific language, for example:
5353

54-
- `make gen-python`: codegen and lint all Python modules
54+
- `make gen-python`: codegen and lint all Python modules
5555
- `make build-rust`: build all Rust modules
5656

5757
You can also run `make <command>-<language>-<module>` inside a language folder to run the command against a specific module, for example:

java/lance-namespace-apache-client/.openapi-generator/FILES

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,20 @@
22
README.md
33
api/openapi.yaml
44
docs/CreateNamespaceRequest.md
5+
docs/DropNamespaceRequest.md
56
docs/ErrorResponse.md
7+
docs/GetNamespaceRequest.md
68
docs/GetNamespaceResponse.md
9+
docs/GetTableRequest.md
710
docs/GetTableResponse.md
11+
docs/ListNamespacesRequest.md
812
docs/ListNamespacesResponse.md
913
docs/NamespaceApi.md
14+
docs/NamespaceExistsRequest.md
1015
docs/RegisterTableRequest.md
16+
docs/RegisterTableResponse.md
1117
docs/TableApi.md
18+
docs/TableExistsRequest.md
1219
pom.xml
1320
src/main/AndroidManifest.xml
1421
src/main/java/com/lancedb/lance/namespace/client/apache/ApiClient.java
@@ -28,8 +35,15 @@ src/main/java/com/lancedb/lance/namespace/client/apache/auth/Authentication.java
2835
src/main/java/com/lancedb/lance/namespace/client/apache/auth/HttpBasicAuth.java
2936
src/main/java/com/lancedb/lance/namespace/client/apache/auth/HttpBearerAuth.java
3037
src/main/java/com/lancedb/lance/namespace/client/apache/model/CreateNamespaceRequest.java
38+
src/main/java/com/lancedb/lance/namespace/client/apache/model/DropNamespaceRequest.java
3139
src/main/java/com/lancedb/lance/namespace/client/apache/model/ErrorResponse.java
40+
src/main/java/com/lancedb/lance/namespace/client/apache/model/GetNamespaceRequest.java
3241
src/main/java/com/lancedb/lance/namespace/client/apache/model/GetNamespaceResponse.java
42+
src/main/java/com/lancedb/lance/namespace/client/apache/model/GetTableRequest.java
3343
src/main/java/com/lancedb/lance/namespace/client/apache/model/GetTableResponse.java
44+
src/main/java/com/lancedb/lance/namespace/client/apache/model/ListNamespacesRequest.java
3445
src/main/java/com/lancedb/lance/namespace/client/apache/model/ListNamespacesResponse.java
46+
src/main/java/com/lancedb/lance/namespace/client/apache/model/NamespaceExistsRequest.java
3547
src/main/java/com/lancedb/lance/namespace/client/apache/model/RegisterTableRequest.java
48+
src/main/java/com/lancedb/lance/namespace/client/apache/model/RegisterTableResponse.java
49+
src/main/java/com/lancedb/lance/namespace/client/apache/model/TableExistsRequest.java

java/lance-namespace-apache-client/README.md

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ Lance REST Namespace Specification
66

77
- Generator version: 7.12.0
88

9-
**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables.
9+
**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables.
1010
A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy.
11-
In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml).
11+
In an enterprise environment, typically there is a requirement to store tables in a metadata service for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way.
1212

1313

1414

@@ -111,24 +111,31 @@ All URIs are relative to *http://localhost:2333*
111111

112112
Class | Method | HTTP request | Description
113113
------------ | ------------- | ------------- | -------------
114-
*NamespaceApi* | [**createNamespace**](docs/NamespaceApi.md#createNamespace) | **POST** /v1/namespaces | Create a new namespace
115-
*NamespaceApi* | [**dropNamespace**](docs/NamespaceApi.md#dropNamespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace
116-
*NamespaceApi* | [**getNamespace**](docs/NamespaceApi.md#getNamespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace
117-
*NamespaceApi* | [**listNamespaces**](docs/NamespaceApi.md#listNamespaces) | **GET** /v1/namespaces | List namespaces
118-
*NamespaceApi* | [**namespaceExists**](docs/NamespaceApi.md#namespaceExists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists
119-
*TableApi* | [**getTable**](docs/TableApi.md#getTable) | **GET** /v1/tables/{table} | Get a table from the namespace
120-
*TableApi* | [**registerTable**](docs/TableApi.md#registerTable) | **POST** /v1/table/register | Register a table to a namespace
121-
*TableApi* | [**tableExists**](docs/TableApi.md#tableExists) | **HEAD** /v1/tables/{table} | Check if a table exists
114+
*NamespaceApi* | [**createNamespace**](docs/NamespaceApi.md#createNamespace) | **POST** /CreateNamespace | Create a new namespace
115+
*NamespaceApi* | [**dropNamespace**](docs/NamespaceApi.md#dropNamespace) | **POST** /DropNamespace | Drop a namespace
116+
*NamespaceApi* | [**getNamespace**](docs/NamespaceApi.md#getNamespace) | **POST** /GetNamespace | Get information about a namespace
117+
*NamespaceApi* | [**listNamespaces**](docs/NamespaceApi.md#listNamespaces) | **POST** /ListNamespaces | List namespaces
118+
*NamespaceApi* | [**namespaceExists**](docs/NamespaceApi.md#namespaceExists) | **POST** /NamespaceExists | Check if a namespace exists
119+
*TableApi* | [**getTable**](docs/TableApi.md#getTable) | **POST** /GetTable | Get a table from the namespace
120+
*TableApi* | [**registerTable**](docs/TableApi.md#registerTable) | **POST** /RegisterTable | Register a table to a namespace
121+
*TableApi* | [**tableExists**](docs/TableApi.md#tableExists) | **POST** /TableExists | Check if a table exists
122122

123123

124124
## Documentation for Models
125125

126126
- [CreateNamespaceRequest](docs/CreateNamespaceRequest.md)
127+
- [DropNamespaceRequest](docs/DropNamespaceRequest.md)
127128
- [ErrorResponse](docs/ErrorResponse.md)
129+
- [GetNamespaceRequest](docs/GetNamespaceRequest.md)
128130
- [GetNamespaceResponse](docs/GetNamespaceResponse.md)
131+
- [GetTableRequest](docs/GetTableRequest.md)
129132
- [GetTableResponse](docs/GetTableResponse.md)
133+
- [ListNamespacesRequest](docs/ListNamespacesRequest.md)
130134
- [ListNamespacesResponse](docs/ListNamespacesResponse.md)
135+
- [NamespaceExistsRequest](docs/NamespaceExistsRequest.md)
131136
- [RegisterTableRequest](docs/RegisterTableRequest.md)
137+
- [RegisterTableResponse](docs/RegisterTableResponse.md)
138+
- [TableExistsRequest](docs/TableExistsRequest.md)
132139

133140

134141
<a id="documentation-for-authorization"></a>

0 commit comments

Comments
 (0)