diff --git a/README.md b/README.md index b2f0f4633..869f48704 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,9 @@ -# Lance Catalog - -**Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. -It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. -to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, -which is a REST server that converts the Lance catalog requests to native requests against the catalog service. -Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, -and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. +# Lance Namespace Specification + +**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format +to standardize access to a collection of Lance tables (a.k.a. Lance datasets). +It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. +should store and use Lance tables, as well as how ML/AI tools and analytics compute engines should integrate with Lance tables. ## Repository structure diff --git a/java/Makefile b/java/Makefile index 1c268a70c..cd1a87b31 100644 --- a/java/Makefile +++ b/java/Makefile @@ -17,7 +17,7 @@ clean-java-apache-client: gen-java-apache-client: clean-java-apache-client openapi-generator-cli generate \ - -i ../spec/catalog.yaml \ + -i ../spec/rest.yaml \ -g java \ -o lance-catalog-apache-client \ --ignore-file-override=.apache-client-ignore \ @@ -36,7 +36,7 @@ clean-java-springboot-server: gen-java-springboot-server: clean-java-springboot-server openapi-generator-cli generate \ - -i ../spec/catalog.yaml \ + -i ../spec/rest.yaml \ -g spring \ -o lance-catalog-springboot-server \ --additional-properties=groupId=com.lancedb,artifactId=lance-catalog-springboot-server,artifactVersion=$(VERSION),parentGroupId=com.lancedb,parentArtifactId=lance-catalog-root,parentVersion=$(VERSION),parentRelativePath=pom.xml,library=spring-boot,interfaceOnly=true,useOptional=true,openApiNullable=false,java8=true,apiPackage=com.lancedb.lance.catalog.server.springboot.api,modelPackage=com.lancedb.lance.catalog.server.springboot.model,useTags=true,skipDefaultInterface=false,hideGenerationTimestamp=true,licenseName=Apache-2.0,licenseUrl=https://www.apache.org/licenses/LICENSE-2.0.txt diff --git a/java/lance-catalog-apache-client/.openapi-generator/FILES b/java/lance-catalog-apache-client/.openapi-generator/FILES index cb4eaff9b..4b9970db6 100644 --- a/java/lance-catalog-apache-client/.openapi-generator/FILES +++ b/java/lance-catalog-apache-client/.openapi-generator/FILES @@ -2,7 +2,6 @@ README.md api/openapi.yaml docs/CreateNamespaceRequest.md -docs/CreateNamespaceResponse.md docs/ErrorResponse.md docs/GetNamespaceResponse.md docs/GetTableResponse.md @@ -29,7 +28,6 @@ src/main/java/com/lancedb/lance/catalog/client/apache/auth/Authentication.java src/main/java/com/lancedb/lance/catalog/client/apache/auth/HttpBasicAuth.java src/main/java/com/lancedb/lance/catalog/client/apache/auth/HttpBearerAuth.java src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceRequest.java -src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceResponse.java src/main/java/com/lancedb/lance/catalog/client/apache/model/ErrorResponse.java src/main/java/com/lancedb/lance/catalog/client/apache/model/GetNamespaceResponse.java src/main/java/com/lancedb/lance/catalog/client/apache/model/GetTableResponse.java diff --git a/java/lance-catalog-apache-client/README.md b/java/lance-catalog-apache-client/README.md index 52c006f12..9a0d7caf7 100644 --- a/java/lance-catalog-apache-client/README.md +++ b/java/lance-catalog-apache-client/README.md @@ -1,17 +1,14 @@ # lance-catalog-apache-client -Lance Catalog REST Specification +Lance REST Namespace Specification - API version: 0.0.1 - Generator version: 7.12.0 -**Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. -It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. -to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, -which is a REST server that converts the Lance catalog requests to native requests against the catalog service. -Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, -and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. +**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. +A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. +In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). @@ -94,7 +91,7 @@ public class NamespaceApiExample { NamespaceApi apiInstance = new NamespaceApi(defaultClient); CreateNamespaceRequest createNamespaceRequest = new CreateNamespaceRequest(); // CreateNamespaceRequest | try { - CreateNamespaceResponse result = apiInstance.createNamespace(createNamespaceRequest); + GetNamespaceResponse result = apiInstance.createNamespace(createNamespaceRequest); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling NamespaceApi#createNamespace"); @@ -114,20 +111,19 @@ All URIs are relative to *http://localhost:2333* Class | Method | HTTP request | Description ------------ | ------------- | ------------- | ------------- -*NamespaceApi* | [**createNamespace**](docs/NamespaceApi.md#createNamespace) | **POST** /v1/namespaces | Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. -*NamespaceApi* | [**dropNamespace**](docs/NamespaceApi.md#dropNamespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace from the catalog. Namespace must be empty. -*NamespaceApi* | [**getNamespace**](docs/NamespaceApi.md#getNamespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace -*NamespaceApi* | [**listNamespaces**](docs/NamespaceApi.md#listNamespaces) | **GET** /v1/namespaces | List all namespaces in the catalog. -*NamespaceApi* | [**namespaceExists**](docs/NamespaceApi.md#namespaceExists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists -*TableApi* | [**getTable**](docs/TableApi.md#getTable) | **GET** /v1/namespaces/{ns}/tables/{table} | Get a table from the catalog -*TableApi* | [**registerTable**](docs/TableApi.md#registerTable) | **POST** /v1/namespaces/{ns}/register | Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. -*TableApi* | [**tableExists**](docs/TableApi.md#tableExists) | **HEAD** /v1/namespaces/{ns}/tables/{table} | Check if a table exists +*NamespaceApi* | [**createNamespace**](docs/NamespaceApi.md#createNamespace) | **POST** /v1/namespaces | Create a new namespace +*NamespaceApi* | [**dropNamespace**](docs/NamespaceApi.md#dropNamespace) | **DELETE** /v1/namespaces/{namespace} | Drop a namespace +*NamespaceApi* | [**getNamespace**](docs/NamespaceApi.md#getNamespace) | **GET** /v1/namespaces/{namespace} | Get information about a namespace +*NamespaceApi* | [**listNamespaces**](docs/NamespaceApi.md#listNamespaces) | **GET** /v1/namespaces | List namespaces +*NamespaceApi* | [**namespaceExists**](docs/NamespaceApi.md#namespaceExists) | **HEAD** /v1/namespaces/{namespace} | Check if a namespace exists +*TableApi* | [**getTable**](docs/TableApi.md#getTable) | **GET** /v1/tables/{table} | Get a table from the namespace +*TableApi* | [**registerTable**](docs/TableApi.md#registerTable) | **POST** /v1/table/register | Register a table to a namespace +*TableApi* | [**tableExists**](docs/TableApi.md#tableExists) | **HEAD** /v1/tables/{table} | Check if a table exists ## Documentation for Models - [CreateNamespaceRequest](docs/CreateNamespaceRequest.md) - - [CreateNamespaceResponse](docs/CreateNamespaceResponse.md) - [ErrorResponse](docs/ErrorResponse.md) - [GetNamespaceResponse](docs/GetNamespaceResponse.md) - [GetTableResponse](docs/GetTableResponse.md) diff --git a/java/lance-catalog-apache-client/api/openapi.yaml b/java/lance-catalog-apache-client/api/openapi.yaml index 9e27bd717..5dc06fe9a 100644 --- a/java/lance-catalog-apache-client/api/openapi.yaml +++ b/java/lance-catalog-apache-client/api/openapi.yaml @@ -1,16 +1,13 @@ openapi: 3.1.1 info: description: | - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. - It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. - to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, - which is a REST server that converts the Lance catalog requests to native requests against the catalog service. - Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, - and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _"tools"_ in this document) should integrate with Lance tables. + A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. + In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). license: name: Apache 2.0 url: https://www.apache.org/licenses/LICENSE-2.0.html - title: Lance Catalog REST Specification + title: Lance REST Namespace Specification version: 0.0.1 servers: - description: Generic server URL with all parts configurable @@ -39,10 +36,14 @@ security: paths: /v1/namespaces: get: + description: | + List all child namespace names of the root namespace or a given parent namespace. operationId: ListNamespaces parameters: - $ref: '#/components/parameters/pageToken' - $ref: '#/components/parameters/pageSize' + - $ref: '#/components/parameters/parent' + - $ref: '#/components/parameters/delimiter' responses: "200": $ref: '#/components/responses/ListNamespacesResponse' @@ -58,13 +59,19 @@ paths: $ref: '#/components/responses/ServiceUnavailableErrorResponse' "5XX": $ref: '#/components/responses/ServerErrorResponse' - summary: | - List all namespaces in the catalog. + summary: List namespaces tags: - Namespace x-accepts: - application/json post: + description: | + Create a new namespace. + A namespace can manage either a collection of child namespaces, or a collection of tables. + There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: + * CREATE: the operation fails with 400. + * EXIST_OK: the operation succeeds and the existing namespace is kept. + * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. operationId: CreateNamespace requestBody: content: @@ -74,7 +81,7 @@ paths: required: true responses: "200": - $ref: '#/components/responses/CreateNamespaceResponse' + $ref: '#/components/responses/GetNamespaceResponse' "400": $ref: '#/components/responses/BadRequestErrorResponse' "401": @@ -89,21 +96,20 @@ paths: $ref: '#/components/responses/ServiceUnavailableErrorResponse' "5XX": $ref: '#/components/responses/ServerErrorResponse' - summary: | - Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: - * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. - * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. - * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + summary: Create a new namespace tags: - Namespace x-content-type: application/json x-accepts: - application/json - /v1/namespaces/{ns}: + /v1/namespaces/{namespace}: delete: + description: | + Drop a namespace. The namespace must be empty. operationId: DropNamespace parameters: - $ref: '#/components/parameters/namespace' + - $ref: '#/components/parameters/delimiter' responses: "204": description: "Success, no content" @@ -121,16 +127,18 @@ paths: $ref: '#/components/responses/ServiceUnavailableErrorResponse' "5XX": $ref: '#/components/responses/ServerErrorResponse' - summary: Drop a namespace from the catalog. Namespace must be empty. + summary: Drop a namespace tags: - Namespace x-accepts: - application/json get: - description: Return a detailed information for a given namespace + description: | + Return the detailed information for a given namespace operationId: GetNamespace parameters: - $ref: '#/components/parameters/namespace' + - $ref: '#/components/parameters/delimiter' responses: "200": $ref: '#/components/responses/GetNamespaceResponse' @@ -152,10 +160,13 @@ paths: x-accepts: - application/json head: - description: Check if a namespace exists. The response does not contain a body. + description: | + Check if a namespace exists. + This API should behave exactly like the GetNamespace API, except it does not contain a body. operationId: NamespaceExists parameters: - $ref: '#/components/parameters/namespace' + - $ref: '#/components/parameters/delimiter' responses: "200": description: "Success, no content" @@ -178,11 +189,12 @@ paths: - application/json parameters: - $ref: '#/components/parameters/namespace' - /v1/namespaces/{ns}/register: + - $ref: '#/components/parameters/delimiter' + /v1/table/register: post: + description: | + Register an existing table at a given storage location to a namespace. operationId: RegisterTable - parameters: - - $ref: '#/components/parameters/namespace' requestBody: content: application/json: @@ -206,21 +218,20 @@ paths: $ref: '#/components/responses/ServiceUnavailableErrorResponse' "5XX": $ref: '#/components/responses/ServerErrorResponse' - summary: | - Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. + summary: Register a table to a namespace tags: - Table x-content-type: application/json x-accepts: - application/json - /v1/namespaces/{ns}/tables/{table}: + /v1/tables/{table}: get: - description: Get a table's detailed information under a specified namespace - from the catalog. + description: | + Get a table's detailed information. operationId: GetTable parameters: - - $ref: '#/components/parameters/namespace' - $ref: '#/components/parameters/table' + - $ref: '#/components/parameters/delimiter' responses: "200": $ref: '#/components/responses/GetTableResponse' @@ -236,17 +247,19 @@ paths: $ref: '#/components/responses/ServiceUnavailableErrorResponse' "5XX": $ref: '#/components/responses/ServerErrorResponse' - summary: Get a table from the catalog + summary: Get a table from the namespace tags: - Table x-accepts: - application/json head: - description: Check if a table exists within a given namespace. + description: | + Check if a table exists. + This API should behave exactly like the GetTable API, except it does not contain a body. operationId: TableExists parameters: - - $ref: '#/components/parameters/namespace' - $ref: '#/components/parameters/table' + - $ref: '#/components/parameters/delimiter' responses: "200": description: "Success, no content" @@ -268,8 +281,8 @@ paths: x-accepts: - application/json parameters: - - $ref: '#/components/parameters/namespace' - $ref: '#/components/parameters/table' + - $ref: '#/components/parameters/delimiter' components: examples: ListNamespacesEmptyExample: @@ -284,14 +297,23 @@ components: - credits parameters: namespace: - description: The name of the namespace. + description: A string identifier of the namespace. explode: false in: path - name: ns + name: namespace required: true schema: type: string style: simple + parent: + description: A string identifier of the parent namespace. + explode: true + in: query + name: parent + required: false + schema: + type: string + style: form pageToken: allowEmptyValue: true explode: true @@ -313,7 +335,7 @@ components: type: integer style: form table: - description: A table name. + description: A string identifier of the table explode: false in: path name: table @@ -321,21 +343,16 @@ components: schema: type: string style: simple + delimiter: + description: The delimiter for the identifier used in the context + explode: true + in: query + name: delimiter + required: false + schema: + type: string + style: form responses: - CreateNamespaceResponse: - content: - application/json: - example: - namespace: - - lance.ns1 - properties: - created_at: "1452120468" - schema: - $ref: '#/components/schemas/CreateNamespaceResponse' - description: "Represents a successful call to create a namespace. Returns the\ - \ namespace created, as well as any properties that were stored for the namespace,\ - \ including those the server might have added. Implementations are not required\ - \ to support namespace properties." ListNamespacesResponse: content: application/json: @@ -408,7 +425,7 @@ components: title: Not found Error status: 404 detail: "" - instance: "/v1/namespaces/{ns}" + instance: "/v1/namespaces/{namespace}" schema: $ref: '#/components/schemas/ErrorResponse' description: A server-side problem that means can not find the specified resource. @@ -433,7 +450,7 @@ components: title: The namespace has been concurrently modified status: 409 detail: "" - instance: "/v1/namespaces/{ns}" + instance: "/v1/namespaces/{namespace}" schema: $ref: '#/components/schemas/ErrorResponse' description: The request conflicts with the current state of the target resource. @@ -503,12 +520,19 @@ components: CreateNamespaceRequest: example: mode: CREATE + parent: + - parent + - parent name: name options: key: options properties: name: type: string + parent: + items: + type: string + type: array mode: enum: - CREATE @@ -521,23 +545,6 @@ components: required: - mode - name - CreateNamespaceResponse: - example: - name: name - properties: - created_at: "1452120468" - properties: - name: - type: string - properties: - additionalProperties: - type: string - default: {} - description: "Properties stored on the namespace, if supported by the server." - example: - created_at: "1452120468" - required: - - name ListNamespacesResponse: example: nextPageToken: nextPageToken @@ -546,7 +553,6 @@ components: - accounting properties: namespaces: - description: An array of namespace names in the catalog. items: example: accounting type: string @@ -564,13 +570,20 @@ components: - namespaces GetNamespaceResponse: example: - namespace: namespace + parent: + - parent + - parent + name: name properties: owner: Ralph created_at: "1452120468" properties: - namespace: + name: type: string + parent: + items: + type: string + type: array properties: additionalProperties: type: string @@ -584,7 +597,7 @@ components: created_at: "1452120468" nullable: true required: - - namespace + - name PageToken: description: |- An opaque token that allows pagination for list APIs (e.g. ListNamespaces). @@ -596,26 +609,39 @@ components: RegisterTableRequest: example: name: name + namespace: + - namespace + - namespace location: location properties: name: type: string + namespace: + items: + type: string + type: array location: type: string required: - location - name + - namespace GetTableResponse: - description: | - Result used when a table is successfully loaded. example: name: name + namespace: + - namespace + - namespace location: location properties: key: properties properties: name: type: string + namespace: + items: + type: string + type: array location: type: string properties: @@ -624,4 +650,5 @@ components: required: - location - name + - namespace diff --git a/java/lance-catalog-apache-client/docs/CreateNamespaceRequest.md b/java/lance-catalog-apache-client/docs/CreateNamespaceRequest.md index 84880dcde..48cc592ba 100644 --- a/java/lance-catalog-apache-client/docs/CreateNamespaceRequest.md +++ b/java/lance-catalog-apache-client/docs/CreateNamespaceRequest.md @@ -8,6 +8,7 @@ | Name | Type | Description | Notes | |------------ | ------------- | ------------- | -------------| |**name** | **String** | | | +|**parent** | **List<String>** | | [optional] | |**mode** | [**ModeEnum**](#ModeEnum) | | | |**options** | **Map<String, String>** | | [optional] | diff --git a/java/lance-catalog-apache-client/docs/CreateNamespaceResponse.md b/java/lance-catalog-apache-client/docs/CreateNamespaceResponse.md deleted file mode 100644 index eee7db5ca..000000000 --- a/java/lance-catalog-apache-client/docs/CreateNamespaceResponse.md +++ /dev/null @@ -1,14 +0,0 @@ - - -# CreateNamespaceResponse - - -## Properties - -| Name | Type | Description | Notes | -|------------ | ------------- | ------------- | -------------| -|**name** | **String** | | | -|**properties** | **Map<String, String>** | Properties stored on the namespace, if supported by the server. | [optional] | - - - diff --git a/java/lance-catalog-apache-client/docs/GetNamespaceResponse.md b/java/lance-catalog-apache-client/docs/GetNamespaceResponse.md index e43cd083c..9a36c80d3 100644 --- a/java/lance-catalog-apache-client/docs/GetNamespaceResponse.md +++ b/java/lance-catalog-apache-client/docs/GetNamespaceResponse.md @@ -7,7 +7,8 @@ | Name | Type | Description | Notes | |------------ | ------------- | ------------- | -------------| -|**namespace** | **String** | | | +|**name** | **String** | | | +|**parent** | **List<String>** | | [optional] | |**properties** | **Map<String, String>** | Properties stored on the namespace, if supported by the server. If the server does not support namespace properties, it should return null for this field. If namespace properties are supported, but none are set, it should return an empty object. | [optional] | diff --git a/java/lance-catalog-apache-client/docs/GetTableResponse.md b/java/lance-catalog-apache-client/docs/GetTableResponse.md index d7256422c..8ab2701e1 100644 --- a/java/lance-catalog-apache-client/docs/GetTableResponse.md +++ b/java/lance-catalog-apache-client/docs/GetTableResponse.md @@ -2,13 +2,13 @@ # GetTableResponse -Result used when a table is successfully loaded. ## Properties | Name | Type | Description | Notes | |------------ | ------------- | ------------- | -------------| |**name** | **String** | | | +|**namespace** | **List<String>** | | | |**location** | **String** | | | |**properties** | **Map<String, String>** | | [optional] | diff --git a/java/lance-catalog-apache-client/docs/ListNamespacesResponse.md b/java/lance-catalog-apache-client/docs/ListNamespacesResponse.md index 7197467be..bf1e41982 100644 --- a/java/lance-catalog-apache-client/docs/ListNamespacesResponse.md +++ b/java/lance-catalog-apache-client/docs/ListNamespacesResponse.md @@ -7,7 +7,7 @@ | Name | Type | Description | Notes | |------------ | ------------- | ------------- | -------------| -|**namespaces** | **Set<String>** | An array of namespace names in the catalog. | | +|**namespaces** | **Set<String>** | | | |**nextPageToken** | **String** | An opaque token that allows pagination for list APIs (e.g. ListNamespaces). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results. | [optional] | diff --git a/java/lance-catalog-apache-client/docs/NamespaceApi.md b/java/lance-catalog-apache-client/docs/NamespaceApi.md index c47fc1251..2adbc982d 100644 --- a/java/lance-catalog-apache-client/docs/NamespaceApi.md +++ b/java/lance-catalog-apache-client/docs/NamespaceApi.md @@ -4,19 +4,21 @@ All URIs are relative to *http://localhost:2333* | Method | HTTP request | Description | |------------- | ------------- | -------------| -| [**createNamespace**](NamespaceApi.md#createNamespace) | **POST** /v1/namespaces | Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. | -| [**dropNamespace**](NamespaceApi.md#dropNamespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace from the catalog. Namespace must be empty. | -| [**getNamespace**](NamespaceApi.md#getNamespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace | -| [**listNamespaces**](NamespaceApi.md#listNamespaces) | **GET** /v1/namespaces | List all namespaces in the catalog. | -| [**namespaceExists**](NamespaceApi.md#namespaceExists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists | +| [**createNamespace**](NamespaceApi.md#createNamespace) | **POST** /v1/namespaces | Create a new namespace | +| [**dropNamespace**](NamespaceApi.md#dropNamespace) | **DELETE** /v1/namespaces/{namespace} | Drop a namespace | +| [**getNamespace**](NamespaceApi.md#getNamespace) | **GET** /v1/namespaces/{namespace} | Get information about a namespace | +| [**listNamespaces**](NamespaceApi.md#listNamespaces) | **GET** /v1/namespaces | List namespaces | +| [**namespaceExists**](NamespaceApi.md#namespaceExists) | **HEAD** /v1/namespaces/{namespace} | Check if a namespace exists | ## createNamespace -> CreateNamespaceResponse createNamespace(createNamespaceRequest) +> GetNamespaceResponse createNamespace(createNamespaceRequest) -Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. +Create a new namespace + +Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. ### Example @@ -36,7 +38,7 @@ public class Example { NamespaceApi apiInstance = new NamespaceApi(defaultClient); CreateNamespaceRequest createNamespaceRequest = new CreateNamespaceRequest(); // CreateNamespaceRequest | try { - CreateNamespaceResponse result = apiInstance.createNamespace(createNamespaceRequest); + GetNamespaceResponse result = apiInstance.createNamespace(createNamespaceRequest); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling NamespaceApi#createNamespace"); @@ -58,7 +60,7 @@ public class Example { ### Return type -[**CreateNamespaceResponse**](CreateNamespaceResponse.md) +[**GetNamespaceResponse**](GetNamespaceResponse.md) ### Authorization @@ -73,7 +75,7 @@ No authorization required ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| -| **200** | Represents a successful call to create a namespace. Returns the namespace created, as well as any properties that were stored for the namespace, including those the server might have added. Implementations are not required to support namespace properties. | - | +| **200** | Returns a namespace, as well as any properties stored on the namespace if namespace properties are supported by the server. | - | | **400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | | **401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | | **403** | Forbidden. Authenticated user does not have the necessary permissions. | - | @@ -85,9 +87,11 @@ No authorization required ## dropNamespace -> dropNamespace(ns) +> dropNamespace(namespace, delimiter) -Drop a namespace from the catalog. Namespace must be empty. +Drop a namespace + +Drop a namespace. The namespace must be empty. ### Example @@ -105,9 +109,10 @@ public class Example { defaultClient.setBasePath("http://localhost:2333"); NamespaceApi apiInstance = new NamespaceApi(defaultClient); - String ns = "ns_example"; // String | The name of the namespace. + String namespace = "namespace_example"; // String | A string identifier of the namespace. + String delimiter = "delimiter_example"; // String | The delimiter for the identifier used in the context try { - apiInstance.dropNamespace(ns); + apiInstance.dropNamespace(namespace, delimiter); } catch (ApiException e) { System.err.println("Exception when calling NamespaceApi#dropNamespace"); System.err.println("Status code: " + e.getCode()); @@ -124,7 +129,8 @@ public class Example { | Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **ns** | **String**| The name of the namespace. | | +| **namespace** | **String**| A string identifier of the namespace. | | +| **delimiter** | **String**| The delimiter for the identifier used in the context | [optional] | ### Return type @@ -155,11 +161,11 @@ No authorization required ## getNamespace -> GetNamespaceResponse getNamespace(ns) +> GetNamespaceResponse getNamespace(namespace, delimiter) Get information about a namespace -Return a detailed information for a given namespace +Return the detailed information for a given namespace ### Example @@ -177,9 +183,10 @@ public class Example { defaultClient.setBasePath("http://localhost:2333"); NamespaceApi apiInstance = new NamespaceApi(defaultClient); - String ns = "ns_example"; // String | The name of the namespace. + String namespace = "namespace_example"; // String | A string identifier of the namespace. + String delimiter = "delimiter_example"; // String | The delimiter for the identifier used in the context try { - GetNamespaceResponse result = apiInstance.getNamespace(ns); + GetNamespaceResponse result = apiInstance.getNamespace(namespace, delimiter); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling NamespaceApi#getNamespace"); @@ -197,7 +204,8 @@ public class Example { | Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **ns** | **String**| The name of the namespace. | | +| **namespace** | **String**| A string identifier of the namespace. | | +| **delimiter** | **String**| The delimiter for the identifier used in the context | [optional] | ### Return type @@ -227,9 +235,11 @@ No authorization required ## listNamespaces -> ListNamespacesResponse listNamespaces(pageToken, pageSize) +> ListNamespacesResponse listNamespaces(pageToken, pageSize, parent, delimiter) + +List namespaces -List all namespaces in the catalog. +List all child namespace names of the root namespace or a given parent namespace. ### Example @@ -249,8 +259,10 @@ public class Example { NamespaceApi apiInstance = new NamespaceApi(defaultClient); String pageToken = "pageToken_example"; // String | Integer pageSize = 56; // Integer | An inclusive upper bound of the number of results that a client will receive. + String parent = "parent_example"; // String | A string identifier of the parent namespace. + String delimiter = "delimiter_example"; // String | The delimiter for the identifier used in the context try { - ListNamespacesResponse result = apiInstance.listNamespaces(pageToken, pageSize); + ListNamespacesResponse result = apiInstance.listNamespaces(pageToken, pageSize, parent, delimiter); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling NamespaceApi#listNamespaces"); @@ -270,6 +282,8 @@ public class Example { |------------- | ------------- | ------------- | -------------| | **pageToken** | **String**| | [optional] | | **pageSize** | **Integer**| An inclusive upper bound of the number of results that a client will receive. | [optional] | +| **parent** | **String**| A string identifier of the parent namespace. | [optional] | +| **delimiter** | **String**| The delimiter for the identifier used in the context | [optional] | ### Return type @@ -299,11 +313,11 @@ No authorization required ## namespaceExists -> namespaceExists(ns) +> namespaceExists(namespace, delimiter) Check if a namespace exists -Check if a namespace exists. The response does not contain a body. +Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. ### Example @@ -321,9 +335,10 @@ public class Example { defaultClient.setBasePath("http://localhost:2333"); NamespaceApi apiInstance = new NamespaceApi(defaultClient); - String ns = "ns_example"; // String | The name of the namespace. + String namespace = "namespace_example"; // String | A string identifier of the namespace. + String delimiter = "delimiter_example"; // String | The delimiter for the identifier used in the context try { - apiInstance.namespaceExists(ns); + apiInstance.namespaceExists(namespace, delimiter); } catch (ApiException e) { System.err.println("Exception when calling NamespaceApi#namespaceExists"); System.err.println("Status code: " + e.getCode()); @@ -340,7 +355,8 @@ public class Example { | Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **ns** | **String**| The name of the namespace. | | +| **namespace** | **String**| A string identifier of the namespace. | | +| **delimiter** | **String**| The delimiter for the identifier used in the context | [optional] | ### Return type diff --git a/java/lance-catalog-apache-client/docs/RegisterTableRequest.md b/java/lance-catalog-apache-client/docs/RegisterTableRequest.md index a37048183..ad485ad29 100644 --- a/java/lance-catalog-apache-client/docs/RegisterTableRequest.md +++ b/java/lance-catalog-apache-client/docs/RegisterTableRequest.md @@ -8,6 +8,7 @@ | Name | Type | Description | Notes | |------------ | ------------- | ------------- | -------------| |**name** | **String** | | | +|**namespace** | **List<String>** | | | |**location** | **String** | | | diff --git a/java/lance-catalog-apache-client/docs/TableApi.md b/java/lance-catalog-apache-client/docs/TableApi.md index 0b2258003..69d76792c 100644 --- a/java/lance-catalog-apache-client/docs/TableApi.md +++ b/java/lance-catalog-apache-client/docs/TableApi.md @@ -4,19 +4,19 @@ All URIs are relative to *http://localhost:2333* | Method | HTTP request | Description | |------------- | ------------- | -------------| -| [**getTable**](TableApi.md#getTable) | **GET** /v1/namespaces/{ns}/tables/{table} | Get a table from the catalog | -| [**registerTable**](TableApi.md#registerTable) | **POST** /v1/namespaces/{ns}/register | Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. | -| [**tableExists**](TableApi.md#tableExists) | **HEAD** /v1/namespaces/{ns}/tables/{table} | Check if a table exists | +| [**getTable**](TableApi.md#getTable) | **GET** /v1/tables/{table} | Get a table from the namespace | +| [**registerTable**](TableApi.md#registerTable) | **POST** /v1/table/register | Register a table to a namespace | +| [**tableExists**](TableApi.md#tableExists) | **HEAD** /v1/tables/{table} | Check if a table exists | ## getTable -> GetTableResponse getTable(ns, table) +> GetTableResponse getTable(table, delimiter) -Get a table from the catalog +Get a table from the namespace -Get a table's detailed information under a specified namespace from the catalog. +Get a table's detailed information. ### Example @@ -34,10 +34,10 @@ public class Example { defaultClient.setBasePath("http://localhost:2333"); TableApi apiInstance = new TableApi(defaultClient); - String ns = "ns_example"; // String | The name of the namespace. - String table = "table_example"; // String | A table name. + String table = "table_example"; // String | A string identifier of the table + String delimiter = "delimiter_example"; // String | The delimiter for the identifier used in the context try { - GetTableResponse result = apiInstance.getTable(ns, table); + GetTableResponse result = apiInstance.getTable(table, delimiter); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling TableApi#getTable"); @@ -55,8 +55,8 @@ public class Example { | Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **ns** | **String**| The name of the namespace. | | -| **table** | **String**| A table name. | | +| **table** | **String**| A string identifier of the table | | +| **delimiter** | **String**| The delimiter for the identifier used in the context | [optional] | ### Return type @@ -86,9 +86,11 @@ No authorization required ## registerTable -> GetTableResponse registerTable(ns, registerTableRequest) +> GetTableResponse registerTable(registerTableRequest) -Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. +Register a table to a namespace + +Register an existing table at a given storage location to a namespace. ### Example @@ -106,10 +108,9 @@ public class Example { defaultClient.setBasePath("http://localhost:2333"); TableApi apiInstance = new TableApi(defaultClient); - String ns = "ns_example"; // String | The name of the namespace. RegisterTableRequest registerTableRequest = new RegisterTableRequest(); // RegisterTableRequest | try { - GetTableResponse result = apiInstance.registerTable(ns, registerTableRequest); + GetTableResponse result = apiInstance.registerTable(registerTableRequest); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling TableApi#registerTable"); @@ -127,7 +128,6 @@ public class Example { | Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **ns** | **String**| The name of the namespace. | | | **registerTableRequest** | [**RegisterTableRequest**](RegisterTableRequest.md)| | | ### Return type @@ -159,11 +159,11 @@ No authorization required ## tableExists -> tableExists(ns, table) +> tableExists(table, delimiter) Check if a table exists -Check if a table exists within a given namespace. +Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. ### Example @@ -181,10 +181,10 @@ public class Example { defaultClient.setBasePath("http://localhost:2333"); TableApi apiInstance = new TableApi(defaultClient); - String ns = "ns_example"; // String | The name of the namespace. - String table = "table_example"; // String | A table name. + String table = "table_example"; // String | A string identifier of the table + String delimiter = "delimiter_example"; // String | The delimiter for the identifier used in the context try { - apiInstance.tableExists(ns, table); + apiInstance.tableExists(table, delimiter); } catch (ApiException e) { System.err.println("Exception when calling TableApi#tableExists"); System.err.println("Status code: " + e.getCode()); @@ -201,8 +201,8 @@ public class Example { | Name | Type | Description | Notes | |------------- | ------------- | ------------- | -------------| -| **ns** | **String**| The name of the namespace. | | -| **table** | **String**| A table name. | | +| **table** | **String**| A string identifier of the table | | +| **delimiter** | **String**| The delimiter for the identifier used in the context | [optional] | ### Return type diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/NamespaceApi.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/NamespaceApi.java index 244206770..da49dddeb 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/NamespaceApi.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/NamespaceApi.java @@ -19,7 +19,6 @@ import com.lancedb.lance.catalog.client.apache.Configuration; import com.lancedb.lance.catalog.client.apache.Pair; import com.lancedb.lance.catalog.client.apache.model.CreateNamespaceRequest; -import com.lancedb.lance.catalog.client.apache.model.CreateNamespaceResponse; import com.lancedb.lance.catalog.client.apache.model.GetNamespaceResponse; import com.lancedb.lance.catalog.client.apache.model.ListNamespacesResponse; @@ -46,40 +45,36 @@ public NamespaceApi(ApiClient apiClient) { } /** - * Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to - * manage one or more tables. There are three modes when trying to create a namespace: * CREATE: - * Create the namespace if it does not exist. If a namespace of the same name already exists, the - * operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace - * of the same name already exists, the operation succeeds and the existing namespace is kept. * - * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already - * exists, the existing namespace is dropped and a new namespace with this name with no table is - * created. + * Create a new namespace Create a new namespace. A namespace can manage either a collection of + * child namespaces, or a collection of tables. There are three modes when trying to create a + * namespace, to differentiate the behavior when a namespace of the same name already exists: * + * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing + * namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace + * with this name is created. * * @param createNamespaceRequest (required) - * @return CreateNamespaceResponse + * @return GetNamespaceResponse * @throws ApiException if fails to make API call */ - public CreateNamespaceResponse createNamespace(CreateNamespaceRequest createNamespaceRequest) + public GetNamespaceResponse createNamespace(CreateNamespaceRequest createNamespaceRequest) throws ApiException { return this.createNamespace(createNamespaceRequest, Collections.emptyMap()); } /** - * Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to - * manage one or more tables. There are three modes when trying to create a namespace: * CREATE: - * Create the namespace if it does not exist. If a namespace of the same name already exists, the - * operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace - * of the same name already exists, the operation succeeds and the existing namespace is kept. * - * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already - * exists, the existing namespace is dropped and a new namespace with this name with no table is - * created. + * Create a new namespace Create a new namespace. A namespace can manage either a collection of + * child namespaces, or a collection of tables. There are three modes when trying to create a + * namespace, to differentiate the behavior when a namespace of the same name already exists: * + * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing + * namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace + * with this name is created. * * @param createNamespaceRequest (required) * @param additionalHeaders additionalHeaders for this call - * @return CreateNamespaceResponse + * @return GetNamespaceResponse * @throws ApiException if fails to make API call */ - public CreateNamespaceResponse createNamespace( + public GetNamespaceResponse createNamespace( CreateNamespaceRequest createNamespaceRequest, Map additionalHeaders) throws ApiException { Object localVarPostBody = createNamespaceRequest; @@ -112,8 +107,8 @@ public CreateNamespaceResponse createNamespace( String[] localVarAuthNames = new String[] {}; - TypeReference localVarReturnType = - new TypeReference() {}; + TypeReference localVarReturnType = + new TypeReference() {}; return apiClient.invokeAPI( localVarPath, "POST", @@ -131,35 +126,41 @@ public CreateNamespaceResponse createNamespace( } /** - * Drop a namespace from the catalog. Namespace must be empty. + * Drop a namespace Drop a namespace. The namespace must be empty. * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @throws ApiException if fails to make API call */ - public void dropNamespace(String ns) throws ApiException { - this.dropNamespace(ns, Collections.emptyMap()); + public void dropNamespace(String namespace, String delimiter) throws ApiException { + this.dropNamespace(namespace, delimiter, Collections.emptyMap()); } /** - * Drop a namespace from the catalog. Namespace must be empty. + * Drop a namespace Drop a namespace. The namespace must be empty. * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @param additionalHeaders additionalHeaders for this call * @throws ApiException if fails to make API call */ - public void dropNamespace(String ns, Map additionalHeaders) throws ApiException { + public void dropNamespace( + String namespace, String delimiter, Map additionalHeaders) + throws ApiException { Object localVarPostBody = null; - // verify the required parameter 'ns' is set - if (ns == null) { - throw new ApiException(400, "Missing the required parameter 'ns' when calling dropNamespace"); + // verify the required parameter 'namespace' is set + if (namespace == null) { + throw new ApiException( + 400, "Missing the required parameter 'namespace' when calling dropNamespace"); } // create path and map variables String localVarPath = - "/v1/namespaces/{ns}" + "/v1/namespaces/{namespace}" .replaceAll( - "\\{" + "ns" + "\\}", apiClient.escapeString(apiClient.parameterToString(ns))); + "\\{" + "namespace" + "\\}", + apiClient.escapeString(apiClient.parameterToString(namespace))); StringJoiner localVarQueryStringJoiner = new StringJoiner("&"); String localVarQueryParameterBaseName; @@ -169,6 +170,8 @@ public void dropNamespace(String ns, Map additionalHeaders) thro Map localVarCookieParams = new HashMap(); Map localVarFormParams = new HashMap(); + localVarQueryParams.addAll(apiClient.parameterToPair("delimiter", delimiter)); + localVarHeaderParams.putAll(additionalHeaders); final String[] localVarAccepts = {"application/json"}; @@ -197,38 +200,43 @@ public void dropNamespace(String ns, Map additionalHeaders) thro } /** - * Get information about a namespace Return a detailed information for a given namespace + * Get information about a namespace Return the detailed information for a given namespace * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return GetNamespaceResponse * @throws ApiException if fails to make API call */ - public GetNamespaceResponse getNamespace(String ns) throws ApiException { - return this.getNamespace(ns, Collections.emptyMap()); + public GetNamespaceResponse getNamespace(String namespace, String delimiter) throws ApiException { + return this.getNamespace(namespace, delimiter, Collections.emptyMap()); } /** - * Get information about a namespace Return a detailed information for a given namespace + * Get information about a namespace Return the detailed information for a given namespace * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @param additionalHeaders additionalHeaders for this call * @return GetNamespaceResponse * @throws ApiException if fails to make API call */ - public GetNamespaceResponse getNamespace(String ns, Map additionalHeaders) + public GetNamespaceResponse getNamespace( + String namespace, String delimiter, Map additionalHeaders) throws ApiException { Object localVarPostBody = null; - // verify the required parameter 'ns' is set - if (ns == null) { - throw new ApiException(400, "Missing the required parameter 'ns' when calling getNamespace"); + // verify the required parameter 'namespace' is set + if (namespace == null) { + throw new ApiException( + 400, "Missing the required parameter 'namespace' when calling getNamespace"); } // create path and map variables String localVarPath = - "/v1/namespaces/{ns}" + "/v1/namespaces/{namespace}" .replaceAll( - "\\{" + "ns" + "\\}", apiClient.escapeString(apiClient.parameterToString(ns))); + "\\{" + "namespace" + "\\}", + apiClient.escapeString(apiClient.parameterToString(namespace))); StringJoiner localVarQueryStringJoiner = new StringJoiner("&"); String localVarQueryParameterBaseName; @@ -238,6 +246,8 @@ public GetNamespaceResponse getNamespace(String ns, Map addition Map localVarCookieParams = new HashMap(); Map localVarFormParams = new HashMap(); + localVarQueryParams.addAll(apiClient.parameterToPair("delimiter", delimiter)); + localVarHeaderParams.putAll(additionalHeaders); final String[] localVarAccepts = {"application/json"}; @@ -268,31 +278,41 @@ public GetNamespaceResponse getNamespace(String ns, Map addition } /** - * List all namespaces in the catalog. + * List namespaces List all child namespace names of the root namespace or a given parent + * namespace. * * @param pageToken (optional) * @param pageSize An inclusive upper bound of the number of results that a client will receive. * (optional) + * @param parent A string identifier of the parent namespace. (optional) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return ListNamespacesResponse * @throws ApiException if fails to make API call */ - public ListNamespacesResponse listNamespaces(String pageToken, Integer pageSize) - throws ApiException { - return this.listNamespaces(pageToken, pageSize, Collections.emptyMap()); + public ListNamespacesResponse listNamespaces( + String pageToken, Integer pageSize, String parent, String delimiter) throws ApiException { + return this.listNamespaces(pageToken, pageSize, parent, delimiter, Collections.emptyMap()); } /** - * List all namespaces in the catalog. + * List namespaces List all child namespace names of the root namespace or a given parent + * namespace. * * @param pageToken (optional) * @param pageSize An inclusive upper bound of the number of results that a client will receive. * (optional) + * @param parent A string identifier of the parent namespace. (optional) + * @param delimiter The delimiter for the identifier used in the context (optional) * @param additionalHeaders additionalHeaders for this call * @return ListNamespacesResponse * @throws ApiException if fails to make API call */ public ListNamespacesResponse listNamespaces( - String pageToken, Integer pageSize, Map additionalHeaders) + String pageToken, + Integer pageSize, + String parent, + String delimiter, + Map additionalHeaders) throws ApiException { Object localVarPostBody = null; @@ -309,6 +329,8 @@ public ListNamespacesResponse listNamespaces( localVarQueryParams.addAll(apiClient.parameterToPair("pageToken", pageToken)); localVarQueryParams.addAll(apiClient.parameterToPair("pageSize", pageSize)); + localVarQueryParams.addAll(apiClient.parameterToPair("parent", parent)); + localVarQueryParams.addAll(apiClient.parameterToPair("delimiter", delimiter)); localVarHeaderParams.putAll(additionalHeaders); @@ -340,37 +362,43 @@ public ListNamespacesResponse listNamespaces( } /** - * Check if a namespace exists Check if a namespace exists. The response does not contain a body. + * Check if a namespace exists Check if a namespace exists. This API should behave exactly like + * the GetNamespace API, except it does not contain a body. * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @throws ApiException if fails to make API call */ - public void namespaceExists(String ns) throws ApiException { - this.namespaceExists(ns, Collections.emptyMap()); + public void namespaceExists(String namespace, String delimiter) throws ApiException { + this.namespaceExists(namespace, delimiter, Collections.emptyMap()); } /** - * Check if a namespace exists Check if a namespace exists. The response does not contain a body. + * Check if a namespace exists Check if a namespace exists. This API should behave exactly like + * the GetNamespace API, except it does not contain a body. * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @param additionalHeaders additionalHeaders for this call * @throws ApiException if fails to make API call */ - public void namespaceExists(String ns, Map additionalHeaders) + public void namespaceExists( + String namespace, String delimiter, Map additionalHeaders) throws ApiException { Object localVarPostBody = null; - // verify the required parameter 'ns' is set - if (ns == null) { + // verify the required parameter 'namespace' is set + if (namespace == null) { throw new ApiException( - 400, "Missing the required parameter 'ns' when calling namespaceExists"); + 400, "Missing the required parameter 'namespace' when calling namespaceExists"); } // create path and map variables String localVarPath = - "/v1/namespaces/{ns}" + "/v1/namespaces/{namespace}" .replaceAll( - "\\{" + "ns" + "\\}", apiClient.escapeString(apiClient.parameterToString(ns))); + "\\{" + "namespace" + "\\}", + apiClient.escapeString(apiClient.parameterToString(namespace))); StringJoiner localVarQueryStringJoiner = new StringJoiner("&"); String localVarQueryParameterBaseName; @@ -380,6 +408,8 @@ public void namespaceExists(String ns, Map additionalHeaders) Map localVarCookieParams = new HashMap(); Map localVarFormParams = new HashMap(); + localVarQueryParams.addAll(apiClient.parameterToPair("delimiter", delimiter)); + localVarHeaderParams.putAll(additionalHeaders); final String[] localVarAccepts = {"application/json"}; diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/TableApi.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/TableApi.java index b04755d27..13514a518 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/TableApi.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/api/TableApi.java @@ -44,37 +44,30 @@ public TableApi(ApiClient apiClient) { } /** - * Get a table from the catalog Get a table's detailed information under a specified namespace - * from the catalog. + * Get a table from the namespace Get a table's detailed information. * - * @param ns The name of the namespace. (required) - * @param table A table name. (required) + * @param table A string identifier of the table (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return GetTableResponse * @throws ApiException if fails to make API call */ - public GetTableResponse getTable(String ns, String table) throws ApiException { - return this.getTable(ns, table, Collections.emptyMap()); + public GetTableResponse getTable(String table, String delimiter) throws ApiException { + return this.getTable(table, delimiter, Collections.emptyMap()); } /** - * Get a table from the catalog Get a table's detailed information under a specified namespace - * from the catalog. + * Get a table from the namespace Get a table's detailed information. * - * @param ns The name of the namespace. (required) - * @param table A table name. (required) + * @param table A string identifier of the table (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @param additionalHeaders additionalHeaders for this call * @return GetTableResponse * @throws ApiException if fails to make API call */ - public GetTableResponse getTable(String ns, String table, Map additionalHeaders) - throws ApiException { + public GetTableResponse getTable( + String table, String delimiter, Map additionalHeaders) throws ApiException { Object localVarPostBody = null; - // verify the required parameter 'ns' is set - if (ns == null) { - throw new ApiException(400, "Missing the required parameter 'ns' when calling getTable"); - } - // verify the required parameter 'table' is set if (table == null) { throw new ApiException(400, "Missing the required parameter 'table' when calling getTable"); @@ -82,9 +75,7 @@ public GetTableResponse getTable(String ns, String table, Map ad // create path and map variables String localVarPath = - "/v1/namespaces/{ns}/tables/{table}" - .replaceAll( - "\\{" + "ns" + "\\}", apiClient.escapeString(apiClient.parameterToString(ns))) + "/v1/tables/{table}" .replaceAll( "\\{" + "table" + "\\}", apiClient.escapeString(apiClient.parameterToString(table))); @@ -97,6 +88,8 @@ public GetTableResponse getTable(String ns, String table, Map ad Map localVarCookieParams = new HashMap(); Map localVarFormParams = new HashMap(); + localVarQueryParams.addAll(apiClient.parameterToPair("delimiter", delimiter)); + localVarHeaderParams.putAll(additionalHeaders); final String[] localVarAccepts = {"application/json"}; @@ -126,39 +119,32 @@ public GetTableResponse getTable(String ns, String table, Map ad } /** - * Register a new table in the given namespace. A table represents a lance dataset. In Lance - * catalog, a table must be hosted in a namespace. + * Register a table to a namespace Register an existing table at a given storage location to a + * namespace. * - * @param ns The name of the namespace. (required) * @param registerTableRequest (required) * @return GetTableResponse * @throws ApiException if fails to make API call */ - public GetTableResponse registerTable(String ns, RegisterTableRequest registerTableRequest) + public GetTableResponse registerTable(RegisterTableRequest registerTableRequest) throws ApiException { - return this.registerTable(ns, registerTableRequest, Collections.emptyMap()); + return this.registerTable(registerTableRequest, Collections.emptyMap()); } /** - * Register a new table in the given namespace. A table represents a lance dataset. In Lance - * catalog, a table must be hosted in a namespace. + * Register a table to a namespace Register an existing table at a given storage location to a + * namespace. * - * @param ns The name of the namespace. (required) * @param registerTableRequest (required) * @param additionalHeaders additionalHeaders for this call * @return GetTableResponse * @throws ApiException if fails to make API call */ public GetTableResponse registerTable( - String ns, RegisterTableRequest registerTableRequest, Map additionalHeaders) + RegisterTableRequest registerTableRequest, Map additionalHeaders) throws ApiException { Object localVarPostBody = registerTableRequest; - // verify the required parameter 'ns' is set - if (ns == null) { - throw new ApiException(400, "Missing the required parameter 'ns' when calling registerTable"); - } - // verify the required parameter 'registerTableRequest' is set if (registerTableRequest == null) { throw new ApiException( @@ -166,10 +152,7 @@ public GetTableResponse registerTable( } // create path and map variables - String localVarPath = - "/v1/namespaces/{ns}/register" - .replaceAll( - "\\{" + "ns" + "\\}", apiClient.escapeString(apiClient.parameterToString(ns))); + String localVarPath = "/v1/table/register"; StringJoiner localVarQueryStringJoiner = new StringJoiner("&"); String localVarQueryParameterBaseName; @@ -207,33 +190,30 @@ public GetTableResponse registerTable( } /** - * Check if a table exists Check if a table exists within a given namespace. + * Check if a table exists Check if a table exists. This API should behave exactly like the + * GetTable API, except it does not contain a body. * - * @param ns The name of the namespace. (required) - * @param table A table name. (required) + * @param table A string identifier of the table (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @throws ApiException if fails to make API call */ - public void tableExists(String ns, String table) throws ApiException { - this.tableExists(ns, table, Collections.emptyMap()); + public void tableExists(String table, String delimiter) throws ApiException { + this.tableExists(table, delimiter, Collections.emptyMap()); } /** - * Check if a table exists Check if a table exists within a given namespace. + * Check if a table exists Check if a table exists. This API should behave exactly like the + * GetTable API, except it does not contain a body. * - * @param ns The name of the namespace. (required) - * @param table A table name. (required) + * @param table A string identifier of the table (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @param additionalHeaders additionalHeaders for this call * @throws ApiException if fails to make API call */ - public void tableExists(String ns, String table, Map additionalHeaders) + public void tableExists(String table, String delimiter, Map additionalHeaders) throws ApiException { Object localVarPostBody = null; - // verify the required parameter 'ns' is set - if (ns == null) { - throw new ApiException(400, "Missing the required parameter 'ns' when calling tableExists"); - } - // verify the required parameter 'table' is set if (table == null) { throw new ApiException( @@ -242,9 +222,7 @@ public void tableExists(String ns, String table, Map additionalH // create path and map variables String localVarPath = - "/v1/namespaces/{ns}/tables/{table}" - .replaceAll( - "\\{" + "ns" + "\\}", apiClient.escapeString(apiClient.parameterToString(ns))) + "/v1/tables/{table}" .replaceAll( "\\{" + "table" + "\\}", apiClient.escapeString(apiClient.parameterToString(table))); @@ -257,6 +235,8 @@ public void tableExists(String ns, String table, Map additionalH Map localVarCookieParams = new HashMap(); Map localVarFormParams = new HashMap(); + localVarQueryParams.addAll(apiClient.parameterToPair("delimiter", delimiter)); + localVarHeaderParams.putAll(additionalHeaders); final String[] localVarAccepts = {"application/json"}; diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceRequest.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceRequest.java index fb9e422af..f812adda5 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceRequest.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceRequest.java @@ -21,7 +21,9 @@ import java.io.UnsupportedEncodingException; import java.net.URLEncoder; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.Objects; import java.util.StringJoiner; @@ -29,6 +31,7 @@ /** CreateNamespaceRequest */ @JsonPropertyOrder({ CreateNamespaceRequest.JSON_PROPERTY_NAME, + CreateNamespaceRequest.JSON_PROPERTY_PARENT, CreateNamespaceRequest.JSON_PROPERTY_MODE, CreateNamespaceRequest.JSON_PROPERTY_OPTIONS }) @@ -39,6 +42,9 @@ public class CreateNamespaceRequest { public static final String JSON_PROPERTY_NAME = "name"; @javax.annotation.Nonnull private String name; + public static final String JSON_PROPERTY_PARENT = "parent"; + @javax.annotation.Nullable private List parent = new ArrayList<>(); + /** Gets or Sets mode */ public enum ModeEnum { CREATE(String.valueOf("CREATE")), @@ -106,6 +112,38 @@ public void setName(@javax.annotation.Nonnull String name) { this.name = name; } + public CreateNamespaceRequest parent(@javax.annotation.Nullable List parent) { + + this.parent = parent; + return this; + } + + public CreateNamespaceRequest addParentItem(String parentItem) { + if (this.parent == null) { + this.parent = new ArrayList<>(); + } + this.parent.add(parentItem); + return this; + } + + /** + * Get parent + * + * @return parent + */ + @javax.annotation.Nullable + @JsonProperty(JSON_PROPERTY_PARENT) + @JsonInclude(value = JsonInclude.Include.USE_DEFAULTS) + public List getParent() { + return parent; + } + + @JsonProperty(JSON_PROPERTY_PARENT) + @JsonInclude(value = JsonInclude.Include.USE_DEFAULTS) + public void setParent(@javax.annotation.Nullable List parent) { + this.parent = parent; + } + public CreateNamespaceRequest mode(@javax.annotation.Nonnull ModeEnum mode) { this.mode = mode; @@ -172,13 +210,14 @@ public boolean equals(Object o) { } CreateNamespaceRequest createNamespaceRequest = (CreateNamespaceRequest) o; return Objects.equals(this.name, createNamespaceRequest.name) + && Objects.equals(this.parent, createNamespaceRequest.parent) && Objects.equals(this.mode, createNamespaceRequest.mode) && Objects.equals(this.options, createNamespaceRequest.options); } @Override public int hashCode() { - return Objects.hash(name, mode, options); + return Objects.hash(name, parent, mode, options); } @Override @@ -186,6 +225,7 @@ public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class CreateNamespaceRequest {\n"); sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" parent: ").append(toIndentedString(parent)).append("\n"); sb.append(" mode: ").append(toIndentedString(mode)).append("\n"); sb.append(" options: ").append(toIndentedString(options)).append("\n"); sb.append("}"); @@ -249,6 +289,27 @@ public String toUrlQueryString(String prefix) { } } + // add `parent` to the URL query string + if (getParent() != null) { + for (int i = 0; i < getParent().size(); i++) { + try { + joiner.add( + String.format( + "%sparent%s%s=%s", + prefix, + suffix, + "".equals(suffix) + ? "" + : String.format("%s%d%s", containerPrefix, i, containerSuffix), + URLEncoder.encode(String.valueOf(getParent().get(i)), "UTF-8") + .replaceAll("\\+", "%20"))); + } catch (UnsupportedEncodingException e) { + // Should never happen, UTF-8 is always supported + throw new RuntimeException(e); + } + } + } + // add `mode` to the URL query string if (getMode() != null) { try { diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceResponse.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceResponse.java deleted file mode 100644 index 3f678c7d7..000000000 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/CreateNamespaceResponse.java +++ /dev/null @@ -1,210 +0,0 @@ -/* - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.lancedb.lance.catalog.client.apache.model; - -import com.fasterxml.jackson.annotation.JsonInclude; -import com.fasterxml.jackson.annotation.JsonProperty; -import com.fasterxml.jackson.annotation.JsonPropertyOrder; - -import java.io.UnsupportedEncodingException; -import java.net.URLEncoder; -import java.util.HashMap; -import java.util.Map; -import java.util.Objects; -import java.util.StringJoiner; - -/** CreateNamespaceResponse */ -@JsonPropertyOrder({ - CreateNamespaceResponse.JSON_PROPERTY_NAME, - CreateNamespaceResponse.JSON_PROPERTY_PROPERTIES -}) -@javax.annotation.Generated( - value = "org.openapitools.codegen.languages.JavaClientCodegen", - comments = "Generator version: 7.12.0") -public class CreateNamespaceResponse { - public static final String JSON_PROPERTY_NAME = "name"; - @javax.annotation.Nonnull private String name; - - public static final String JSON_PROPERTY_PROPERTIES = "properties"; - @javax.annotation.Nullable private Map properties = new HashMap<>(); - - public CreateNamespaceResponse() {} - - public CreateNamespaceResponse name(@javax.annotation.Nonnull String name) { - - this.name = name; - return this; - } - - /** - * Get name - * - * @return name - */ - @javax.annotation.Nonnull - @JsonProperty(JSON_PROPERTY_NAME) - @JsonInclude(value = JsonInclude.Include.ALWAYS) - public String getName() { - return name; - } - - @JsonProperty(JSON_PROPERTY_NAME) - @JsonInclude(value = JsonInclude.Include.ALWAYS) - public void setName(@javax.annotation.Nonnull String name) { - this.name = name; - } - - public CreateNamespaceResponse properties( - @javax.annotation.Nullable Map properties) { - - this.properties = properties; - return this; - } - - public CreateNamespaceResponse putPropertiesItem(String key, String propertiesItem) { - if (this.properties == null) { - this.properties = new HashMap<>(); - } - this.properties.put(key, propertiesItem); - return this; - } - - /** - * Properties stored on the namespace, if supported by the server. - * - * @return properties - */ - @javax.annotation.Nullable - @JsonProperty(JSON_PROPERTY_PROPERTIES) - @JsonInclude(value = JsonInclude.Include.USE_DEFAULTS) - public Map getProperties() { - return properties; - } - - @JsonProperty(JSON_PROPERTY_PROPERTIES) - @JsonInclude(value = JsonInclude.Include.USE_DEFAULTS) - public void setProperties(@javax.annotation.Nullable Map properties) { - this.properties = properties; - } - - @Override - public boolean equals(Object o) { - if (this == o) { - return true; - } - if (o == null || getClass() != o.getClass()) { - return false; - } - CreateNamespaceResponse createNamespaceResponse = (CreateNamespaceResponse) o; - return Objects.equals(this.name, createNamespaceResponse.name) - && Objects.equals(this.properties, createNamespaceResponse.properties); - } - - @Override - public int hashCode() { - return Objects.hash(name, properties); - } - - @Override - public String toString() { - StringBuilder sb = new StringBuilder(); - sb.append("class CreateNamespaceResponse {\n"); - sb.append(" name: ").append(toIndentedString(name)).append("\n"); - sb.append(" properties: ").append(toIndentedString(properties)).append("\n"); - sb.append("}"); - return sb.toString(); - } - - /** - * Convert the given object to string with each line indented by 4 spaces (except the first line). - */ - private String toIndentedString(Object o) { - if (o == null) { - return "null"; - } - return o.toString().replace("\n", "\n "); - } - - /** - * Convert the instance into URL query string. - * - * @return URL query string - */ - public String toUrlQueryString() { - return toUrlQueryString(null); - } - - /** - * Convert the instance into URL query string. - * - * @param prefix prefix of the query string - * @return URL query string - */ - public String toUrlQueryString(String prefix) { - String suffix = ""; - String containerSuffix = ""; - String containerPrefix = ""; - if (prefix == null) { - // style=form, explode=true, e.g. /pet?name=cat&type=manx - prefix = ""; - } else { - // deepObject style e.g. /pet?id[name]=cat&id[type]=manx - prefix = prefix + "["; - suffix = "]"; - containerSuffix = "]"; - containerPrefix = "["; - } - - StringJoiner joiner = new StringJoiner("&"); - - // add `name` to the URL query string - if (getName() != null) { - try { - joiner.add( - String.format( - "%sname%s=%s", - prefix, - suffix, - URLEncoder.encode(String.valueOf(getName()), "UTF-8").replaceAll("\\+", "%20"))); - } catch (UnsupportedEncodingException e) { - // Should never happen, UTF-8 is always supported - throw new RuntimeException(e); - } - } - - // add `properties` to the URL query string - if (getProperties() != null) { - for (String _key : getProperties().keySet()) { - try { - joiner.add( - String.format( - "%sproperties%s%s=%s", - prefix, - suffix, - "".equals(suffix) - ? "" - : String.format("%s%d%s", containerPrefix, _key, containerSuffix), - getProperties().get(_key), - URLEncoder.encode(String.valueOf(getProperties().get(_key)), "UTF-8") - .replaceAll("\\+", "%20"))); - } catch (UnsupportedEncodingException e) { - // Should never happen, UTF-8 is always supported - throw new RuntimeException(e); - } - } - } - - return joiner.toString(); - } -} diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetNamespaceResponse.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetNamespaceResponse.java index fc152f52b..ce88c9b53 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetNamespaceResponse.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetNamespaceResponse.java @@ -19,50 +19,88 @@ import java.io.UnsupportedEncodingException; import java.net.URLEncoder; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.Objects; import java.util.StringJoiner; /** GetNamespaceResponse */ @JsonPropertyOrder({ - GetNamespaceResponse.JSON_PROPERTY_NAMESPACE, + GetNamespaceResponse.JSON_PROPERTY_NAME, + GetNamespaceResponse.JSON_PROPERTY_PARENT, GetNamespaceResponse.JSON_PROPERTY_PROPERTIES }) @javax.annotation.Generated( value = "org.openapitools.codegen.languages.JavaClientCodegen", comments = "Generator version: 7.12.0") public class GetNamespaceResponse { - public static final String JSON_PROPERTY_NAMESPACE = "namespace"; - @javax.annotation.Nonnull private String namespace; + public static final String JSON_PROPERTY_NAME = "name"; + @javax.annotation.Nonnull private String name; + + public static final String JSON_PROPERTY_PARENT = "parent"; + @javax.annotation.Nullable private List parent = new ArrayList<>(); public static final String JSON_PROPERTY_PROPERTIES = "properties"; @javax.annotation.Nullable private Map properties = new HashMap<>(); public GetNamespaceResponse() {} - public GetNamespaceResponse namespace(@javax.annotation.Nonnull String namespace) { + public GetNamespaceResponse name(@javax.annotation.Nonnull String name) { - this.namespace = namespace; + this.name = name; return this; } /** - * Get namespace + * Get name * - * @return namespace + * @return name */ @javax.annotation.Nonnull - @JsonProperty(JSON_PROPERTY_NAMESPACE) + @JsonProperty(JSON_PROPERTY_NAME) @JsonInclude(value = JsonInclude.Include.ALWAYS) - public String getNamespace() { - return namespace; + public String getName() { + return name; } - @JsonProperty(JSON_PROPERTY_NAMESPACE) + @JsonProperty(JSON_PROPERTY_NAME) @JsonInclude(value = JsonInclude.Include.ALWAYS) - public void setNamespace(@javax.annotation.Nonnull String namespace) { - this.namespace = namespace; + public void setName(@javax.annotation.Nonnull String name) { + this.name = name; + } + + public GetNamespaceResponse parent(@javax.annotation.Nullable List parent) { + + this.parent = parent; + return this; + } + + public GetNamespaceResponse addParentItem(String parentItem) { + if (this.parent == null) { + this.parent = new ArrayList<>(); + } + this.parent.add(parentItem); + return this; + } + + /** + * Get parent + * + * @return parent + */ + @javax.annotation.Nullable + @JsonProperty(JSON_PROPERTY_PARENT) + @JsonInclude(value = JsonInclude.Include.USE_DEFAULTS) + public List getParent() { + return parent; + } + + @JsonProperty(JSON_PROPERTY_PARENT) + @JsonInclude(value = JsonInclude.Include.USE_DEFAULTS) + public void setParent(@javax.annotation.Nullable List parent) { + this.parent = parent; } public GetNamespaceResponse properties( @@ -109,20 +147,22 @@ public boolean equals(Object o) { return false; } GetNamespaceResponse getNamespaceResponse = (GetNamespaceResponse) o; - return Objects.equals(this.namespace, getNamespaceResponse.namespace) + return Objects.equals(this.name, getNamespaceResponse.name) + && Objects.equals(this.parent, getNamespaceResponse.parent) && Objects.equals(this.properties, getNamespaceResponse.properties); } @Override public int hashCode() { - return Objects.hash(namespace, properties); + return Objects.hash(name, parent, properties); } @Override public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class GetNamespaceResponse {\n"); - sb.append(" namespace: ").append(toIndentedString(namespace)).append("\n"); + sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" parent: ").append(toIndentedString(parent)).append("\n"); sb.append(" properties: ").append(toIndentedString(properties)).append("\n"); sb.append("}"); return sb.toString(); @@ -170,22 +210,42 @@ public String toUrlQueryString(String prefix) { StringJoiner joiner = new StringJoiner("&"); - // add `namespace` to the URL query string - if (getNamespace() != null) { + // add `name` to the URL query string + if (getName() != null) { try { joiner.add( String.format( - "%snamespace%s=%s", + "%sname%s=%s", prefix, suffix, - URLEncoder.encode(String.valueOf(getNamespace()), "UTF-8") - .replaceAll("\\+", "%20"))); + URLEncoder.encode(String.valueOf(getName()), "UTF-8").replaceAll("\\+", "%20"))); } catch (UnsupportedEncodingException e) { // Should never happen, UTF-8 is always supported throw new RuntimeException(e); } } + // add `parent` to the URL query string + if (getParent() != null) { + for (int i = 0; i < getParent().size(); i++) { + try { + joiner.add( + String.format( + "%sparent%s%s=%s", + prefix, + suffix, + "".equals(suffix) + ? "" + : String.format("%s%d%s", containerPrefix, i, containerSuffix), + URLEncoder.encode(String.valueOf(getParent().get(i)), "UTF-8") + .replaceAll("\\+", "%20"))); + } catch (UnsupportedEncodingException e) { + // Should never happen, UTF-8 is always supported + throw new RuntimeException(e); + } + } + } + // add `properties` to the URL query string if (getProperties() != null) { for (String _key : getProperties().keySet()) { diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetTableResponse.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetTableResponse.java index 2514c8c44..8450084c8 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetTableResponse.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/GetTableResponse.java @@ -19,14 +19,17 @@ import java.io.UnsupportedEncodingException; import java.net.URLEncoder; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.Objects; import java.util.StringJoiner; -/** Result used when a table is successfully loaded. */ +/** GetTableResponse */ @JsonPropertyOrder({ GetTableResponse.JSON_PROPERTY_NAME, + GetTableResponse.JSON_PROPERTY_NAMESPACE, GetTableResponse.JSON_PROPERTY_LOCATION, GetTableResponse.JSON_PROPERTY_PROPERTIES }) @@ -37,6 +40,9 @@ public class GetTableResponse { public static final String JSON_PROPERTY_NAME = "name"; @javax.annotation.Nonnull private String name; + public static final String JSON_PROPERTY_NAMESPACE = "namespace"; + @javax.annotation.Nonnull private List namespace = new ArrayList<>(); + public static final String JSON_PROPERTY_LOCATION = "location"; @javax.annotation.Nonnull private String location; @@ -69,6 +75,38 @@ public void setName(@javax.annotation.Nonnull String name) { this.name = name; } + public GetTableResponse namespace(@javax.annotation.Nonnull List namespace) { + + this.namespace = namespace; + return this; + } + + public GetTableResponse addNamespaceItem(String namespaceItem) { + if (this.namespace == null) { + this.namespace = new ArrayList<>(); + } + this.namespace.add(namespaceItem); + return this; + } + + /** + * Get namespace + * + * @return namespace + */ + @javax.annotation.Nonnull + @JsonProperty(JSON_PROPERTY_NAMESPACE) + @JsonInclude(value = JsonInclude.Include.ALWAYS) + public List getNamespace() { + return namespace; + } + + @JsonProperty(JSON_PROPERTY_NAMESPACE) + @JsonInclude(value = JsonInclude.Include.ALWAYS) + public void setNamespace(@javax.annotation.Nonnull List namespace) { + this.namespace = namespace; + } + public GetTableResponse location(@javax.annotation.Nonnull String location) { this.location = location; @@ -135,13 +173,14 @@ public boolean equals(Object o) { } GetTableResponse getTableResponse = (GetTableResponse) o; return Objects.equals(this.name, getTableResponse.name) + && Objects.equals(this.namespace, getTableResponse.namespace) && Objects.equals(this.location, getTableResponse.location) && Objects.equals(this.properties, getTableResponse.properties); } @Override public int hashCode() { - return Objects.hash(name, location, properties); + return Objects.hash(name, namespace, location, properties); } @Override @@ -149,6 +188,7 @@ public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class GetTableResponse {\n"); sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" namespace: ").append(toIndentedString(namespace)).append("\n"); sb.append(" location: ").append(toIndentedString(location)).append("\n"); sb.append(" properties: ").append(toIndentedString(properties)).append("\n"); sb.append("}"); @@ -212,6 +252,27 @@ public String toUrlQueryString(String prefix) { } } + // add `namespace` to the URL query string + if (getNamespace() != null) { + for (int i = 0; i < getNamespace().size(); i++) { + try { + joiner.add( + String.format( + "%snamespace%s%s=%s", + prefix, + suffix, + "".equals(suffix) + ? "" + : String.format("%s%d%s", containerPrefix, i, containerSuffix), + URLEncoder.encode(String.valueOf(getNamespace().get(i)), "UTF-8") + .replaceAll("\\+", "%20"))); + } catch (UnsupportedEncodingException e) { + // Should never happen, UTF-8 is always supported + throw new RuntimeException(e); + } + } + } + // add `location` to the URL query string if (getLocation() != null) { try { diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/ListNamespacesResponse.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/ListNamespacesResponse.java index 94333e84c..44e8eabcd 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/ListNamespacesResponse.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/ListNamespacesResponse.java @@ -57,7 +57,7 @@ public ListNamespacesResponse addNamespacesItem(String namespacesItem) { } /** - * An array of namespace names in the catalog. + * Get namespaces * * @return namespaces */ diff --git a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/RegisterTableRequest.java b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/RegisterTableRequest.java index 9286e323b..2e55c81d1 100644 --- a/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/RegisterTableRequest.java +++ b/java/lance-catalog-apache-client/src/main/java/com/lancedb/lance/catalog/client/apache/model/RegisterTableRequest.java @@ -19,12 +19,15 @@ import java.io.UnsupportedEncodingException; import java.net.URLEncoder; +import java.util.ArrayList; +import java.util.List; import java.util.Objects; import java.util.StringJoiner; /** RegisterTableRequest */ @JsonPropertyOrder({ RegisterTableRequest.JSON_PROPERTY_NAME, + RegisterTableRequest.JSON_PROPERTY_NAMESPACE, RegisterTableRequest.JSON_PROPERTY_LOCATION }) @javax.annotation.Generated( @@ -34,6 +37,9 @@ public class RegisterTableRequest { public static final String JSON_PROPERTY_NAME = "name"; @javax.annotation.Nonnull private String name; + public static final String JSON_PROPERTY_NAMESPACE = "namespace"; + @javax.annotation.Nonnull private List namespace = new ArrayList<>(); + public static final String JSON_PROPERTY_LOCATION = "location"; @javax.annotation.Nonnull private String location; @@ -63,6 +69,38 @@ public void setName(@javax.annotation.Nonnull String name) { this.name = name; } + public RegisterTableRequest namespace(@javax.annotation.Nonnull List namespace) { + + this.namespace = namespace; + return this; + } + + public RegisterTableRequest addNamespaceItem(String namespaceItem) { + if (this.namespace == null) { + this.namespace = new ArrayList<>(); + } + this.namespace.add(namespaceItem); + return this; + } + + /** + * Get namespace + * + * @return namespace + */ + @javax.annotation.Nonnull + @JsonProperty(JSON_PROPERTY_NAMESPACE) + @JsonInclude(value = JsonInclude.Include.ALWAYS) + public List getNamespace() { + return namespace; + } + + @JsonProperty(JSON_PROPERTY_NAMESPACE) + @JsonInclude(value = JsonInclude.Include.ALWAYS) + public void setNamespace(@javax.annotation.Nonnull List namespace) { + this.namespace = namespace; + } + public RegisterTableRequest location(@javax.annotation.Nonnull String location) { this.location = location; @@ -97,12 +135,13 @@ public boolean equals(Object o) { } RegisterTableRequest registerTableRequest = (RegisterTableRequest) o; return Objects.equals(this.name, registerTableRequest.name) + && Objects.equals(this.namespace, registerTableRequest.namespace) && Objects.equals(this.location, registerTableRequest.location); } @Override public int hashCode() { - return Objects.hash(name, location); + return Objects.hash(name, namespace, location); } @Override @@ -110,6 +149,7 @@ public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class RegisterTableRequest {\n"); sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" namespace: ").append(toIndentedString(namespace)).append("\n"); sb.append(" location: ").append(toIndentedString(location)).append("\n"); sb.append("}"); return sb.toString(); @@ -172,6 +212,27 @@ public String toUrlQueryString(String prefix) { } } + // add `namespace` to the URL query string + if (getNamespace() != null) { + for (int i = 0; i < getNamespace().size(); i++) { + try { + joiner.add( + String.format( + "%snamespace%s%s=%s", + prefix, + suffix, + "".equals(suffix) + ? "" + : String.format("%s%d%s", containerPrefix, i, containerSuffix), + URLEncoder.encode(String.valueOf(getNamespace().get(i)), "UTF-8") + .replaceAll("\\+", "%20"))); + } catch (UnsupportedEncodingException e) { + // Should never happen, UTF-8 is always supported + throw new RuntimeException(e); + } + } + } + // add `location` to the URL query string if (getLocation() != null) { try { diff --git a/java/lance-catalog-springboot-server/.openapi-generator/FILES b/java/lance-catalog-springboot-server/.openapi-generator/FILES index 87f141631..e48eb89f6 100644 --- a/java/lance-catalog-springboot-server/.openapi-generator/FILES +++ b/java/lance-catalog-springboot-server/.openapi-generator/FILES @@ -5,7 +5,6 @@ src/main/java/com/lancedb/lance/catalog/server/springboot/api/ApiUtil.java src/main/java/com/lancedb/lance/catalog/server/springboot/api/NamespaceApi.java src/main/java/com/lancedb/lance/catalog/server/springboot/api/TableApi.java src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceRequest.java -src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceResponse.java src/main/java/com/lancedb/lance/catalog/server/springboot/model/ErrorResponse.java src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetNamespaceResponse.java src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetTableResponse.java diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/NamespaceApi.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/NamespaceApi.java index 21c42d6d9..116966b9e 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/NamespaceApi.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/NamespaceApi.java @@ -14,7 +14,6 @@ package com.lancedb.lance.catalog.server.springboot.api; import com.lancedb.lance.catalog.server.springboot.model.CreateNamespaceRequest; -import com.lancedb.lance.catalog.server.springboot.model.CreateNamespaceResponse; import com.lancedb.lance.catalog.server.springboot.model.ErrorResponse; import com.lancedb.lance.catalog.server.springboot.model.GetNamespaceResponse; import com.lancedb.lance.catalog.server.springboot.model.ListNamespacesResponse; @@ -51,47 +50,44 @@ default Optional getRequest() { } /** - * POST /v1/namespaces : Create a new namespace. A catalog can manage one or more namespaces. A - * namespace is used to manage one or more tables. There are three modes when trying to create a - * namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name - * already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not - * exist. If a namespace of the same name already exists, the operation succeeds and the existing - * namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of - * the same name already exists, the existing namespace is dropped and a new namespace with this - * name with no table is created. + * POST /v1/namespaces : Create a new namespace Create a new namespace. A namespace can manage + * either a collection of child namespaces, or a collection of tables. There are three modes when + * trying to create a namespace, to differentiate the behavior when a namespace of the same name + * already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and + * the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty + * namespace with this name is created. * * @param createNamespaceRequest (required) - * @return Represents a successful call to create a namespace. Returns the namespace created, as - * well as any properties that were stored for the namespace, including those the server might - * have added. Implementations are not required to support namespace properties. (status code - * 200) or Indicates a bad request error. It could be caused by an unexpected request body - * format or other forms of request validation failure, such as invalid json. Usually serves - * application/json content, although in some cases simple text/plain content might be - * returned by the server's middleware. (status code 400) or Unauthorized. The request - * lacks valid authentication credentials for the operation. (status code 401) or Forbidden. - * Authenticated user does not have the necessary permissions. (status code 403) or Not - * Acceptable / Unsupported Operation. The server does not support this operation. (status - * code 406) or The request conflicts with the current state of the target resource. (status - * code 409) or The service is not ready to handle the request. The client should wait and - * retry. The service may additionally send a Retry-After header to indicate when to retry. - * (status code 503) or A server-side problem that might not be addressable from the client - * side. Used for server 5xx errors without more specific documentation in individual routes. - * (status code 5XX) + * @return Returns a namespace, as well as any properties stored on the namespace if namespace + * properties are supported by the server. (status code 200) or Indicates a bad request error. + * It could be caused by an unexpected request body format or other forms of request + * validation failure, such as invalid json. Usually serves application/json content, although + * in some cases simple text/plain content might be returned by the server's middleware. + * (status code 400) or Unauthorized. The request lacks valid authentication credentials for + * the operation. (status code 401) or Forbidden. Authenticated user does not have the + * necessary permissions. (status code 403) or Not Acceptable / Unsupported Operation. The + * server does not support this operation. (status code 406) or The request conflicts with the + * current state of the target resource. (status code 409) or The service is not ready to + * handle the request. The client should wait and retry. The service may additionally send a + * Retry-After header to indicate when to retry. (status code 503) or A server-side problem + * that might not be addressable from the client side. Used for server 5xx errors without more + * specific documentation in individual routes. (status code 5XX) */ @Operation( operationId = "createNamespace", - summary = - "Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. ", + summary = "Create a new namespace", + description = + "Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. ", tags = {"Namespace"}, responses = { @ApiResponse( responseCode = "200", description = - "Represents a successful call to create a namespace. Returns the namespace created, as well as any properties that were stored for the namespace, including those the server might have added. Implementations are not required to support namespace properties.", + "Returns a namespace, as well as any properties stored on the namespace if namespace properties are supported by the server.", content = { @Content( mediaType = "application/json", - schema = @Schema(implementation = CreateNamespaceResponse.class)) + schema = @Schema(implementation = GetNamespaceResponse.class)) }), @ApiResponse( responseCode = "400", @@ -160,7 +156,7 @@ default Optional getRequest() { value = "/v1/namespaces", produces = {"application/json"}, consumes = {"application/json"}) - default ResponseEntity createNamespace( + default ResponseEntity createNamespace( @Parameter(name = "CreateNamespaceRequest", description = "", required = true) @Valid @RequestBody @@ -171,7 +167,7 @@ default ResponseEntity createNamespace( for (MediaType mediaType : MediaType.parseMediaTypes(request.getHeader("Accept"))) { if (mediaType.isCompatibleWith(MediaType.valueOf("application/json"))) { String exampleString = - "{ \"name\" : \"name\", \"properties\" : { \"created_at\" : \"1452120468\" } }"; + "{ \"parent\" : [ \"parent\", \"parent\" ], \"name\" : \"name\", \"properties\" : { \"owner\" : \"Ralph\", \"created_at\" : \"1452120468\" } }"; ApiUtil.setExampleResponse(request, "application/json", exampleString); break; } @@ -223,9 +219,11 @@ default ResponseEntity createNamespace( } /** - * DELETE /v1/namespaces/{ns} : Drop a namespace from the catalog. Namespace must be empty. + * DELETE /v1/namespaces/{namespace} : Drop a namespace Drop a namespace. The namespace must be + * empty. * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return Success, no content (status code 204) or Indicates a bad request error. It could be * caused by an unexpected request body format or other forms of request validation failure, * such as invalid json. Usually serves application/json content, although in some cases @@ -242,7 +240,8 @@ default ResponseEntity createNamespace( */ @Operation( operationId = "dropNamespace", - summary = "Drop a namespace from the catalog. Namespace must be empty.", + summary = "Drop a namespace", + description = "Drop a namespace. The namespace must be empty. ", tags = {"Namespace"}, responses = { @ApiResponse(responseCode = "204", description = "Success, no content"), @@ -309,16 +308,23 @@ default ResponseEntity createNamespace( }) @RequestMapping( method = RequestMethod.DELETE, - value = "/v1/namespaces/{ns}", + value = "/v1/namespaces/{namespace}", produces = {"application/json"}) default ResponseEntity dropNamespace( @Parameter( - name = "ns", - description = "The name of the namespace.", + name = "namespace", + description = "A string identifier of the namespace.", required = true, in = ParameterIn.PATH) - @PathVariable("ns") - String ns) { + @PathVariable("namespace") + String namespace, + @Parameter( + name = "delimiter", + description = "The delimiter for the identifier used in the context", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "delimiter", required = false) + Optional delimiter) { getRequest() .ifPresent( request -> { @@ -371,10 +377,11 @@ default ResponseEntity dropNamespace( } /** - * GET /v1/namespaces/{ns} : Get information about a namespace Return a detailed information for a - * given namespace + * GET /v1/namespaces/{namespace} : Get information about a namespace Return the detailed + * information for a given namespace * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return Returns a namespace, as well as any properties stored on the namespace if namespace * properties are supported by the server. (status code 200) or Indicates a bad request error. * It could be caused by an unexpected request body format or other forms of request @@ -392,7 +399,7 @@ default ResponseEntity dropNamespace( @Operation( operationId = "getNamespace", summary = "Get information about a namespace", - description = "Return a detailed information for a given namespace", + description = "Return the detailed information for a given namespace ", tags = {"Namespace"}, responses = { @ApiResponse( @@ -459,23 +466,30 @@ default ResponseEntity dropNamespace( }) @RequestMapping( method = RequestMethod.GET, - value = "/v1/namespaces/{ns}", + value = "/v1/namespaces/{namespace}", produces = {"application/json"}) default ResponseEntity getNamespace( @Parameter( - name = "ns", - description = "The name of the namespace.", + name = "namespace", + description = "A string identifier of the namespace.", required = true, in = ParameterIn.PATH) - @PathVariable("ns") - String ns) { + @PathVariable("namespace") + String namespace, + @Parameter( + name = "delimiter", + description = "The delimiter for the identifier used in the context", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "delimiter", required = false) + Optional delimiter) { getRequest() .ifPresent( request -> { for (MediaType mediaType : MediaType.parseMediaTypes(request.getHeader("Accept"))) { if (mediaType.isCompatibleWith(MediaType.valueOf("application/json"))) { String exampleString = - "{ \"namespace\" : \"namespace\", \"properties\" : { \"owner\" : \"Ralph\", \"created_at\" : \"1452120468\" } }"; + "{ \"parent\" : [ \"parent\", \"parent\" ], \"name\" : \"name\", \"properties\" : { \"owner\" : \"Ralph\", \"created_at\" : \"1452120468\" } }"; ApiUtil.setExampleResponse(request, "application/json", exampleString); break; } @@ -521,11 +535,14 @@ default ResponseEntity getNamespace( } /** - * GET /v1/namespaces : List all namespaces in the catalog. + * GET /v1/namespaces : List namespaces List all child namespace names of the root namespace or a + * given parent namespace. * * @param pageToken (optional) * @param pageSize An inclusive upper bound of the number of results that a client will receive. * (optional) + * @param parent A string identifier of the parent namespace. (optional) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return A list of namespaces (status code 200) or Indicates a bad request error. It could be * caused by an unexpected request body format or other forms of request validation failure, * such as invalid json. Usually serves application/json content, although in some cases @@ -541,7 +558,9 @@ default ResponseEntity getNamespace( */ @Operation( operationId = "listNamespaces", - summary = "List all namespaces in the catalog. ", + summary = "List namespaces", + description = + "List all child namespace names of the root namespace or a given parent namespace. ", tags = {"Namespace"}, responses = { @ApiResponse( @@ -622,7 +641,21 @@ default ResponseEntity listNamespaces( in = ParameterIn.QUERY) @Valid @RequestParam(value = "pageSize", required = false) - Optional<@Min(1) Integer> pageSize) { + Optional<@Min(1) Integer> pageSize, + @Parameter( + name = "parent", + description = "A string identifier of the parent namespace.", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "parent", required = false) + Optional parent, + @Parameter( + name = "delimiter", + description = "The delimiter for the identifier used in the context", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "delimiter", required = false) + Optional delimiter) { getRequest() .ifPresent( request -> { @@ -675,10 +708,11 @@ default ResponseEntity listNamespaces( } /** - * HEAD /v1/namespaces/{ns} : Check if a namespace exists Check if a namespace exists. The - * response does not contain a body. + * HEAD /v1/namespaces/{namespace} : Check if a namespace exists Check if a namespace exists. This + * API should behave exactly like the GetNamespace API, except it does not contain a body. * - * @param ns The name of the namespace. (required) + * @param namespace A string identifier of the namespace. (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return Success, no content (status code 200) or Indicates a bad request error. It could be * caused by an unexpected request body format or other forms of request validation failure, * such as invalid json. Usually serves application/json content, although in some cases @@ -695,7 +729,8 @@ default ResponseEntity listNamespaces( @Operation( operationId = "namespaceExists", summary = "Check if a namespace exists", - description = "Check if a namespace exists. The response does not contain a body.", + description = + "Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. ", tags = {"Namespace"}, responses = { @ApiResponse(responseCode = "200", description = "Success, no content"), @@ -754,16 +789,23 @@ default ResponseEntity listNamespaces( }) @RequestMapping( method = RequestMethod.HEAD, - value = "/v1/namespaces/{ns}", + value = "/v1/namespaces/{namespace}", produces = {"application/json"}) default ResponseEntity namespaceExists( @Parameter( - name = "ns", - description = "The name of the namespace.", + name = "namespace", + description = "A string identifier of the namespace.", required = true, in = ParameterIn.PATH) - @PathVariable("ns") - String ns) { + @PathVariable("namespace") + String namespace, + @Parameter( + name = "delimiter", + description = "The delimiter for the identifier used in the context", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "delimiter", required = false) + Optional delimiter) { getRequest() .ifPresent( request -> { diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/TableApi.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/TableApi.java index 503834bc4..22965c910 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/TableApi.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/api/TableApi.java @@ -49,11 +49,10 @@ default Optional getRequest() { } /** - * GET /v1/namespaces/{ns}/tables/{table} : Get a table from the catalog Get a table's - * detailed information under a specified namespace from the catalog. + * GET /v1/tables/{table} : Get a table from the namespace Get a table's detailed information. * - * @param ns The name of the namespace. (required) - * @param table A table name. (required) + * @param table A string identifier of the table (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return Table properties result when loading a table (status code 200) or Indicates a bad * request error. It could be caused by an unexpected request body format or other forms of * request validation failure, such as invalid json. Usually serves application/json content, @@ -69,9 +68,8 @@ default Optional getRequest() { */ @Operation( operationId = "getTable", - summary = "Get a table from the catalog", - description = - "Get a table's detailed information under a specified namespace from the catalog.", + summary = "Get a table from the namespace", + description = "Get a table's detailed information. ", tags = {"Table"}, responses = { @ApiResponse( @@ -137,30 +135,30 @@ default Optional getRequest() { }) @RequestMapping( method = RequestMethod.GET, - value = "/v1/namespaces/{ns}/tables/{table}", + value = "/v1/tables/{table}", produces = {"application/json"}) default ResponseEntity getTable( - @Parameter( - name = "ns", - description = "The name of the namespace.", - required = true, - in = ParameterIn.PATH) - @PathVariable("ns") - String ns, @Parameter( name = "table", - description = "A table name.", + description = "A string identifier of the table", required = true, in = ParameterIn.PATH) @PathVariable("table") - String table) { + String table, + @Parameter( + name = "delimiter", + description = "The delimiter for the identifier used in the context", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "delimiter", required = false) + Optional delimiter) { getRequest() .ifPresent( request -> { for (MediaType mediaType : MediaType.parseMediaTypes(request.getHeader("Accept"))) { if (mediaType.isCompatibleWith(MediaType.valueOf("application/json"))) { String exampleString = - "{ \"name\" : \"name\", \"location\" : \"location\", \"properties\" : { \"key\" : \"properties\" } }"; + "{ \"name\" : \"name\", \"namespace\" : [ \"namespace\", \"namespace\" ], \"location\" : \"location\", \"properties\" : { \"key\" : \"properties\" } }"; ApiUtil.setExampleResponse(request, "application/json", exampleString); break; } @@ -206,10 +204,9 @@ default ResponseEntity getTable( } /** - * POST /v1/namespaces/{ns}/register : Register a new table in the given namespace. A table - * represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. + * POST /v1/table/register : Register a table to a namespace Register an existing table at a given + * storage location to a namespace. * - * @param ns The name of the namespace. (required) * @param registerTableRequest (required) * @return Table properties result when loading a table (status code 200) or Indicates a bad * request error. It could be caused by an unexpected request body format or other forms of @@ -227,8 +224,8 @@ default ResponseEntity getTable( */ @Operation( operationId = "registerTable", - summary = - "Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. ", + summary = "Register a table to a namespace", + description = "Register an existing table at a given storage location to a namespace. ", tags = {"Table"}, responses = { @ApiResponse( @@ -303,17 +300,10 @@ default ResponseEntity getTable( }) @RequestMapping( method = RequestMethod.POST, - value = "/v1/namespaces/{ns}/register", + value = "/v1/table/register", produces = {"application/json"}, consumes = {"application/json"}) default ResponseEntity registerTable( - @Parameter( - name = "ns", - description = "The name of the namespace.", - required = true, - in = ParameterIn.PATH) - @PathVariable("ns") - String ns, @Parameter(name = "RegisterTableRequest", description = "", required = true) @Valid @RequestBody @@ -324,7 +314,7 @@ default ResponseEntity registerTable( for (MediaType mediaType : MediaType.parseMediaTypes(request.getHeader("Accept"))) { if (mediaType.isCompatibleWith(MediaType.valueOf("application/json"))) { String exampleString = - "{ \"name\" : \"name\", \"location\" : \"location\", \"properties\" : { \"key\" : \"properties\" } }"; + "{ \"name\" : \"name\", \"namespace\" : [ \"namespace\", \"namespace\" ], \"location\" : \"location\", \"properties\" : { \"key\" : \"properties\" } }"; ApiUtil.setExampleResponse(request, "application/json", exampleString); break; } @@ -376,11 +366,11 @@ default ResponseEntity registerTable( } /** - * HEAD /v1/namespaces/{ns}/tables/{table} : Check if a table exists Check if a table exists - * within a given namespace. + * HEAD /v1/tables/{table} : Check if a table exists Check if a table exists. This API should + * behave exactly like the GetTable API, except it does not contain a body. * - * @param ns The name of the namespace. (required) - * @param table A table name. (required) + * @param table A string identifier of the table (required) + * @param delimiter The delimiter for the identifier used in the context (optional) * @return Success, no content (status code 200) or Indicates a bad request error. It could be * caused by an unexpected request body format or other forms of request validation failure, * such as invalid json. Usually serves application/json content, although in some cases @@ -397,7 +387,8 @@ default ResponseEntity registerTable( @Operation( operationId = "tableExists", summary = "Check if a table exists", - description = "Check if a table exists within a given namespace.", + description = + "Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. ", tags = {"Table"}, responses = { @ApiResponse(responseCode = "200", description = "Success, no content"), @@ -456,23 +447,23 @@ default ResponseEntity registerTable( }) @RequestMapping( method = RequestMethod.HEAD, - value = "/v1/namespaces/{ns}/tables/{table}", + value = "/v1/tables/{table}", produces = {"application/json"}) default ResponseEntity tableExists( - @Parameter( - name = "ns", - description = "The name of the namespace.", - required = true, - in = ParameterIn.PATH) - @PathVariable("ns") - String ns, @Parameter( name = "table", - description = "A table name.", + description = "A string identifier of the table", required = true, in = ParameterIn.PATH) @PathVariable("table") - String table) { + String table, + @Parameter( + name = "delimiter", + description = "The delimiter for the identifier used in the context", + in = ParameterIn.QUERY) + @Valid + @RequestParam(value = "delimiter", required = false) + Optional delimiter) { getRequest() .ifPresent( request -> { diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceRequest.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceRequest.java index 35b0abcb3..f86b906ea 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceRequest.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceRequest.java @@ -23,7 +23,9 @@ import javax.validation.constraints.*; import java.util.*; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.Objects; @@ -35,6 +37,8 @@ public class CreateNamespaceRequest { private String name; + @Valid private List parent = new ArrayList<>(); + /** Gets or Sets mode */ public enum ModeEnum { CREATE("CREATE"), @@ -105,6 +109,34 @@ public void setName(String name) { this.name = name; } + public CreateNamespaceRequest parent(List parent) { + this.parent = parent; + return this; + } + + public CreateNamespaceRequest addParentItem(String parentItem) { + if (this.parent == null) { + this.parent = new ArrayList<>(); + } + this.parent.add(parentItem); + return this; + } + + /** + * Get parent + * + * @return parent + */ + @Schema(name = "parent", requiredMode = Schema.RequiredMode.NOT_REQUIRED) + @JsonProperty("parent") + public List getParent() { + return parent; + } + + public void setParent(List parent) { + this.parent = parent; + } + public CreateNamespaceRequest mode(ModeEnum mode) { this.mode = mode; return this; @@ -164,13 +196,14 @@ public boolean equals(Object o) { } CreateNamespaceRequest createNamespaceRequest = (CreateNamespaceRequest) o; return Objects.equals(this.name, createNamespaceRequest.name) + && Objects.equals(this.parent, createNamespaceRequest.parent) && Objects.equals(this.mode, createNamespaceRequest.mode) && Objects.equals(this.options, createNamespaceRequest.options); } @Override public int hashCode() { - return Objects.hash(name, mode, options); + return Objects.hash(name, parent, mode, options); } @Override @@ -178,6 +211,7 @@ public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class CreateNamespaceRequest {\n"); sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" parent: ").append(toIndentedString(parent)).append("\n"); sb.append(" mode: ").append(toIndentedString(mode)).append("\n"); sb.append(" options: ").append(toIndentedString(options)).append("\n"); sb.append("}"); diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceResponse.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceResponse.java deleted file mode 100644 index 61a08fa0e..000000000 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/CreateNamespaceResponse.java +++ /dev/null @@ -1,137 +0,0 @@ -/* - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.lancedb.lance.catalog.server.springboot.model; - -import com.fasterxml.jackson.annotation.JsonProperty; -import io.swagger.v3.oas.annotations.media.Schema; - -import javax.annotation.Generated; -import javax.validation.Valid; -import javax.validation.constraints.*; - -import java.util.*; -import java.util.HashMap; -import java.util.Map; -import java.util.Objects; - -/** CreateNamespaceResponse */ -@Generated( - value = "org.openapitools.codegen.languages.SpringCodegen", - comments = "Generator version: 7.12.0") -public class CreateNamespaceResponse { - - private String name; - - @Valid private Map properties = new HashMap<>(); - - public CreateNamespaceResponse() { - super(); - } - - /** Constructor with only required parameters */ - public CreateNamespaceResponse(String name) { - this.name = name; - } - - public CreateNamespaceResponse name(String name) { - this.name = name; - return this; - } - - /** - * Get name - * - * @return name - */ - @NotNull - @Schema(name = "name", requiredMode = Schema.RequiredMode.REQUIRED) - @JsonProperty("name") - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public CreateNamespaceResponse properties(Map properties) { - this.properties = properties; - return this; - } - - public CreateNamespaceResponse putPropertiesItem(String key, String propertiesItem) { - if (this.properties == null) { - this.properties = new HashMap<>(); - } - this.properties.put(key, propertiesItem); - return this; - } - - /** - * Properties stored on the namespace, if supported by the server. - * - * @return properties - */ - @Schema( - name = "properties", - example = "{created_at=1452120468}", - description = "Properties stored on the namespace, if supported by the server.", - requiredMode = Schema.RequiredMode.NOT_REQUIRED) - @JsonProperty("properties") - public Map getProperties() { - return properties; - } - - public void setProperties(Map properties) { - this.properties = properties; - } - - @Override - public boolean equals(Object o) { - if (this == o) { - return true; - } - if (o == null || getClass() != o.getClass()) { - return false; - } - CreateNamespaceResponse createNamespaceResponse = (CreateNamespaceResponse) o; - return Objects.equals(this.name, createNamespaceResponse.name) - && Objects.equals(this.properties, createNamespaceResponse.properties); - } - - @Override - public int hashCode() { - return Objects.hash(name, properties); - } - - @Override - public String toString() { - StringBuilder sb = new StringBuilder(); - sb.append("class CreateNamespaceResponse {\n"); - sb.append(" name: ").append(toIndentedString(name)).append("\n"); - sb.append(" properties: ").append(toIndentedString(properties)).append("\n"); - sb.append("}"); - return sb.toString(); - } - - /** - * Convert the given object to string with each line indented by 4 spaces (except the first line). - */ - private String toIndentedString(Object o) { - if (o == null) { - return "null"; - } - return o.toString().replace("\n", "\n "); - } -} diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetNamespaceResponse.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetNamespaceResponse.java index 41a818e15..0af0798b3 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetNamespaceResponse.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetNamespaceResponse.java @@ -21,7 +21,9 @@ import javax.validation.constraints.*; import java.util.*; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.Objects; @@ -31,7 +33,9 @@ comments = "Generator version: 7.12.0") public class GetNamespaceResponse { - private String namespace; + private String name; + + @Valid private List parent = new ArrayList<>(); @Valid private Map properties = new HashMap<>(); @@ -40,29 +44,57 @@ public GetNamespaceResponse() { } /** Constructor with only required parameters */ - public GetNamespaceResponse(String namespace) { - this.namespace = namespace; + public GetNamespaceResponse(String name) { + this.name = name; } - public GetNamespaceResponse namespace(String namespace) { - this.namespace = namespace; + public GetNamespaceResponse name(String name) { + this.name = name; return this; } /** - * Get namespace + * Get name * - * @return namespace + * @return name */ @NotNull - @Schema(name = "namespace", requiredMode = Schema.RequiredMode.REQUIRED) - @JsonProperty("namespace") - public String getNamespace() { - return namespace; + @Schema(name = "name", requiredMode = Schema.RequiredMode.REQUIRED) + @JsonProperty("name") + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public GetNamespaceResponse parent(List parent) { + this.parent = parent; + return this; + } + + public GetNamespaceResponse addParentItem(String parentItem) { + if (this.parent == null) { + this.parent = new ArrayList<>(); + } + this.parent.add(parentItem); + return this; + } + + /** + * Get parent + * + * @return parent + */ + @Schema(name = "parent", requiredMode = Schema.RequiredMode.NOT_REQUIRED) + @JsonProperty("parent") + public List getParent() { + return parent; } - public void setNamespace(String namespace) { - this.namespace = namespace; + public void setParent(List parent) { + this.parent = parent; } public GetNamespaceResponse properties(Map properties) { @@ -109,20 +141,22 @@ public boolean equals(Object o) { return false; } GetNamespaceResponse getNamespaceResponse = (GetNamespaceResponse) o; - return Objects.equals(this.namespace, getNamespaceResponse.namespace) + return Objects.equals(this.name, getNamespaceResponse.name) + && Objects.equals(this.parent, getNamespaceResponse.parent) && Objects.equals(this.properties, getNamespaceResponse.properties); } @Override public int hashCode() { - return Objects.hash(namespace, properties); + return Objects.hash(name, parent, properties); } @Override public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class GetNamespaceResponse {\n"); - sb.append(" namespace: ").append(toIndentedString(namespace)).append("\n"); + sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" parent: ").append(toIndentedString(parent)).append("\n"); sb.append(" properties: ").append(toIndentedString(properties)).append("\n"); sb.append("}"); return sb.toString(); diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetTableResponse.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetTableResponse.java index 5bebf1bde..001bcceb4 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetTableResponse.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/GetTableResponse.java @@ -21,14 +21,13 @@ import javax.validation.constraints.*; import java.util.*; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.Objects; -/** Result used when a table is successfully loaded. */ -@Schema( - name = "GetTableResponse", - description = "Result used when a table is successfully loaded. ") +/** GetTableResponse */ @Generated( value = "org.openapitools.codegen.languages.SpringCodegen", comments = "Generator version: 7.12.0") @@ -36,6 +35,8 @@ public class GetTableResponse { private String name; + @Valid private List namespace = new ArrayList<>(); + private String location; @Valid private Map properties = new HashMap<>(); @@ -45,8 +46,9 @@ public GetTableResponse() { } /** Constructor with only required parameters */ - public GetTableResponse(String name, String location) { + public GetTableResponse(String name, List namespace, String location) { this.name = name; + this.namespace = namespace; this.location = location; } @@ -71,6 +73,35 @@ public void setName(String name) { this.name = name; } + public GetTableResponse namespace(List namespace) { + this.namespace = namespace; + return this; + } + + public GetTableResponse addNamespaceItem(String namespaceItem) { + if (this.namespace == null) { + this.namespace = new ArrayList<>(); + } + this.namespace.add(namespaceItem); + return this; + } + + /** + * Get namespace + * + * @return namespace + */ + @NotNull + @Schema(name = "namespace", requiredMode = Schema.RequiredMode.REQUIRED) + @JsonProperty("namespace") + public List getNamespace() { + return namespace; + } + + public void setNamespace(List namespace) { + this.namespace = namespace; + } + public GetTableResponse location(String location) { this.location = location; return this; @@ -130,13 +161,14 @@ public boolean equals(Object o) { } GetTableResponse getTableResponse = (GetTableResponse) o; return Objects.equals(this.name, getTableResponse.name) + && Objects.equals(this.namespace, getTableResponse.namespace) && Objects.equals(this.location, getTableResponse.location) && Objects.equals(this.properties, getTableResponse.properties); } @Override public int hashCode() { - return Objects.hash(name, location, properties); + return Objects.hash(name, namespace, location, properties); } @Override @@ -144,6 +176,7 @@ public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class GetTableResponse {\n"); sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" namespace: ").append(toIndentedString(namespace)).append("\n"); sb.append(" location: ").append(toIndentedString(location)).append("\n"); sb.append(" properties: ").append(toIndentedString(properties)).append("\n"); sb.append("}"); diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/ListNamespacesResponse.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/ListNamespacesResponse.java index 725e86e58..6d76e0aff 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/ListNamespacesResponse.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/ListNamespacesResponse.java @@ -59,15 +59,12 @@ public ListNamespacesResponse addNamespacesItem(String namespacesItem) { } /** - * An array of namespace names in the catalog. + * Get namespaces * * @return namespaces */ @NotNull - @Schema( - name = "namespaces", - description = "An array of namespace names in the catalog.", - requiredMode = Schema.RequiredMode.REQUIRED) + @Schema(name = "namespaces", requiredMode = Schema.RequiredMode.REQUIRED) @JsonProperty("namespaces") public Set getNamespaces() { return namespaces; diff --git a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/RegisterTableRequest.java b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/RegisterTableRequest.java index 086006f7c..2f3fc5c97 100644 --- a/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/RegisterTableRequest.java +++ b/java/lance-catalog-springboot-server/src/main/java/com/lancedb/lance/catalog/server/springboot/model/RegisterTableRequest.java @@ -17,9 +17,12 @@ import io.swagger.v3.oas.annotations.media.Schema; import javax.annotation.Generated; +import javax.validation.Valid; import javax.validation.constraints.*; import java.util.*; +import java.util.ArrayList; +import java.util.List; import java.util.Objects; /** RegisterTableRequest */ @@ -30,6 +33,8 @@ public class RegisterTableRequest { private String name; + @Valid private List namespace = new ArrayList<>(); + private String location; public RegisterTableRequest() { @@ -37,8 +42,9 @@ public RegisterTableRequest() { } /** Constructor with only required parameters */ - public RegisterTableRequest(String name, String location) { + public RegisterTableRequest(String name, List namespace, String location) { this.name = name; + this.namespace = namespace; this.location = location; } @@ -63,6 +69,35 @@ public void setName(String name) { this.name = name; } + public RegisterTableRequest namespace(List namespace) { + this.namespace = namespace; + return this; + } + + public RegisterTableRequest addNamespaceItem(String namespaceItem) { + if (this.namespace == null) { + this.namespace = new ArrayList<>(); + } + this.namespace.add(namespaceItem); + return this; + } + + /** + * Get namespace + * + * @return namespace + */ + @NotNull + @Schema(name = "namespace", requiredMode = Schema.RequiredMode.REQUIRED) + @JsonProperty("namespace") + public List getNamespace() { + return namespace; + } + + public void setNamespace(List namespace) { + this.namespace = namespace; + } + public RegisterTableRequest location(String location) { this.location = location; return this; @@ -94,12 +129,13 @@ public boolean equals(Object o) { } RegisterTableRequest registerTableRequest = (RegisterTableRequest) o; return Objects.equals(this.name, registerTableRequest.name) + && Objects.equals(this.namespace, registerTableRequest.namespace) && Objects.equals(this.location, registerTableRequest.location); } @Override public int hashCode() { - return Objects.hash(name, location); + return Objects.hash(name, namespace, location); } @Override @@ -107,6 +143,7 @@ public String toString() { StringBuilder sb = new StringBuilder(); sb.append("class RegisterTableRequest {\n"); sb.append(" name: ").append(toIndentedString(name)).append("\n"); + sb.append(" namespace: ").append(toIndentedString(namespace)).append("\n"); sb.append(" location: ").append(toIndentedString(location)).append("\n"); sb.append("}"); return sb.toString(); diff --git a/python/Makefile b/python/Makefile index 816de3a70..bfd2dee54 100644 --- a/python/Makefile +++ b/python/Makefile @@ -17,7 +17,7 @@ clean-python-urllib3-client: gen-python-urllib3-client: clean-python-urllib3-client openapi-generator-cli generate \ - -i ../spec/catalog.yaml \ + -i ../spec/rest.yaml \ -g python \ -o lance_catalog_urllib3_client \ --additional-properties=packageName=lance_catalog_urllib3_client,packageVersion=$(VERSION),library=urllib3 diff --git a/python/lance_catalog_urllib3_client/.openapi-generator/FILES b/python/lance_catalog_urllib3_client/.openapi-generator/FILES index cc57bbd7f..8e8e1a025 100644 --- a/python/lance_catalog_urllib3_client/.openapi-generator/FILES +++ b/python/lance_catalog_urllib3_client/.openapi-generator/FILES @@ -4,7 +4,6 @@ .travis.yml README.md docs/CreateNamespaceRequest.md -docs/CreateNamespaceResponse.md docs/ErrorResponse.md docs/GetNamespaceResponse.md docs/GetTableResponse.md @@ -23,7 +22,6 @@ lance_catalog_urllib3_client/configuration.py lance_catalog_urllib3_client/exceptions.py lance_catalog_urllib3_client/models/__init__.py lance_catalog_urllib3_client/models/create_namespace_request.py -lance_catalog_urllib3_client/models/create_namespace_response.py lance_catalog_urllib3_client/models/error_response.py lance_catalog_urllib3_client/models/get_namespace_response.py lance_catalog_urllib3_client/models/get_table_response.py diff --git a/python/lance_catalog_urllib3_client/README.md b/python/lance_catalog_urllib3_client/README.md index 260fbd560..b2577e7e1 100644 --- a/python/lance_catalog_urllib3_client/README.md +++ b/python/lance_catalog_urllib3_client/README.md @@ -1,10 +1,7 @@ # lance-catalog-urllib3-client -**Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. -It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. -to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, -which is a REST server that converts the Lance catalog requests to native requests against the catalog service. -Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, -and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. +**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. +A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. +In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). This Python package is automatically generated by the [OpenAPI Generator](https://openapi-generator.tech) project: @@ -76,7 +73,7 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: create_namespace_request = lance_catalog_urllib3_client.CreateNamespaceRequest() # CreateNamespaceRequest | try: - # Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + # Create a new namespace api_response = api_instance.create_namespace(create_namespace_request) print("The response of NamespaceApi->create_namespace:\n") pprint(api_response) @@ -91,20 +88,19 @@ All URIs are relative to *http://localhost:2333* Class | Method | HTTP request | Description ------------ | ------------- | ------------- | ------------- -*NamespaceApi* | [**create_namespace**](docs/NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. -*NamespaceApi* | [**drop_namespace**](docs/NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace from the catalog. Namespace must be empty. -*NamespaceApi* | [**get_namespace**](docs/NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace -*NamespaceApi* | [**list_namespaces**](docs/NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List all namespaces in the catalog. -*NamespaceApi* | [**namespace_exists**](docs/NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists -*TableApi* | [**get_table**](docs/TableApi.md#get_table) | **GET** /v1/namespaces/{ns}/tables/{table} | Get a table from the catalog -*TableApi* | [**register_table**](docs/TableApi.md#register_table) | **POST** /v1/namespaces/{ns}/register | Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. -*TableApi* | [**table_exists**](docs/TableApi.md#table_exists) | **HEAD** /v1/namespaces/{ns}/tables/{table} | Check if a table exists +*NamespaceApi* | [**create_namespace**](docs/NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace +*NamespaceApi* | [**drop_namespace**](docs/NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{namespace} | Drop a namespace +*NamespaceApi* | [**get_namespace**](docs/NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{namespace} | Get information about a namespace +*NamespaceApi* | [**list_namespaces**](docs/NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List namespaces +*NamespaceApi* | [**namespace_exists**](docs/NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{namespace} | Check if a namespace exists +*TableApi* | [**get_table**](docs/TableApi.md#get_table) | **GET** /v1/tables/{table} | Get a table from the namespace +*TableApi* | [**register_table**](docs/TableApi.md#register_table) | **POST** /v1/table/register | Register a table to a namespace +*TableApi* | [**table_exists**](docs/TableApi.md#table_exists) | **HEAD** /v1/tables/{table} | Check if a table exists ## Documentation For Models - [CreateNamespaceRequest](docs/CreateNamespaceRequest.md) - - [CreateNamespaceResponse](docs/CreateNamespaceResponse.md) - [ErrorResponse](docs/ErrorResponse.md) - [GetNamespaceResponse](docs/GetNamespaceResponse.md) - [GetTableResponse](docs/GetTableResponse.md) diff --git a/python/lance_catalog_urllib3_client/docs/CatalogApi.md b/python/lance_catalog_urllib3_client/docs/CatalogApi.md new file mode 100644 index 000000000..816c15daa --- /dev/null +++ b/python/lance_catalog_urllib3_client/docs/CatalogApi.md @@ -0,0 +1,390 @@ +# lance_catalog_urllib3_client.CatalogApi + +All URIs are relative to *http://localhost:2333* + +Method | HTTP request | Description +------------- | ------------- | ------------- +[**catalog_exists**](CatalogApi.md#catalog_exists) | **HEAD** /v1/catalogs/{catalog} | Check if a catalog exists +[**create_catalog**](CatalogApi.md#create_catalog) | **POST** /v1/catalogs | Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. +[**drop_catalog**](CatalogApi.md#drop_catalog) | **DELETE** /v1/catalogs/{catalog} | Drop a catalog. The catalog must be empty. +[**get_catalog**](CatalogApi.md#get_catalog) | **GET** /v1/catalogs/{catalog} | Get information about a catalog +[**list_catalogs**](CatalogApi.md#list_catalogs) | **GET** /v1/catalogs | List all direct child catalogs of the root catalog. + + +# **catalog_exists** +> catalog_exists(catalog, catalog_delimiter=catalog_delimiter) + +Check if a catalog exists + +Check if a catalog exists. The response does not contain a body. + +### Example + + +```python +import lance_catalog_urllib3_client +from lance_catalog_urllib3_client.rest import ApiException +from pprint import pprint + +# Defining the host is optional and defaults to http://localhost:2333 +# See configuration.py for a list of all supported configuration parameters. +configuration = lance_catalog_urllib3_client.Configuration( + host = "http://localhost:2333" +) + + +# Enter a context with an instance of the API client +with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: + # Create an instance of the API class + api_instance = lance_catalog_urllib3_client.CatalogApi(api_client) + catalog = 'catalog_example' # str | An identifier of the catalog. + catalog_delimiter = '.' # str | The delimiter used by the catalog identifier string (optional) (default to '.') + + try: + # Check if a catalog exists + api_instance.catalog_exists(catalog, catalog_delimiter=catalog_delimiter) + except Exception as e: + print("Exception when calling CatalogApi->catalog_exists: %s\n" % e) +``` + + + +### Parameters + + +Name | Type | Description | Notes +------------- | ------------- | ------------- | ------------- + **catalog** | **str**| An identifier of the catalog. | + **catalog_delimiter** | **str**| The delimiter used by the catalog identifier string | [optional] [default to '.'] + +### Return type + +void (empty response body) + +### Authorization + +No authorization required + +### HTTP request headers + + - **Content-Type**: Not defined + - **Accept**: application/json + +### HTTP response details + +| Status code | Description | Response headers | +|-------------|-------------|------------------| +**200** | Success, no content | - | +**400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | +**401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | +**403** | Forbidden. Authenticated user does not have the necessary permissions. | - | +**404** | A server-side problem that means can not find the specified resource. | - | +**503** | The service is not ready to handle the request. The client should wait and retry. The service may additionally send a Retry-After header to indicate when to retry. | - | +**5XX** | A server-side problem that might not be addressable from the client side. Used for server 5xx errors without more specific documentation in individual routes. | - | + +[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) + +# **create_catalog** +> CreateCatalogResponse create_catalog(create_catalog_request, parent_catalog=parent_catalog, parent_catalog_delimiter=parent_catalog_delimiter) + +Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. + +### Example + + +```python +import lance_catalog_urllib3_client +from lance_catalog_urllib3_client.models.create_catalog_request import CreateCatalogRequest +from lance_catalog_urllib3_client.models.create_catalog_response import CreateCatalogResponse +from lance_catalog_urllib3_client.rest import ApiException +from pprint import pprint + +# Defining the host is optional and defaults to http://localhost:2333 +# See configuration.py for a list of all supported configuration parameters. +configuration = lance_catalog_urllib3_client.Configuration( + host = "http://localhost:2333" +) + + +# Enter a context with an instance of the API client +with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: + # Create an instance of the API class + api_instance = lance_catalog_urllib3_client.CatalogApi(api_client) + create_catalog_request = lance_catalog_urllib3_client.CreateCatalogRequest() # CreateCatalogRequest | + parent_catalog = 'parent_catalog_example' # str | An identifier of the parent catalog. (optional) + parent_catalog_delimiter = '.' # str | The delimiter used by the parent catalog identifier (optional) (default to '.') + + try: + # Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. + api_response = api_instance.create_catalog(create_catalog_request, parent_catalog=parent_catalog, parent_catalog_delimiter=parent_catalog_delimiter) + print("The response of CatalogApi->create_catalog:\n") + pprint(api_response) + except Exception as e: + print("Exception when calling CatalogApi->create_catalog: %s\n" % e) +``` + + + +### Parameters + + +Name | Type | Description | Notes +------------- | ------------- | ------------- | ------------- + **create_catalog_request** | [**CreateCatalogRequest**](CreateCatalogRequest.md)| | + **parent_catalog** | **str**| An identifier of the parent catalog. | [optional] + **parent_catalog_delimiter** | **str**| The delimiter used by the parent catalog identifier | [optional] [default to '.'] + +### Return type + +[**CreateCatalogResponse**](CreateCatalogResponse.md) + +### Authorization + +No authorization required + +### HTTP request headers + + - **Content-Type**: application/json + - **Accept**: application/json + +### HTTP response details + +| Status code | Description | Response headers | +|-------------|-------------|------------------| +**200** | Represents a successful call to create a catalog. Returns the catalog created, as well as any properties that were stored for the catalog, including those the server might have added. Implementations are not required to support catalog properties. | - | +**400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | +**401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | +**403** | Forbidden. Authenticated user does not have the necessary permissions. | - | +**406** | Not Acceptable / Unsupported Operation. The server does not support this operation. | - | +**409** | The request conflicts with the current state of the target resource. | - | +**503** | The service is not ready to handle the request. The client should wait and retry. The service may additionally send a Retry-After header to indicate when to retry. | - | +**5XX** | A server-side problem that might not be addressable from the client side. Used for server 5xx errors without more specific documentation in individual routes. | - | + +[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) + +# **drop_catalog** +> drop_catalog(catalog, catalog_delimiter=catalog_delimiter) + +Drop a catalog. The catalog must be empty. + +### Example + + +```python +import lance_catalog_urllib3_client +from lance_catalog_urllib3_client.rest import ApiException +from pprint import pprint + +# Defining the host is optional and defaults to http://localhost:2333 +# See configuration.py for a list of all supported configuration parameters. +configuration = lance_catalog_urllib3_client.Configuration( + host = "http://localhost:2333" +) + + +# Enter a context with an instance of the API client +with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: + # Create an instance of the API class + api_instance = lance_catalog_urllib3_client.CatalogApi(api_client) + catalog = 'catalog_example' # str | An identifier of the catalog. + catalog_delimiter = '.' # str | The delimiter used by the catalog identifier string (optional) (default to '.') + + try: + # Drop a catalog. The catalog must be empty. + api_instance.drop_catalog(catalog, catalog_delimiter=catalog_delimiter) + except Exception as e: + print("Exception when calling CatalogApi->drop_catalog: %s\n" % e) +``` + + + +### Parameters + + +Name | Type | Description | Notes +------------- | ------------- | ------------- | ------------- + **catalog** | **str**| An identifier of the catalog. | + **catalog_delimiter** | **str**| The delimiter used by the catalog identifier string | [optional] [default to '.'] + +### Return type + +void (empty response body) + +### Authorization + +No authorization required + +### HTTP request headers + + - **Content-Type**: Not defined + - **Accept**: application/json + +### HTTP response details + +| Status code | Description | Response headers | +|-------------|-------------|------------------| +**204** | Success, no content | - | +**400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | +**401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | +**403** | Forbidden. Authenticated user does not have the necessary permissions. | - | +**404** | A server-side problem that means can not find the specified resource. | - | +**409** | The request conflicts with the current state of the target resource. | - | +**503** | The service is not ready to handle the request. The client should wait and retry. The service may additionally send a Retry-After header to indicate when to retry. | - | +**5XX** | A server-side problem that might not be addressable from the client side. Used for server 5xx errors without more specific documentation in individual routes. | - | + +[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) + +# **get_catalog** +> GetCatalogResponse get_catalog(catalog, catalog_delimiter=catalog_delimiter) + +Get information about a catalog + +Return a detailed information for a given catalog + +### Example + + +```python +import lance_catalog_urllib3_client +from lance_catalog_urllib3_client.models.get_catalog_response import GetCatalogResponse +from lance_catalog_urllib3_client.rest import ApiException +from pprint import pprint + +# Defining the host is optional and defaults to http://localhost:2333 +# See configuration.py for a list of all supported configuration parameters. +configuration = lance_catalog_urllib3_client.Configuration( + host = "http://localhost:2333" +) + + +# Enter a context with an instance of the API client +with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: + # Create an instance of the API class + api_instance = lance_catalog_urllib3_client.CatalogApi(api_client) + catalog = 'catalog_example' # str | An identifier of the catalog. + catalog_delimiter = '.' # str | The delimiter used by the catalog identifier string (optional) (default to '.') + + try: + # Get information about a catalog + api_response = api_instance.get_catalog(catalog, catalog_delimiter=catalog_delimiter) + print("The response of CatalogApi->get_catalog:\n") + pprint(api_response) + except Exception as e: + print("Exception when calling CatalogApi->get_catalog: %s\n" % e) +``` + + + +### Parameters + + +Name | Type | Description | Notes +------------- | ------------- | ------------- | ------------- + **catalog** | **str**| An identifier of the catalog. | + **catalog_delimiter** | **str**| The delimiter used by the catalog identifier string | [optional] [default to '.'] + +### Return type + +[**GetCatalogResponse**](GetCatalogResponse.md) + +### Authorization + +No authorization required + +### HTTP request headers + + - **Content-Type**: Not defined + - **Accept**: application/json + +### HTTP response details + +| Status code | Description | Response headers | +|-------------|-------------|------------------| +**200** | Returns a catalog, as well as any properties stored on the catalog if catalog properties are supported by the server. | - | +**400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | +**401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | +**403** | Forbidden. Authenticated user does not have the necessary permissions. | - | +**404** | A server-side problem that means can not find the specified resource. | - | +**503** | The service is not ready to handle the request. The client should wait and retry. The service may additionally send a Retry-After header to indicate when to retry. | - | +**5XX** | A server-side problem that might not be addressable from the client side. Used for server 5xx errors without more specific documentation in individual routes. | - | + +[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) + +# **list_catalogs** +> ListCatalogsResponse list_catalogs(page_token=page_token, page_size=page_size, parent_catalog=parent_catalog, parent_catalog_delimiter=parent_catalog_delimiter) + +List all direct child catalogs of the root catalog. + +### Example + + +```python +import lance_catalog_urllib3_client +from lance_catalog_urllib3_client.models.list_catalogs_response import ListCatalogsResponse +from lance_catalog_urllib3_client.rest import ApiException +from pprint import pprint + +# Defining the host is optional and defaults to http://localhost:2333 +# See configuration.py for a list of all supported configuration parameters. +configuration = lance_catalog_urllib3_client.Configuration( + host = "http://localhost:2333" +) + + +# Enter a context with an instance of the API client +with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: + # Create an instance of the API class + api_instance = lance_catalog_urllib3_client.CatalogApi(api_client) + page_token = 'page_token_example' # str | (optional) + page_size = 56 # int | An inclusive upper bound of the number of results that a client will receive. (optional) + parent_catalog = 'parent_catalog_example' # str | An identifier of the parent catalog. (optional) + parent_catalog_delimiter = '.' # str | The delimiter used by the parent catalog identifier (optional) (default to '.') + + try: + # List all direct child catalogs of the root catalog. + api_response = api_instance.list_catalogs(page_token=page_token, page_size=page_size, parent_catalog=parent_catalog, parent_catalog_delimiter=parent_catalog_delimiter) + print("The response of CatalogApi->list_catalogs:\n") + pprint(api_response) + except Exception as e: + print("Exception when calling CatalogApi->list_catalogs: %s\n" % e) +``` + + + +### Parameters + + +Name | Type | Description | Notes +------------- | ------------- | ------------- | ------------- + **page_token** | **str**| | [optional] + **page_size** | **int**| An inclusive upper bound of the number of results that a client will receive. | [optional] + **parent_catalog** | **str**| An identifier of the parent catalog. | [optional] + **parent_catalog_delimiter** | **str**| The delimiter used by the parent catalog identifier | [optional] [default to '.'] + +### Return type + +[**ListCatalogsResponse**](ListCatalogsResponse.md) + +### Authorization + +No authorization required + +### HTTP request headers + + - **Content-Type**: Not defined + - **Accept**: application/json + +### HTTP response details + +| Status code | Description | Response headers | +|-------------|-------------|------------------| +**200** | A list of catalogs | - | +**400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | +**401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | +**403** | Forbidden. Authenticated user does not have the necessary permissions. | - | +**406** | Not Acceptable / Unsupported Operation. The server does not support this operation. | - | +**503** | The service is not ready to handle the request. The client should wait and retry. The service may additionally send a Retry-After header to indicate when to retry. | - | +**5XX** | A server-side problem that might not be addressable from the client side. Used for server 5xx errors without more specific documentation in individual routes. | - | + +[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) + diff --git a/python/lance_catalog_urllib3_client/docs/CreateCatalogRequest.md b/python/lance_catalog_urllib3_client/docs/CreateCatalogRequest.md new file mode 100644 index 000000000..32b801dde --- /dev/null +++ b/python/lance_catalog_urllib3_client/docs/CreateCatalogRequest.md @@ -0,0 +1,31 @@ +# CreateCatalogRequest + + +## Properties + +Name | Type | Description | Notes +------------ | ------------- | ------------- | ------------- +**name** | **str** | | +**mode** | **str** | | +**options** | **Dict[str, str]** | | [optional] + +## Example + +```python +from lance_catalog_urllib3_client.models.create_catalog_request import CreateCatalogRequest + +# TODO update the JSON string below +json = "{}" +# create an instance of CreateCatalogRequest from a JSON string +create_catalog_request_instance = CreateCatalogRequest.from_json(json) +# print the JSON string representation of the object +print(CreateCatalogRequest.to_json()) + +# convert the object into a dict +create_catalog_request_dict = create_catalog_request_instance.to_dict() +# create an instance of CreateCatalogRequest from a dict +create_catalog_request_from_dict = CreateCatalogRequest.from_dict(create_catalog_request_dict) +``` +[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) + + diff --git a/python/lance_catalog_urllib3_client/docs/CreateCatalogResponse.md b/python/lance_catalog_urllib3_client/docs/CreateCatalogResponse.md new file mode 100644 index 000000000..cb3f92bc7 --- /dev/null +++ b/python/lance_catalog_urllib3_client/docs/CreateCatalogResponse.md @@ -0,0 +1,30 @@ +# CreateCatalogResponse + + +## Properties + +Name | Type | Description | Notes +------------ | ------------- | ------------- | ------------- +**name** | **str** | | +**properties** | **Dict[str, str]** | Properties stored on the catalog, if supported by the server. | [optional] + +## Example + +```python +from lance_catalog_urllib3_client.models.create_catalog_response import CreateCatalogResponse + +# TODO update the JSON string below +json = "{}" +# create an instance of CreateCatalogResponse from a JSON string +create_catalog_response_instance = CreateCatalogResponse.from_json(json) +# print the JSON string representation of the object +print(CreateCatalogResponse.to_json()) + +# convert the object into a dict +create_catalog_response_dict = create_catalog_response_instance.to_dict() +# create an instance of CreateCatalogResponse from a dict +create_catalog_response_from_dict = CreateCatalogResponse.from_dict(create_catalog_response_dict) +``` +[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) + + diff --git a/python/lance_catalog_urllib3_client/docs/CreateNamespaceRequest.md b/python/lance_catalog_urllib3_client/docs/CreateNamespaceRequest.md index b720ebf55..8562bfb4c 100644 --- a/python/lance_catalog_urllib3_client/docs/CreateNamespaceRequest.md +++ b/python/lance_catalog_urllib3_client/docs/CreateNamespaceRequest.md @@ -6,6 +6,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **name** | **str** | | +**parent** | **List[str]** | | [optional] **mode** | **str** | | **options** | **Dict[str, str]** | | [optional] diff --git a/python/lance_catalog_urllib3_client/docs/GetCatalogResponse.md b/python/lance_catalog_urllib3_client/docs/GetCatalogResponse.md new file mode 100644 index 000000000..a7ab293be --- /dev/null +++ b/python/lance_catalog_urllib3_client/docs/GetCatalogResponse.md @@ -0,0 +1,30 @@ +# GetCatalogResponse + + +## Properties + +Name | Type | Description | Notes +------------ | ------------- | ------------- | ------------- +**catalog** | **str** | | +**properties** | **Dict[str, str]** | Properties stored on the catalog, if supported by the server. If the server does not support catalog properties, it should return null for this field. If catalog properties are supported, but none are set, it should return an empty object. | [optional] + +## Example + +```python +from lance_catalog_urllib3_client.models.get_catalog_response import GetCatalogResponse + +# TODO update the JSON string below +json = "{}" +# create an instance of GetCatalogResponse from a JSON string +get_catalog_response_instance = GetCatalogResponse.from_json(json) +# print the JSON string representation of the object +print(GetCatalogResponse.to_json()) + +# convert the object into a dict +get_catalog_response_dict = get_catalog_response_instance.to_dict() +# create an instance of GetCatalogResponse from a dict +get_catalog_response_from_dict = GetCatalogResponse.from_dict(get_catalog_response_dict) +``` +[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) + + diff --git a/python/lance_catalog_urllib3_client/docs/GetNamespaceResponse.md b/python/lance_catalog_urllib3_client/docs/GetNamespaceResponse.md index 2d488d4c0..f2aa37f7a 100644 --- a/python/lance_catalog_urllib3_client/docs/GetNamespaceResponse.md +++ b/python/lance_catalog_urllib3_client/docs/GetNamespaceResponse.md @@ -5,7 +5,8 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- -**namespace** | **str** | | +**name** | **str** | | +**parent** | **List[str]** | | [optional] **properties** | **Dict[str, str]** | Properties stored on the namespace, if supported by the server. If the server does not support namespace properties, it should return null for this field. If namespace properties are supported, but none are set, it should return an empty object. | [optional] ## Example diff --git a/python/lance_catalog_urllib3_client/docs/GetTableResponse.md b/python/lance_catalog_urllib3_client/docs/GetTableResponse.md index 613e06f37..bd65a8276 100644 --- a/python/lance_catalog_urllib3_client/docs/GetTableResponse.md +++ b/python/lance_catalog_urllib3_client/docs/GetTableResponse.md @@ -1,12 +1,12 @@ # GetTableResponse -Result used when a table is successfully loaded. ## Properties Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **name** | **str** | | +**namespace** | **List[str]** | | **location** | **str** | | **properties** | **Dict[str, str]** | | [optional] diff --git a/python/lance_catalog_urllib3_client/docs/ListCatalogsResponse.md b/python/lance_catalog_urllib3_client/docs/ListCatalogsResponse.md new file mode 100644 index 000000000..5646c0055 --- /dev/null +++ b/python/lance_catalog_urllib3_client/docs/ListCatalogsResponse.md @@ -0,0 +1,30 @@ +# ListCatalogsResponse + + +## Properties + +Name | Type | Description | Notes +------------ | ------------- | ------------- | ------------- +**catalogs** | **List[str]** | An array of child catalog names in the catalog. | +**next_page_token** | **str** | An opaque token that allows pagination for list APIs (e.g. ListCatalogs). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results. | [optional] + +## Example + +```python +from lance_catalog_urllib3_client.models.list_catalogs_response import ListCatalogsResponse + +# TODO update the JSON string below +json = "{}" +# create an instance of ListCatalogsResponse from a JSON string +list_catalogs_response_instance = ListCatalogsResponse.from_json(json) +# print the JSON string representation of the object +print(ListCatalogsResponse.to_json()) + +# convert the object into a dict +list_catalogs_response_dict = list_catalogs_response_instance.to_dict() +# create an instance of ListCatalogsResponse from a dict +list_catalogs_response_from_dict = ListCatalogsResponse.from_dict(list_catalogs_response_dict) +``` +[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) + + diff --git a/python/lance_catalog_urllib3_client/docs/ListNamespacesResponse.md b/python/lance_catalog_urllib3_client/docs/ListNamespacesResponse.md index 1a4a0c150..83279975c 100644 --- a/python/lance_catalog_urllib3_client/docs/ListNamespacesResponse.md +++ b/python/lance_catalog_urllib3_client/docs/ListNamespacesResponse.md @@ -5,7 +5,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- -**namespaces** | **List[str]** | An array of namespace names in the catalog. | +**namespaces** | **List[str]** | | **next_page_token** | **str** | An opaque token that allows pagination for list APIs (e.g. ListNamespaces). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results. | [optional] ## Example diff --git a/python/lance_catalog_urllib3_client/docs/NamespaceApi.md b/python/lance_catalog_urllib3_client/docs/NamespaceApi.md index cbc78a19e..6bb71ad6d 100644 --- a/python/lance_catalog_urllib3_client/docs/NamespaceApi.md +++ b/python/lance_catalog_urllib3_client/docs/NamespaceApi.md @@ -4,17 +4,25 @@ All URIs are relative to *http://localhost:2333* Method | HTTP request | Description ------------- | ------------- | ------------- -[**create_namespace**](NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. -[**drop_namespace**](NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace from the catalog. Namespace must be empty. -[**get_namespace**](NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace -[**list_namespaces**](NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List all namespaces in the catalog. -[**namespace_exists**](NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists +[**create_namespace**](NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace +[**drop_namespace**](NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{namespace} | Drop a namespace +[**get_namespace**](NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{namespace} | Get information about a namespace +[**list_namespaces**](NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List namespaces +[**namespace_exists**](NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{namespace} | Check if a namespace exists # **create_namespace** -> CreateNamespaceResponse create_namespace(create_namespace_request) +> GetNamespaceResponse create_namespace(create_namespace_request) + +Create a new namespace + +Create a new namespace. +A namespace can manage either a collection of child namespaces, or a collection of tables. +There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: + * CREATE: the operation fails with 400. + * EXIST_OK: the operation succeeds and the existing namespace is kept. + * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. -Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. ### Example @@ -22,7 +30,7 @@ Create a new namespace. A catalog can manage one or more namespaces. A namespace ```python import lance_catalog_urllib3_client from lance_catalog_urllib3_client.models.create_namespace_request import CreateNamespaceRequest -from lance_catalog_urllib3_client.models.create_namespace_response import CreateNamespaceResponse +from lance_catalog_urllib3_client.models.get_namespace_response import GetNamespaceResponse from lance_catalog_urllib3_client.rest import ApiException from pprint import pprint @@ -40,7 +48,7 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: create_namespace_request = lance_catalog_urllib3_client.CreateNamespaceRequest() # CreateNamespaceRequest | try: - # Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + # Create a new namespace api_response = api_instance.create_namespace(create_namespace_request) print("The response of NamespaceApi->create_namespace:\n") pprint(api_response) @@ -59,7 +67,7 @@ Name | Type | Description | Notes ### Return type -[**CreateNamespaceResponse**](CreateNamespaceResponse.md) +[**GetNamespaceResponse**](GetNamespaceResponse.md) ### Authorization @@ -74,7 +82,7 @@ No authorization required | Status code | Description | Response headers | |-------------|-------------|------------------| -**200** | Represents a successful call to create a namespace. Returns the namespace created, as well as any properties that were stored for the namespace, including those the server might have added. Implementations are not required to support namespace properties. | - | +**200** | Returns a namespace, as well as any properties stored on the namespace if namespace properties are supported by the server. | - | **400** | Indicates a bad request error. It could be caused by an unexpected request body format or other forms of request validation failure, such as invalid json. Usually serves application/json content, although in some cases simple text/plain content might be returned by the server's middleware. | - | **401** | Unauthorized. The request lacks valid authentication credentials for the operation. | - | **403** | Forbidden. Authenticated user does not have the necessary permissions. | - | @@ -86,9 +94,12 @@ No authorization required [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **drop_namespace** -> drop_namespace(ns) +> drop_namespace(namespace, delimiter=delimiter) + +Drop a namespace + +Drop a namespace. The namespace must be empty. -Drop a namespace from the catalog. Namespace must be empty. ### Example @@ -109,11 +120,12 @@ configuration = lance_catalog_urllib3_client.Configuration( with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = lance_catalog_urllib3_client.NamespaceApi(api_client) - ns = 'ns_example' # str | The name of the namespace. + namespace = 'namespace_example' # str | A string identifier of the namespace. + delimiter = 'delimiter_example' # str | The delimiter for the identifier used in the context (optional) try: - # Drop a namespace from the catalog. Namespace must be empty. - api_instance.drop_namespace(ns) + # Drop a namespace + api_instance.drop_namespace(namespace, delimiter=delimiter) except Exception as e: print("Exception when calling NamespaceApi->drop_namespace: %s\n" % e) ``` @@ -125,7 +137,8 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- - **ns** | **str**| The name of the namespace. | + **namespace** | **str**| A string identifier of the namespace. | + **delimiter** | **str**| The delimiter for the identifier used in the context | [optional] ### Return type @@ -156,11 +169,12 @@ No authorization required [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **get_namespace** -> GetNamespaceResponse get_namespace(ns) +> GetNamespaceResponse get_namespace(namespace, delimiter=delimiter) Get information about a namespace -Return a detailed information for a given namespace +Return the detailed information for a given namespace + ### Example @@ -182,11 +196,12 @@ configuration = lance_catalog_urllib3_client.Configuration( with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = lance_catalog_urllib3_client.NamespaceApi(api_client) - ns = 'ns_example' # str | The name of the namespace. + namespace = 'namespace_example' # str | A string identifier of the namespace. + delimiter = 'delimiter_example' # str | The delimiter for the identifier used in the context (optional) try: # Get information about a namespace - api_response = api_instance.get_namespace(ns) + api_response = api_instance.get_namespace(namespace, delimiter=delimiter) print("The response of NamespaceApi->get_namespace:\n") pprint(api_response) except Exception as e: @@ -200,7 +215,8 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- - **ns** | **str**| The name of the namespace. | + **namespace** | **str**| A string identifier of the namespace. | + **delimiter** | **str**| The delimiter for the identifier used in the context | [optional] ### Return type @@ -230,9 +246,12 @@ No authorization required [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **list_namespaces** -> ListNamespacesResponse list_namespaces(page_token=page_token, page_size=page_size) +> ListNamespacesResponse list_namespaces(page_token=page_token, page_size=page_size, parent=parent, delimiter=delimiter) + +List namespaces + +List all child namespace names of the root namespace or a given parent namespace. -List all namespaces in the catalog. ### Example @@ -256,10 +275,12 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: api_instance = lance_catalog_urllib3_client.NamespaceApi(api_client) page_token = 'page_token_example' # str | (optional) page_size = 56 # int | An inclusive upper bound of the number of results that a client will receive. (optional) + parent = 'parent_example' # str | A string identifier of the parent namespace. (optional) + delimiter = 'delimiter_example' # str | The delimiter for the identifier used in the context (optional) try: - # List all namespaces in the catalog. - api_response = api_instance.list_namespaces(page_token=page_token, page_size=page_size) + # List namespaces + api_response = api_instance.list_namespaces(page_token=page_token, page_size=page_size, parent=parent, delimiter=delimiter) print("The response of NamespaceApi->list_namespaces:\n") pprint(api_response) except Exception as e: @@ -275,6 +296,8 @@ Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **page_token** | **str**| | [optional] **page_size** | **int**| An inclusive upper bound of the number of results that a client will receive. | [optional] + **parent** | **str**| A string identifier of the parent namespace. | [optional] + **delimiter** | **str**| The delimiter for the identifier used in the context | [optional] ### Return type @@ -304,11 +327,13 @@ No authorization required [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **namespace_exists** -> namespace_exists(ns) +> namespace_exists(namespace, delimiter=delimiter) Check if a namespace exists -Check if a namespace exists. The response does not contain a body. +Check if a namespace exists. +This API should behave exactly like the GetNamespace API, except it does not contain a body. + ### Example @@ -329,11 +354,12 @@ configuration = lance_catalog_urllib3_client.Configuration( with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = lance_catalog_urllib3_client.NamespaceApi(api_client) - ns = 'ns_example' # str | The name of the namespace. + namespace = 'namespace_example' # str | A string identifier of the namespace. + delimiter = 'delimiter_example' # str | The delimiter for the identifier used in the context (optional) try: # Check if a namespace exists - api_instance.namespace_exists(ns) + api_instance.namespace_exists(namespace, delimiter=delimiter) except Exception as e: print("Exception when calling NamespaceApi->namespace_exists: %s\n" % e) ``` @@ -345,7 +371,8 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- - **ns** | **str**| The name of the namespace. | + **namespace** | **str**| A string identifier of the namespace. | + **delimiter** | **str**| The delimiter for the identifier used in the context | [optional] ### Return type diff --git a/python/lance_catalog_urllib3_client/docs/RegisterTableRequest.md b/python/lance_catalog_urllib3_client/docs/RegisterTableRequest.md index 7f3133e62..22d3447c2 100644 --- a/python/lance_catalog_urllib3_client/docs/RegisterTableRequest.md +++ b/python/lance_catalog_urllib3_client/docs/RegisterTableRequest.md @@ -6,6 +6,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **name** | **str** | | +**namespace** | **List[str]** | | **location** | **str** | | ## Example diff --git a/python/lance_catalog_urllib3_client/docs/TableApi.md b/python/lance_catalog_urllib3_client/docs/TableApi.md index c89f76d18..8fd16b953 100644 --- a/python/lance_catalog_urllib3_client/docs/TableApi.md +++ b/python/lance_catalog_urllib3_client/docs/TableApi.md @@ -4,17 +4,18 @@ All URIs are relative to *http://localhost:2333* Method | HTTP request | Description ------------- | ------------- | ------------- -[**get_table**](TableApi.md#get_table) | **GET** /v1/namespaces/{ns}/tables/{table} | Get a table from the catalog -[**register_table**](TableApi.md#register_table) | **POST** /v1/namespaces/{ns}/register | Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. -[**table_exists**](TableApi.md#table_exists) | **HEAD** /v1/namespaces/{ns}/tables/{table} | Check if a table exists +[**get_table**](TableApi.md#get_table) | **GET** /v1/tables/{table} | Get a table from the namespace +[**register_table**](TableApi.md#register_table) | **POST** /v1/table/register | Register a table to a namespace +[**table_exists**](TableApi.md#table_exists) | **HEAD** /v1/tables/{table} | Check if a table exists # **get_table** -> GetTableResponse get_table(ns, table) +> GetTableResponse get_table(table, delimiter=delimiter) -Get a table from the catalog +Get a table from the namespace + +Get a table's detailed information. -Get a table's detailed information under a specified namespace from the catalog. ### Example @@ -36,12 +37,12 @@ configuration = lance_catalog_urllib3_client.Configuration( with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = lance_catalog_urllib3_client.TableApi(api_client) - ns = 'ns_example' # str | The name of the namespace. - table = 'table_example' # str | A table name. + table = 'table_example' # str | A string identifier of the table + delimiter = 'delimiter_example' # str | The delimiter for the identifier used in the context (optional) try: - # Get a table from the catalog - api_response = api_instance.get_table(ns, table) + # Get a table from the namespace + api_response = api_instance.get_table(table, delimiter=delimiter) print("The response of TableApi->get_table:\n") pprint(api_response) except Exception as e: @@ -55,8 +56,8 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- - **ns** | **str**| The name of the namespace. | - **table** | **str**| A table name. | + **table** | **str**| A string identifier of the table | + **delimiter** | **str**| The delimiter for the identifier used in the context | [optional] ### Return type @@ -86,9 +87,12 @@ No authorization required [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **register_table** -> GetTableResponse register_table(ns, register_table_request) +> GetTableResponse register_table(register_table_request) + +Register a table to a namespace + +Register an existing table at a given storage location to a namespace. -Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. ### Example @@ -111,12 +115,11 @@ configuration = lance_catalog_urllib3_client.Configuration( with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = lance_catalog_urllib3_client.TableApi(api_client) - ns = 'ns_example' # str | The name of the namespace. register_table_request = lance_catalog_urllib3_client.RegisterTableRequest() # RegisterTableRequest | try: - # Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. - api_response = api_instance.register_table(ns, register_table_request) + # Register a table to a namespace + api_response = api_instance.register_table(register_table_request) print("The response of TableApi->register_table:\n") pprint(api_response) except Exception as e: @@ -130,7 +133,6 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- - **ns** | **str**| The name of the namespace. | **register_table_request** | [**RegisterTableRequest**](RegisterTableRequest.md)| | ### Return type @@ -162,11 +164,13 @@ No authorization required [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **table_exists** -> table_exists(ns, table) +> table_exists(table, delimiter=delimiter) Check if a table exists -Check if a table exists within a given namespace. +Check if a table exists. +This API should behave exactly like the GetTable API, except it does not contain a body. + ### Example @@ -187,12 +191,12 @@ configuration = lance_catalog_urllib3_client.Configuration( with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = lance_catalog_urllib3_client.TableApi(api_client) - ns = 'ns_example' # str | The name of the namespace. - table = 'table_example' # str | A table name. + table = 'table_example' # str | A string identifier of the table + delimiter = 'delimiter_example' # str | The delimiter for the identifier used in the context (optional) try: # Check if a table exists - api_instance.table_exists(ns, table) + api_instance.table_exists(table, delimiter=delimiter) except Exception as e: print("Exception when calling TableApi->table_exists: %s\n" % e) ``` @@ -204,8 +208,8 @@ with lance_catalog_urllib3_client.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- - **ns** | **str**| The name of the namespace. | - **table** | **str**| A table name. | + **table** | **str**| A string identifier of the table | + **delimiter** | **str**| The delimiter for the identifier used in the context | [optional] ### Return type diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/__init__.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/__init__.py index 043e35bfa..420c1ffdc 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/__init__.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/__init__.py @@ -3,9 +3,9 @@ # flake8: noqa """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -33,7 +33,6 @@ # import models into sdk package from lance_catalog_urllib3_client.models.create_namespace_request import CreateNamespaceRequest -from lance_catalog_urllib3_client.models.create_namespace_response import CreateNamespaceResponse from lance_catalog_urllib3_client.models.error_response import ErrorResponse from lance_catalog_urllib3_client.models.get_namespace_response import GetNamespaceResponse from lance_catalog_urllib3_client.models.get_table_response import GetTableResponse diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/catalog_api.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/catalog_api.py new file mode 100644 index 000000000..de53a5a45 --- /dev/null +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/catalog_api.py @@ -0,0 +1,1580 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog Specification** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance catalogs recursively. It is designed to encapsulates concepts like namespace, metastore, database, schema, etc. that could appear in other similar systems, so that it can better integrate with any system with any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those catalog services or building a custom catalog server in a standardized way. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + +import warnings +from pydantic import validate_call, Field, StrictFloat, StrictStr, StrictInt +from typing import Any, Dict, List, Optional, Tuple, Union +from typing_extensions import Annotated + +from pydantic import Field, StrictStr +from typing import Optional +from typing_extensions import Annotated +from lance_catalog_urllib3_client.models.create_catalog_request import CreateCatalogRequest +from lance_catalog_urllib3_client.models.create_catalog_response import CreateCatalogResponse +from lance_catalog_urllib3_client.models.get_catalog_response import GetCatalogResponse +from lance_catalog_urllib3_client.models.list_catalogs_response import ListCatalogsResponse + +from lance_catalog_urllib3_client.api_client import ApiClient, RequestSerialized +from lance_catalog_urllib3_client.api_response import ApiResponse +from lance_catalog_urllib3_client.rest import RESTResponseType + + +class CatalogApi: + """NOTE: This class is auto generated by OpenAPI Generator + Ref: https://openapi-generator.tech + + Do not edit the class manually. + """ + + def __init__(self, api_client=None) -> None: + if api_client is None: + api_client = ApiClient.get_default() + self.api_client = api_client + + + @validate_call + def catalog_exists( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> None: + """Check if a catalog exists + + Check if a catalog exists. The response does not contain a body. + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._catalog_exists_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': None, + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ).data + + + @validate_call + def catalog_exists_with_http_info( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> ApiResponse[None]: + """Check if a catalog exists + + Check if a catalog exists. The response does not contain a body. + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._catalog_exists_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': None, + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ) + + + @validate_call + def catalog_exists_without_preload_content( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> RESTResponseType: + """Check if a catalog exists + + Check if a catalog exists. The response does not contain a body. + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._catalog_exists_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': None, + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + return response_data.response + + + def _catalog_exists_serialize( + self, + catalog, + catalog_delimiter, + _request_auth, + _content_type, + _headers, + _host_index, + ) -> RequestSerialized: + + _host = None + + _collection_formats: Dict[str, str] = { + } + + _path_params: Dict[str, str] = {} + _query_params: List[Tuple[str, str]] = [] + _header_params: Dict[str, Optional[str]] = _headers or {} + _form_params: List[Tuple[str, str]] = [] + _files: Dict[ + str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]] + ] = {} + _body_params: Optional[bytes] = None + + # process the path parameters + if catalog is not None: + _path_params['catalog'] = catalog + # process the query parameters + if catalog_delimiter is not None: + + _query_params.append(('catalogDelimiter', catalog_delimiter)) + + # process the header parameters + # process the form parameters + # process the body parameter + + + # set the HTTP header `Accept` + if 'Accept' not in _header_params: + _header_params['Accept'] = self.api_client.select_header_accept( + [ + 'application/json' + ] + ) + + + # authentication setting + _auth_settings: List[str] = [ + ] + + return self.api_client.param_serialize( + method='HEAD', + resource_path='/v1/catalogs/{catalog}', + path_params=_path_params, + query_params=_query_params, + header_params=_header_params, + body=_body_params, + post_params=_form_params, + files=_files, + auth_settings=_auth_settings, + collection_formats=_collection_formats, + _host=_host, + _request_auth=_request_auth + ) + + + + + @validate_call + def create_catalog( + self, + create_catalog_request: CreateCatalogRequest, + parent_catalog: Annotated[Optional[StrictStr], Field(description="An identifier of the parent catalog.")] = None, + parent_catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the parent catalog identifier")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> CreateCatalogResponse: + """Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. + + + :param create_catalog_request: (required) + :type create_catalog_request: CreateCatalogRequest + :param parent_catalog: An identifier of the parent catalog. + :type parent_catalog: str + :param parent_catalog_delimiter: The delimiter used by the parent catalog identifier + :type parent_catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._create_catalog_serialize( + create_catalog_request=create_catalog_request, + parent_catalog=parent_catalog, + parent_catalog_delimiter=parent_catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "CreateCatalogResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '406': "ErrorResponse", + '409': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ).data + + + @validate_call + def create_catalog_with_http_info( + self, + create_catalog_request: CreateCatalogRequest, + parent_catalog: Annotated[Optional[StrictStr], Field(description="An identifier of the parent catalog.")] = None, + parent_catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the parent catalog identifier")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> ApiResponse[CreateCatalogResponse]: + """Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. + + + :param create_catalog_request: (required) + :type create_catalog_request: CreateCatalogRequest + :param parent_catalog: An identifier of the parent catalog. + :type parent_catalog: str + :param parent_catalog_delimiter: The delimiter used by the parent catalog identifier + :type parent_catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._create_catalog_serialize( + create_catalog_request=create_catalog_request, + parent_catalog=parent_catalog, + parent_catalog_delimiter=parent_catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "CreateCatalogResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '406': "ErrorResponse", + '409': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ) + + + @validate_call + def create_catalog_without_preload_content( + self, + create_catalog_request: CreateCatalogRequest, + parent_catalog: Annotated[Optional[StrictStr], Field(description="An identifier of the parent catalog.")] = None, + parent_catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the parent catalog identifier")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> RESTResponseType: + """Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. + + + :param create_catalog_request: (required) + :type create_catalog_request: CreateCatalogRequest + :param parent_catalog: An identifier of the parent catalog. + :type parent_catalog: str + :param parent_catalog_delimiter: The delimiter used by the parent catalog identifier + :type parent_catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._create_catalog_serialize( + create_catalog_request=create_catalog_request, + parent_catalog=parent_catalog, + parent_catalog_delimiter=parent_catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "CreateCatalogResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '406': "ErrorResponse", + '409': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + return response_data.response + + + def _create_catalog_serialize( + self, + create_catalog_request, + parent_catalog, + parent_catalog_delimiter, + _request_auth, + _content_type, + _headers, + _host_index, + ) -> RequestSerialized: + + _host = None + + _collection_formats: Dict[str, str] = { + } + + _path_params: Dict[str, str] = {} + _query_params: List[Tuple[str, str]] = [] + _header_params: Dict[str, Optional[str]] = _headers or {} + _form_params: List[Tuple[str, str]] = [] + _files: Dict[ + str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]] + ] = {} + _body_params: Optional[bytes] = None + + # process the path parameters + # process the query parameters + if parent_catalog is not None: + + _query_params.append(('parentCatalog', parent_catalog)) + + if parent_catalog_delimiter is not None: + + _query_params.append(('parentCatalogDelimiter', parent_catalog_delimiter)) + + # process the header parameters + # process the form parameters + # process the body parameter + if create_catalog_request is not None: + _body_params = create_catalog_request + + + # set the HTTP header `Accept` + if 'Accept' not in _header_params: + _header_params['Accept'] = self.api_client.select_header_accept( + [ + 'application/json' + ] + ) + + # set the HTTP header `Content-Type` + if _content_type: + _header_params['Content-Type'] = _content_type + else: + _default_content_type = ( + self.api_client.select_header_content_type( + [ + 'application/json' + ] + ) + ) + if _default_content_type is not None: + _header_params['Content-Type'] = _default_content_type + + # authentication setting + _auth_settings: List[str] = [ + ] + + return self.api_client.param_serialize( + method='POST', + resource_path='/v1/catalogs', + path_params=_path_params, + query_params=_query_params, + header_params=_header_params, + body=_body_params, + post_params=_form_params, + files=_files, + auth_settings=_auth_settings, + collection_formats=_collection_formats, + _host=_host, + _request_auth=_request_auth + ) + + + + + @validate_call + def drop_catalog( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> None: + """Drop a catalog. The catalog must be empty. + + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._drop_catalog_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '204': None, + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '409': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ).data + + + @validate_call + def drop_catalog_with_http_info( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> ApiResponse[None]: + """Drop a catalog. The catalog must be empty. + + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._drop_catalog_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '204': None, + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '409': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ) + + + @validate_call + def drop_catalog_without_preload_content( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> RESTResponseType: + """Drop a catalog. The catalog must be empty. + + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._drop_catalog_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '204': None, + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '409': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + return response_data.response + + + def _drop_catalog_serialize( + self, + catalog, + catalog_delimiter, + _request_auth, + _content_type, + _headers, + _host_index, + ) -> RequestSerialized: + + _host = None + + _collection_formats: Dict[str, str] = { + } + + _path_params: Dict[str, str] = {} + _query_params: List[Tuple[str, str]] = [] + _header_params: Dict[str, Optional[str]] = _headers or {} + _form_params: List[Tuple[str, str]] = [] + _files: Dict[ + str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]] + ] = {} + _body_params: Optional[bytes] = None + + # process the path parameters + if catalog is not None: + _path_params['catalog'] = catalog + # process the query parameters + if catalog_delimiter is not None: + + _query_params.append(('catalogDelimiter', catalog_delimiter)) + + # process the header parameters + # process the form parameters + # process the body parameter + + + # set the HTTP header `Accept` + if 'Accept' not in _header_params: + _header_params['Accept'] = self.api_client.select_header_accept( + [ + 'application/json' + ] + ) + + + # authentication setting + _auth_settings: List[str] = [ + ] + + return self.api_client.param_serialize( + method='DELETE', + resource_path='/v1/catalogs/{catalog}', + path_params=_path_params, + query_params=_query_params, + header_params=_header_params, + body=_body_params, + post_params=_form_params, + files=_files, + auth_settings=_auth_settings, + collection_formats=_collection_formats, + _host=_host, + _request_auth=_request_auth + ) + + + + + @validate_call + def get_catalog( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> GetCatalogResponse: + """Get information about a catalog + + Return a detailed information for a given catalog + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._get_catalog_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "GetCatalogResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ).data + + + @validate_call + def get_catalog_with_http_info( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> ApiResponse[GetCatalogResponse]: + """Get information about a catalog + + Return a detailed information for a given catalog + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._get_catalog_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "GetCatalogResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ) + + + @validate_call + def get_catalog_without_preload_content( + self, + catalog: Annotated[StrictStr, Field(description="An identifier of the catalog.")], + catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the catalog identifier string")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> RESTResponseType: + """Get information about a catalog + + Return a detailed information for a given catalog + + :param catalog: An identifier of the catalog. (required) + :type catalog: str + :param catalog_delimiter: The delimiter used by the catalog identifier string + :type catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._get_catalog_serialize( + catalog=catalog, + catalog_delimiter=catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "GetCatalogResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '404': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + return response_data.response + + + def _get_catalog_serialize( + self, + catalog, + catalog_delimiter, + _request_auth, + _content_type, + _headers, + _host_index, + ) -> RequestSerialized: + + _host = None + + _collection_formats: Dict[str, str] = { + } + + _path_params: Dict[str, str] = {} + _query_params: List[Tuple[str, str]] = [] + _header_params: Dict[str, Optional[str]] = _headers or {} + _form_params: List[Tuple[str, str]] = [] + _files: Dict[ + str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]] + ] = {} + _body_params: Optional[bytes] = None + + # process the path parameters + if catalog is not None: + _path_params['catalog'] = catalog + # process the query parameters + if catalog_delimiter is not None: + + _query_params.append(('catalogDelimiter', catalog_delimiter)) + + # process the header parameters + # process the form parameters + # process the body parameter + + + # set the HTTP header `Accept` + if 'Accept' not in _header_params: + _header_params['Accept'] = self.api_client.select_header_accept( + [ + 'application/json' + ] + ) + + + # authentication setting + _auth_settings: List[str] = [ + ] + + return self.api_client.param_serialize( + method='GET', + resource_path='/v1/catalogs/{catalog}', + path_params=_path_params, + query_params=_query_params, + header_params=_header_params, + body=_body_params, + post_params=_form_params, + files=_files, + auth_settings=_auth_settings, + collection_formats=_collection_formats, + _host=_host, + _request_auth=_request_auth + ) + + + + + @validate_call + def list_catalogs( + self, + page_token: Optional[StrictStr] = None, + page_size: Annotated[Optional[Annotated[int, Field(strict=True, ge=1)]], Field(description="An inclusive upper bound of the number of results that a client will receive.")] = None, + parent_catalog: Annotated[Optional[StrictStr], Field(description="An identifier of the parent catalog.")] = None, + parent_catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the parent catalog identifier")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> ListCatalogsResponse: + """List all direct child catalogs of the root catalog. + + + :param page_token: + :type page_token: str + :param page_size: An inclusive upper bound of the number of results that a client will receive. + :type page_size: int + :param parent_catalog: An identifier of the parent catalog. + :type parent_catalog: str + :param parent_catalog_delimiter: The delimiter used by the parent catalog identifier + :type parent_catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._list_catalogs_serialize( + page_token=page_token, + page_size=page_size, + parent_catalog=parent_catalog, + parent_catalog_delimiter=parent_catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "ListCatalogsResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '406': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ).data + + + @validate_call + def list_catalogs_with_http_info( + self, + page_token: Optional[StrictStr] = None, + page_size: Annotated[Optional[Annotated[int, Field(strict=True, ge=1)]], Field(description="An inclusive upper bound of the number of results that a client will receive.")] = None, + parent_catalog: Annotated[Optional[StrictStr], Field(description="An identifier of the parent catalog.")] = None, + parent_catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the parent catalog identifier")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> ApiResponse[ListCatalogsResponse]: + """List all direct child catalogs of the root catalog. + + + :param page_token: + :type page_token: str + :param page_size: An inclusive upper bound of the number of results that a client will receive. + :type page_size: int + :param parent_catalog: An identifier of the parent catalog. + :type parent_catalog: str + :param parent_catalog_delimiter: The delimiter used by the parent catalog identifier + :type parent_catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._list_catalogs_serialize( + page_token=page_token, + page_size=page_size, + parent_catalog=parent_catalog, + parent_catalog_delimiter=parent_catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "ListCatalogsResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '406': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + response_data.read() + return self.api_client.response_deserialize( + response_data=response_data, + response_types_map=_response_types_map, + ) + + + @validate_call + def list_catalogs_without_preload_content( + self, + page_token: Optional[StrictStr] = None, + page_size: Annotated[Optional[Annotated[int, Field(strict=True, ge=1)]], Field(description="An inclusive upper bound of the number of results that a client will receive.")] = None, + parent_catalog: Annotated[Optional[StrictStr], Field(description="An identifier of the parent catalog.")] = None, + parent_catalog_delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter used by the parent catalog identifier")] = None, + _request_timeout: Union[ + None, + Annotated[StrictFloat, Field(gt=0)], + Tuple[ + Annotated[StrictFloat, Field(gt=0)], + Annotated[StrictFloat, Field(gt=0)] + ] + ] = None, + _request_auth: Optional[Dict[StrictStr, Any]] = None, + _content_type: Optional[StrictStr] = None, + _headers: Optional[Dict[StrictStr, Any]] = None, + _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, + ) -> RESTResponseType: + """List all direct child catalogs of the root catalog. + + + :param page_token: + :type page_token: str + :param page_size: An inclusive upper bound of the number of results that a client will receive. + :type page_size: int + :param parent_catalog: An identifier of the parent catalog. + :type parent_catalog: str + :param parent_catalog_delimiter: The delimiter used by the parent catalog identifier + :type parent_catalog_delimiter: str + :param _request_timeout: timeout setting for this request. If one + number provided, it will be total request + timeout. It can also be a pair (tuple) of + (connection, read) timeouts. + :type _request_timeout: int, tuple(int, int), optional + :param _request_auth: set to override the auth_settings for an a single + request; this effectively ignores the + authentication in the spec for a single request. + :type _request_auth: dict, optional + :param _content_type: force content-type for the request. + :type _content_type: str, Optional + :param _headers: set to override the headers for a single + request; this effectively ignores the headers + in the spec for a single request. + :type _headers: dict, optional + :param _host_index: set to override the host_index for a single + request; this effectively ignores the host_index + in the spec for a single request. + :type _host_index: int, optional + :return: Returns the result object. + """ # noqa: E501 + + _param = self._list_catalogs_serialize( + page_token=page_token, + page_size=page_size, + parent_catalog=parent_catalog, + parent_catalog_delimiter=parent_catalog_delimiter, + _request_auth=_request_auth, + _content_type=_content_type, + _headers=_headers, + _host_index=_host_index + ) + + _response_types_map: Dict[str, Optional[str]] = { + '200': "ListCatalogsResponse", + '400': "ErrorResponse", + '401': "ErrorResponse", + '403': "ErrorResponse", + '406': "ErrorResponse", + '503': "ErrorResponse", + '5XX': "ErrorResponse", + } + response_data = self.api_client.call_api( + *_param, + _request_timeout=_request_timeout + ) + return response_data.response + + + def _list_catalogs_serialize( + self, + page_token, + page_size, + parent_catalog, + parent_catalog_delimiter, + _request_auth, + _content_type, + _headers, + _host_index, + ) -> RequestSerialized: + + _host = None + + _collection_formats: Dict[str, str] = { + } + + _path_params: Dict[str, str] = {} + _query_params: List[Tuple[str, str]] = [] + _header_params: Dict[str, Optional[str]] = _headers or {} + _form_params: List[Tuple[str, str]] = [] + _files: Dict[ + str, Union[str, bytes, List[str], List[bytes], List[Tuple[str, bytes]]] + ] = {} + _body_params: Optional[bytes] = None + + # process the path parameters + # process the query parameters + if page_token is not None: + + _query_params.append(('pageToken', page_token)) + + if page_size is not None: + + _query_params.append(('pageSize', page_size)) + + if parent_catalog is not None: + + _query_params.append(('parentCatalog', parent_catalog)) + + if parent_catalog_delimiter is not None: + + _query_params.append(('parentCatalogDelimiter', parent_catalog_delimiter)) + + # process the header parameters + # process the form parameters + # process the body parameter + + + # set the HTTP header `Accept` + if 'Accept' not in _header_params: + _header_params['Accept'] = self.api_client.select_header_accept( + [ + 'application/json' + ] + ) + + + # authentication setting + _auth_settings: List[str] = [ + ] + + return self.api_client.param_serialize( + method='GET', + resource_path='/v1/catalogs', + path_params=_path_params, + query_params=_query_params, + header_params=_header_params, + body=_body_params, + post_params=_form_params, + files=_files, + auth_settings=_auth_settings, + collection_formats=_collection_formats, + _host=_host, + _request_auth=_request_auth + ) + + diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/namespace_api.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/namespace_api.py index 3b928d495..551421ce4 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/namespace_api.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/namespace_api.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -20,7 +20,6 @@ from typing import Optional from typing_extensions import Annotated from lance_catalog_urllib3_client.models.create_namespace_request import CreateNamespaceRequest -from lance_catalog_urllib3_client.models.create_namespace_response import CreateNamespaceResponse from lance_catalog_urllib3_client.models.get_namespace_response import GetNamespaceResponse from lance_catalog_urllib3_client.models.list_namespaces_response import ListNamespacesResponse @@ -58,9 +57,10 @@ def create_namespace( _content_type: Optional[StrictStr] = None, _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, - ) -> CreateNamespaceResponse: - """Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + ) -> GetNamespaceResponse: + """Create a new namespace + Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. :param create_namespace_request: (required) :type create_namespace_request: CreateNamespaceRequest @@ -95,7 +95,7 @@ def create_namespace( ) _response_types_map: Dict[str, Optional[str]] = { - '200': "CreateNamespaceResponse", + '200': "GetNamespaceResponse", '400': "ErrorResponse", '401': "ErrorResponse", '403': "ErrorResponse", @@ -131,9 +131,10 @@ def create_namespace_with_http_info( _content_type: Optional[StrictStr] = None, _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, - ) -> ApiResponse[CreateNamespaceResponse]: - """Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + ) -> ApiResponse[GetNamespaceResponse]: + """Create a new namespace + Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. :param create_namespace_request: (required) :type create_namespace_request: CreateNamespaceRequest @@ -168,7 +169,7 @@ def create_namespace_with_http_info( ) _response_types_map: Dict[str, Optional[str]] = { - '200': "CreateNamespaceResponse", + '200': "GetNamespaceResponse", '400': "ErrorResponse", '401': "ErrorResponse", '403': "ErrorResponse", @@ -205,8 +206,9 @@ def create_namespace_without_preload_content( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> RESTResponseType: - """Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + """Create a new namespace + Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. :param create_namespace_request: (required) :type create_namespace_request: CreateNamespaceRequest @@ -241,7 +243,7 @@ def create_namespace_without_preload_content( ) _response_types_map: Dict[str, Optional[str]] = { - '200': "CreateNamespaceResponse", + '200': "GetNamespaceResponse", '400': "ErrorResponse", '401': "ErrorResponse", '403': "ErrorResponse", @@ -336,7 +338,8 @@ def _create_namespace_serialize( @validate_call def drop_namespace( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -350,11 +353,14 @@ def drop_namespace( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> None: - """Drop a namespace from the catalog. Namespace must be empty. + """Drop a namespace + Drop a namespace. The namespace must be empty. - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -378,7 +384,8 @@ def drop_namespace( """ # noqa: E501 _param = self._drop_namespace_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -409,7 +416,8 @@ def drop_namespace( @validate_call def drop_namespace_with_http_info( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -423,11 +431,14 @@ def drop_namespace_with_http_info( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> ApiResponse[None]: - """Drop a namespace from the catalog. Namespace must be empty. + """Drop a namespace + Drop a namespace. The namespace must be empty. - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -451,7 +462,8 @@ def drop_namespace_with_http_info( """ # noqa: E501 _param = self._drop_namespace_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -482,7 +494,8 @@ def drop_namespace_with_http_info( @validate_call def drop_namespace_without_preload_content( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -496,11 +509,14 @@ def drop_namespace_without_preload_content( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> RESTResponseType: - """Drop a namespace from the catalog. Namespace must be empty. + """Drop a namespace + Drop a namespace. The namespace must be empty. - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -524,7 +540,8 @@ def drop_namespace_without_preload_content( """ # noqa: E501 _param = self._drop_namespace_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -550,7 +567,8 @@ def drop_namespace_without_preload_content( def _drop_namespace_serialize( self, - ns, + namespace, + delimiter, _request_auth, _content_type, _headers, @@ -572,9 +590,13 @@ def _drop_namespace_serialize( _body_params: Optional[bytes] = None # process the path parameters - if ns is not None: - _path_params['ns'] = ns + if namespace is not None: + _path_params['namespace'] = namespace # process the query parameters + if delimiter is not None: + + _query_params.append(('delimiter', delimiter)) + # process the header parameters # process the form parameters # process the body parameter @@ -595,7 +617,7 @@ def _drop_namespace_serialize( return self.api_client.param_serialize( method='DELETE', - resource_path='/v1/namespaces/{ns}', + resource_path='/v1/namespaces/{namespace}', path_params=_path_params, query_params=_query_params, header_params=_header_params, @@ -614,7 +636,8 @@ def _drop_namespace_serialize( @validate_call def get_namespace( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -630,10 +653,12 @@ def get_namespace( ) -> GetNamespaceResponse: """Get information about a namespace - Return a detailed information for a given namespace + Return the detailed information for a given namespace - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -657,7 +682,8 @@ def get_namespace( """ # noqa: E501 _param = self._get_namespace_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -687,7 +713,8 @@ def get_namespace( @validate_call def get_namespace_with_http_info( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -703,10 +730,12 @@ def get_namespace_with_http_info( ) -> ApiResponse[GetNamespaceResponse]: """Get information about a namespace - Return a detailed information for a given namespace + Return the detailed information for a given namespace - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -730,7 +759,8 @@ def get_namespace_with_http_info( """ # noqa: E501 _param = self._get_namespace_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -760,7 +790,8 @@ def get_namespace_with_http_info( @validate_call def get_namespace_without_preload_content( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -776,10 +807,12 @@ def get_namespace_without_preload_content( ) -> RESTResponseType: """Get information about a namespace - Return a detailed information for a given namespace + Return the detailed information for a given namespace - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -803,7 +836,8 @@ def get_namespace_without_preload_content( """ # noqa: E501 _param = self._get_namespace_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -828,7 +862,8 @@ def get_namespace_without_preload_content( def _get_namespace_serialize( self, - ns, + namespace, + delimiter, _request_auth, _content_type, _headers, @@ -850,9 +885,13 @@ def _get_namespace_serialize( _body_params: Optional[bytes] = None # process the path parameters - if ns is not None: - _path_params['ns'] = ns + if namespace is not None: + _path_params['namespace'] = namespace # process the query parameters + if delimiter is not None: + + _query_params.append(('delimiter', delimiter)) + # process the header parameters # process the form parameters # process the body parameter @@ -873,7 +912,7 @@ def _get_namespace_serialize( return self.api_client.param_serialize( method='GET', - resource_path='/v1/namespaces/{ns}', + resource_path='/v1/namespaces/{namespace}', path_params=_path_params, query_params=_query_params, header_params=_header_params, @@ -894,6 +933,8 @@ def list_namespaces( self, page_token: Optional[StrictStr] = None, page_size: Annotated[Optional[Annotated[int, Field(strict=True, ge=1)]], Field(description="An inclusive upper bound of the number of results that a client will receive.")] = None, + parent: Annotated[Optional[StrictStr], Field(description="A string identifier of the parent namespace.")] = None, + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -907,13 +948,18 @@ def list_namespaces( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> ListNamespacesResponse: - """List all namespaces in the catalog. + """List namespaces + List all child namespace names of the root namespace or a given parent namespace. :param page_token: :type page_token: str :param page_size: An inclusive upper bound of the number of results that a client will receive. :type page_size: int + :param parent: A string identifier of the parent namespace. + :type parent: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -939,6 +985,8 @@ def list_namespaces( _param = self._list_namespaces_serialize( page_token=page_token, page_size=page_size, + parent=parent, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -970,6 +1018,8 @@ def list_namespaces_with_http_info( self, page_token: Optional[StrictStr] = None, page_size: Annotated[Optional[Annotated[int, Field(strict=True, ge=1)]], Field(description="An inclusive upper bound of the number of results that a client will receive.")] = None, + parent: Annotated[Optional[StrictStr], Field(description="A string identifier of the parent namespace.")] = None, + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -983,13 +1033,18 @@ def list_namespaces_with_http_info( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> ApiResponse[ListNamespacesResponse]: - """List all namespaces in the catalog. + """List namespaces + List all child namespace names of the root namespace or a given parent namespace. :param page_token: :type page_token: str :param page_size: An inclusive upper bound of the number of results that a client will receive. :type page_size: int + :param parent: A string identifier of the parent namespace. + :type parent: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -1015,6 +1070,8 @@ def list_namespaces_with_http_info( _param = self._list_namespaces_serialize( page_token=page_token, page_size=page_size, + parent=parent, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -1046,6 +1103,8 @@ def list_namespaces_without_preload_content( self, page_token: Optional[StrictStr] = None, page_size: Annotated[Optional[Annotated[int, Field(strict=True, ge=1)]], Field(description="An inclusive upper bound of the number of results that a client will receive.")] = None, + parent: Annotated[Optional[StrictStr], Field(description="A string identifier of the parent namespace.")] = None, + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -1059,13 +1118,18 @@ def list_namespaces_without_preload_content( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> RESTResponseType: - """List all namespaces in the catalog. + """List namespaces + List all child namespace names of the root namespace or a given parent namespace. :param page_token: :type page_token: str :param page_size: An inclusive upper bound of the number of results that a client will receive. :type page_size: int + :param parent: A string identifier of the parent namespace. + :type parent: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -1091,6 +1155,8 @@ def list_namespaces_without_preload_content( _param = self._list_namespaces_serialize( page_token=page_token, page_size=page_size, + parent=parent, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -1117,6 +1183,8 @@ def _list_namespaces_serialize( self, page_token, page_size, + parent, + delimiter, _request_auth, _content_type, _headers, @@ -1147,6 +1215,14 @@ def _list_namespaces_serialize( _query_params.append(('pageSize', page_size)) + if parent is not None: + + _query_params.append(('parent', parent)) + + if delimiter is not None: + + _query_params.append(('delimiter', delimiter)) + # process the header parameters # process the form parameters # process the body parameter @@ -1186,7 +1262,8 @@ def _list_namespaces_serialize( @validate_call def namespace_exists( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -1202,10 +1279,12 @@ def namespace_exists( ) -> None: """Check if a namespace exists - Check if a namespace exists. The response does not contain a body. + Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -1229,7 +1308,8 @@ def namespace_exists( """ # noqa: E501 _param = self._namespace_exists_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -1259,7 +1339,8 @@ def namespace_exists( @validate_call def namespace_exists_with_http_info( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -1275,10 +1356,12 @@ def namespace_exists_with_http_info( ) -> ApiResponse[None]: """Check if a namespace exists - Check if a namespace exists. The response does not contain a body. + Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -1302,7 +1385,8 @@ def namespace_exists_with_http_info( """ # noqa: E501 _param = self._namespace_exists_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -1332,7 +1416,8 @@ def namespace_exists_with_http_info( @validate_call def namespace_exists_without_preload_content( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], + namespace: Annotated[StrictStr, Field(description="A string identifier of the namespace.")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -1348,10 +1433,12 @@ def namespace_exists_without_preload_content( ) -> RESTResponseType: """Check if a namespace exists - Check if a namespace exists. The response does not contain a body. + Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. - :param ns: The name of the namespace. (required) - :type ns: str + :param namespace: A string identifier of the namespace. (required) + :type namespace: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -1375,7 +1462,8 @@ def namespace_exists_without_preload_content( """ # noqa: E501 _param = self._namespace_exists_serialize( - ns=ns, + namespace=namespace, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -1400,7 +1488,8 @@ def namespace_exists_without_preload_content( def _namespace_exists_serialize( self, - ns, + namespace, + delimiter, _request_auth, _content_type, _headers, @@ -1422,9 +1511,13 @@ def _namespace_exists_serialize( _body_params: Optional[bytes] = None # process the path parameters - if ns is not None: - _path_params['ns'] = ns + if namespace is not None: + _path_params['namespace'] = namespace # process the query parameters + if delimiter is not None: + + _query_params.append(('delimiter', delimiter)) + # process the header parameters # process the form parameters # process the body parameter @@ -1445,7 +1538,7 @@ def _namespace_exists_serialize( return self.api_client.param_serialize( method='HEAD', - resource_path='/v1/namespaces/{ns}', + resource_path='/v1/namespaces/{namespace}', path_params=_path_params, query_params=_query_params, header_params=_header_params, diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/table_api.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/table_api.py index 37c7b3c07..741dcff90 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/table_api.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api/table_api.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -17,6 +17,7 @@ from typing_extensions import Annotated from pydantic import Field, StrictStr +from typing import Optional from typing_extensions import Annotated from lance_catalog_urllib3_client.models.get_table_response import GetTableResponse from lance_catalog_urllib3_client.models.register_table_request import RegisterTableRequest @@ -42,8 +43,8 @@ def __init__(self, api_client=None) -> None: @validate_call def get_table( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], - table: Annotated[StrictStr, Field(description="A table name.")], + table: Annotated[StrictStr, Field(description="A string identifier of the table")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -57,14 +58,14 @@ def get_table( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> GetTableResponse: - """Get a table from the catalog + """Get a table from the namespace - Get a table's detailed information under a specified namespace from the catalog. + Get a table's detailed information. - :param ns: The name of the namespace. (required) - :type ns: str - :param table: A table name. (required) + :param table: A string identifier of the table (required) :type table: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -88,8 +89,8 @@ def get_table( """ # noqa: E501 _param = self._get_table_serialize( - ns=ns, table=table, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -119,8 +120,8 @@ def get_table( @validate_call def get_table_with_http_info( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], - table: Annotated[StrictStr, Field(description="A table name.")], + table: Annotated[StrictStr, Field(description="A string identifier of the table")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -134,14 +135,14 @@ def get_table_with_http_info( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> ApiResponse[GetTableResponse]: - """Get a table from the catalog + """Get a table from the namespace - Get a table's detailed information under a specified namespace from the catalog. + Get a table's detailed information. - :param ns: The name of the namespace. (required) - :type ns: str - :param table: A table name. (required) + :param table: A string identifier of the table (required) :type table: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -165,8 +166,8 @@ def get_table_with_http_info( """ # noqa: E501 _param = self._get_table_serialize( - ns=ns, table=table, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -196,8 +197,8 @@ def get_table_with_http_info( @validate_call def get_table_without_preload_content( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], - table: Annotated[StrictStr, Field(description="A table name.")], + table: Annotated[StrictStr, Field(description="A string identifier of the table")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -211,14 +212,14 @@ def get_table_without_preload_content( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> RESTResponseType: - """Get a table from the catalog + """Get a table from the namespace - Get a table's detailed information under a specified namespace from the catalog. + Get a table's detailed information. - :param ns: The name of the namespace. (required) - :type ns: str - :param table: A table name. (required) + :param table: A string identifier of the table (required) :type table: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -242,8 +243,8 @@ def get_table_without_preload_content( """ # noqa: E501 _param = self._get_table_serialize( - ns=ns, table=table, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -268,8 +269,8 @@ def get_table_without_preload_content( def _get_table_serialize( self, - ns, table, + delimiter, _request_auth, _content_type, _headers, @@ -291,11 +292,13 @@ def _get_table_serialize( _body_params: Optional[bytes] = None # process the path parameters - if ns is not None: - _path_params['ns'] = ns if table is not None: _path_params['table'] = table # process the query parameters + if delimiter is not None: + + _query_params.append(('delimiter', delimiter)) + # process the header parameters # process the form parameters # process the body parameter @@ -316,7 +319,7 @@ def _get_table_serialize( return self.api_client.param_serialize( method='GET', - resource_path='/v1/namespaces/{ns}/tables/{table}', + resource_path='/v1/tables/{table}', path_params=_path_params, query_params=_query_params, header_params=_header_params, @@ -335,7 +338,6 @@ def _get_table_serialize( @validate_call def register_table( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], register_table_request: RegisterTableRequest, _request_timeout: Union[ None, @@ -350,11 +352,10 @@ def register_table( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> GetTableResponse: - """Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. + """Register a table to a namespace + Register an existing table at a given storage location to a namespace. - :param ns: The name of the namespace. (required) - :type ns: str :param register_table_request: (required) :type register_table_request: RegisterTableRequest :param _request_timeout: timeout setting for this request. If one @@ -380,7 +381,6 @@ def register_table( """ # noqa: E501 _param = self._register_table_serialize( - ns=ns, register_table_request=register_table_request, _request_auth=_request_auth, _content_type=_content_type, @@ -412,7 +412,6 @@ def register_table( @validate_call def register_table_with_http_info( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], register_table_request: RegisterTableRequest, _request_timeout: Union[ None, @@ -427,11 +426,10 @@ def register_table_with_http_info( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> ApiResponse[GetTableResponse]: - """Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. + """Register a table to a namespace + Register an existing table at a given storage location to a namespace. - :param ns: The name of the namespace. (required) - :type ns: str :param register_table_request: (required) :type register_table_request: RegisterTableRequest :param _request_timeout: timeout setting for this request. If one @@ -457,7 +455,6 @@ def register_table_with_http_info( """ # noqa: E501 _param = self._register_table_serialize( - ns=ns, register_table_request=register_table_request, _request_auth=_request_auth, _content_type=_content_type, @@ -489,7 +486,6 @@ def register_table_with_http_info( @validate_call def register_table_without_preload_content( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], register_table_request: RegisterTableRequest, _request_timeout: Union[ None, @@ -504,11 +500,10 @@ def register_table_without_preload_content( _headers: Optional[Dict[StrictStr, Any]] = None, _host_index: Annotated[StrictInt, Field(ge=0, le=0)] = 0, ) -> RESTResponseType: - """Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. + """Register a table to a namespace + Register an existing table at a given storage location to a namespace. - :param ns: The name of the namespace. (required) - :type ns: str :param register_table_request: (required) :type register_table_request: RegisterTableRequest :param _request_timeout: timeout setting for this request. If one @@ -534,7 +529,6 @@ def register_table_without_preload_content( """ # noqa: E501 _param = self._register_table_serialize( - ns=ns, register_table_request=register_table_request, _request_auth=_request_auth, _content_type=_content_type, @@ -561,7 +555,6 @@ def register_table_without_preload_content( def _register_table_serialize( self, - ns, register_table_request, _request_auth, _content_type, @@ -584,8 +577,6 @@ def _register_table_serialize( _body_params: Optional[bytes] = None # process the path parameters - if ns is not None: - _path_params['ns'] = ns # process the query parameters # process the header parameters # process the form parameters @@ -622,7 +613,7 @@ def _register_table_serialize( return self.api_client.param_serialize( method='POST', - resource_path='/v1/namespaces/{ns}/register', + resource_path='/v1/table/register', path_params=_path_params, query_params=_query_params, header_params=_header_params, @@ -641,8 +632,8 @@ def _register_table_serialize( @validate_call def table_exists( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], - table: Annotated[StrictStr, Field(description="A table name.")], + table: Annotated[StrictStr, Field(description="A string identifier of the table")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -658,12 +649,12 @@ def table_exists( ) -> None: """Check if a table exists - Check if a table exists within a given namespace. + Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. - :param ns: The name of the namespace. (required) - :type ns: str - :param table: A table name. (required) + :param table: A string identifier of the table (required) :type table: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -687,8 +678,8 @@ def table_exists( """ # noqa: E501 _param = self._table_exists_serialize( - ns=ns, table=table, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -718,8 +709,8 @@ def table_exists( @validate_call def table_exists_with_http_info( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], - table: Annotated[StrictStr, Field(description="A table name.")], + table: Annotated[StrictStr, Field(description="A string identifier of the table")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -735,12 +726,12 @@ def table_exists_with_http_info( ) -> ApiResponse[None]: """Check if a table exists - Check if a table exists within a given namespace. + Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. - :param ns: The name of the namespace. (required) - :type ns: str - :param table: A table name. (required) + :param table: A string identifier of the table (required) :type table: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -764,8 +755,8 @@ def table_exists_with_http_info( """ # noqa: E501 _param = self._table_exists_serialize( - ns=ns, table=table, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -795,8 +786,8 @@ def table_exists_with_http_info( @validate_call def table_exists_without_preload_content( self, - ns: Annotated[StrictStr, Field(description="The name of the namespace.")], - table: Annotated[StrictStr, Field(description="A table name.")], + table: Annotated[StrictStr, Field(description="A string identifier of the table")], + delimiter: Annotated[Optional[StrictStr], Field(description="The delimiter for the identifier used in the context")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -812,12 +803,12 @@ def table_exists_without_preload_content( ) -> RESTResponseType: """Check if a table exists - Check if a table exists within a given namespace. + Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. - :param ns: The name of the namespace. (required) - :type ns: str - :param table: A table name. (required) + :param table: A string identifier of the table (required) :type table: str + :param delimiter: The delimiter for the identifier used in the context + :type delimiter: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -841,8 +832,8 @@ def table_exists_without_preload_content( """ # noqa: E501 _param = self._table_exists_serialize( - ns=ns, table=table, + delimiter=delimiter, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -867,8 +858,8 @@ def table_exists_without_preload_content( def _table_exists_serialize( self, - ns, table, + delimiter, _request_auth, _content_type, _headers, @@ -890,11 +881,13 @@ def _table_exists_serialize( _body_params: Optional[bytes] = None # process the path parameters - if ns is not None: - _path_params['ns'] = ns if table is not None: _path_params['table'] = table # process the query parameters + if delimiter is not None: + + _query_params.append(('delimiter', delimiter)) + # process the header parameters # process the form parameters # process the body parameter @@ -915,7 +908,7 @@ def _table_exists_serialize( return self.api_client.param_serialize( method='HEAD', - resource_path='/v1/namespaces/{ns}/tables/{table}', + resource_path='/v1/tables/{table}', path_params=_path_params, query_params=_query_params, header_params=_header_params, diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api_client.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api_client.py index 10243cbe4..a450ad28b 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api_client.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/api_client.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/configuration.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/configuration.py index c6e6b6c66..8c0cf9f4e 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/configuration.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/configuration.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/exceptions.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/exceptions.py index db5d92646..7b5c83382 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/exceptions.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/exceptions.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/__init__.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/__init__.py index db5a6336b..611438c1f 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/__init__.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/__init__.py @@ -2,9 +2,9 @@ # flake8: noqa """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -15,7 +15,6 @@ # import models into model package from lance_catalog_urllib3_client.models.create_namespace_request import CreateNamespaceRequest -from lance_catalog_urllib3_client.models.create_namespace_response import CreateNamespaceResponse from lance_catalog_urllib3_client.models.error_response import ErrorResponse from lance_catalog_urllib3_client.models.get_namespace_response import GetNamespaceResponse from lance_catalog_urllib3_client.models.get_table_response import GetTableResponse diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_catalog_request.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_catalog_request.py new file mode 100644 index 000000000..51b0bbe6d --- /dev/null +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_catalog_request.py @@ -0,0 +1,98 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog Specification** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance catalogs recursively. It is designed to encapsulates concepts like namespace, metastore, database, schema, etc. that could appear in other similar systems, so that it can better integrate with any system with any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those catalog services or building a custom catalog server in a standardized way. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +from __future__ import annotations +import pprint +import re # noqa: F401 +import json + +from pydantic import BaseModel, ConfigDict, StrictStr, field_validator +from typing import Any, ClassVar, Dict, List, Optional +from typing import Optional, Set +from typing_extensions import Self + +class CreateCatalogRequest(BaseModel): + """ + CreateCatalogRequest + """ # noqa: E501 + name: StrictStr + mode: StrictStr + options: Optional[Dict[str, StrictStr]] = None + __properties: ClassVar[List[str]] = ["name", "mode", "options"] + + @field_validator('mode') + def mode_validate_enum(cls, value): + """Validates the enum""" + if value not in set(['CREATE', 'EXIST_OK', 'OVERWRITE']): + raise ValueError("must be one of enum values ('CREATE', 'EXIST_OK', 'OVERWRITE')") + return value + + model_config = ConfigDict( + populate_by_name=True, + validate_assignment=True, + protected_namespaces=(), + ) + + + def to_str(self) -> str: + """Returns the string representation of the model using alias""" + return pprint.pformat(self.model_dump(by_alias=True)) + + def to_json(self) -> str: + """Returns the JSON representation of the model using alias""" + # TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead + return json.dumps(self.to_dict()) + + @classmethod + def from_json(cls, json_str: str) -> Optional[Self]: + """Create an instance of CreateCatalogRequest from a JSON string""" + return cls.from_dict(json.loads(json_str)) + + def to_dict(self) -> Dict[str, Any]: + """Return the dictionary representation of the model using alias. + + This has the following differences from calling pydantic's + `self.model_dump(by_alias=True)`: + + * `None` is only added to the output dict for nullable fields that + were set at model initialization. Other fields with value `None` + are ignored. + """ + excluded_fields: Set[str] = set([ + ]) + + _dict = self.model_dump( + by_alias=True, + exclude=excluded_fields, + exclude_none=True, + ) + return _dict + + @classmethod + def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: + """Create an instance of CreateCatalogRequest from a dict""" + if obj is None: + return None + + if not isinstance(obj, dict): + return cls.model_validate(obj) + + _obj = cls.model_validate({ + "name": obj.get("name"), + "mode": obj.get("mode"), + "options": obj.get("options") + }) + return _obj + + diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_catalog_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_catalog_response.py new file mode 100644 index 000000000..f310e6236 --- /dev/null +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_catalog_response.py @@ -0,0 +1,89 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog Specification** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance catalogs recursively. It is designed to encapsulates concepts like namespace, metastore, database, schema, etc. that could appear in other similar systems, so that it can better integrate with any system with any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those catalog services or building a custom catalog server in a standardized way. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +from __future__ import annotations +import pprint +import re # noqa: F401 +import json + +from pydantic import BaseModel, ConfigDict, Field, StrictStr +from typing import Any, ClassVar, Dict, List, Optional +from typing import Optional, Set +from typing_extensions import Self + +class CreateCatalogResponse(BaseModel): + """ + CreateCatalogResponse + """ # noqa: E501 + name: StrictStr + properties: Optional[Dict[str, StrictStr]] = Field(default=None, description="Properties stored on the catalog, if supported by the server.") + __properties: ClassVar[List[str]] = ["name", "properties"] + + model_config = ConfigDict( + populate_by_name=True, + validate_assignment=True, + protected_namespaces=(), + ) + + + def to_str(self) -> str: + """Returns the string representation of the model using alias""" + return pprint.pformat(self.model_dump(by_alias=True)) + + def to_json(self) -> str: + """Returns the JSON representation of the model using alias""" + # TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead + return json.dumps(self.to_dict()) + + @classmethod + def from_json(cls, json_str: str) -> Optional[Self]: + """Create an instance of CreateCatalogResponse from a JSON string""" + return cls.from_dict(json.loads(json_str)) + + def to_dict(self) -> Dict[str, Any]: + """Return the dictionary representation of the model using alias. + + This has the following differences from calling pydantic's + `self.model_dump(by_alias=True)`: + + * `None` is only added to the output dict for nullable fields that + were set at model initialization. Other fields with value `None` + are ignored. + """ + excluded_fields: Set[str] = set([ + ]) + + _dict = self.model_dump( + by_alias=True, + exclude=excluded_fields, + exclude_none=True, + ) + return _dict + + @classmethod + def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: + """Create an instance of CreateCatalogResponse from a dict""" + if obj is None: + return None + + if not isinstance(obj, dict): + return cls.model_validate(obj) + + _obj = cls.model_validate({ + "name": obj.get("name"), + "properties": obj.get("properties") + }) + return _obj + + diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_namespace_request.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_namespace_request.py index 13a92b044..aa11dd886 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_namespace_request.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/create_namespace_request.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -27,9 +27,10 @@ class CreateNamespaceRequest(BaseModel): CreateNamespaceRequest """ # noqa: E501 name: StrictStr + parent: Optional[List[StrictStr]] = None mode: StrictStr options: Optional[Dict[str, StrictStr]] = None - __properties: ClassVar[List[str]] = ["name", "mode", "options"] + __properties: ClassVar[List[str]] = ["name", "parent", "mode", "options"] @field_validator('mode') def mode_validate_enum(cls, value): @@ -90,6 +91,7 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: _obj = cls.model_validate({ "name": obj.get("name"), + "parent": obj.get("parent"), "mode": obj.get("mode"), "options": obj.get("options") }) diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/error_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/error_response.py index 2c94b3764..165124494 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/error_response.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/error_response.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_catalog_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_catalog_response.py new file mode 100644 index 000000000..e065b31a9 --- /dev/null +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_catalog_response.py @@ -0,0 +1,89 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog Specification** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance catalogs recursively. It is designed to encapsulates concepts like namespace, metastore, database, schema, etc. that could appear in other similar systems, so that it can better integrate with any system with any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those catalog services or building a custom catalog server in a standardized way. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +from __future__ import annotations +import pprint +import re # noqa: F401 +import json + +from pydantic import BaseModel, ConfigDict, Field, StrictStr +from typing import Any, ClassVar, Dict, List, Optional +from typing import Optional, Set +from typing_extensions import Self + +class GetCatalogResponse(BaseModel): + """ + GetCatalogResponse + """ # noqa: E501 + catalog: StrictStr + properties: Optional[Dict[str, StrictStr]] = Field(default=None, description="Properties stored on the catalog, if supported by the server. If the server does not support catalog properties, it should return null for this field. If catalog properties are supported, but none are set, it should return an empty object.") + __properties: ClassVar[List[str]] = ["catalog", "properties"] + + model_config = ConfigDict( + populate_by_name=True, + validate_assignment=True, + protected_namespaces=(), + ) + + + def to_str(self) -> str: + """Returns the string representation of the model using alias""" + return pprint.pformat(self.model_dump(by_alias=True)) + + def to_json(self) -> str: + """Returns the JSON representation of the model using alias""" + # TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead + return json.dumps(self.to_dict()) + + @classmethod + def from_json(cls, json_str: str) -> Optional[Self]: + """Create an instance of GetCatalogResponse from a JSON string""" + return cls.from_dict(json.loads(json_str)) + + def to_dict(self) -> Dict[str, Any]: + """Return the dictionary representation of the model using alias. + + This has the following differences from calling pydantic's + `self.model_dump(by_alias=True)`: + + * `None` is only added to the output dict for nullable fields that + were set at model initialization. Other fields with value `None` + are ignored. + """ + excluded_fields: Set[str] = set([ + ]) + + _dict = self.model_dump( + by_alias=True, + exclude=excluded_fields, + exclude_none=True, + ) + return _dict + + @classmethod + def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: + """Create an instance of GetCatalogResponse from a dict""" + if obj is None: + return None + + if not isinstance(obj, dict): + return cls.model_validate(obj) + + _obj = cls.model_validate({ + "catalog": obj.get("catalog"), + "properties": obj.get("properties") + }) + return _obj + + diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_namespace_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_namespace_response.py index e38c1b16c..e093ef77a 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_namespace_response.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_namespace_response.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -26,9 +26,10 @@ class GetNamespaceResponse(BaseModel): """ GetNamespaceResponse """ # noqa: E501 - namespace: StrictStr + name: StrictStr + parent: Optional[List[StrictStr]] = None properties: Optional[Dict[str, StrictStr]] = Field(default=None, description="Properties stored on the namespace, if supported by the server. If the server does not support namespace properties, it should return null for this field. If namespace properties are supported, but none are set, it should return an empty object.") - __properties: ClassVar[List[str]] = ["namespace", "properties"] + __properties: ClassVar[List[str]] = ["name", "parent", "properties"] model_config = ConfigDict( populate_by_name=True, @@ -81,7 +82,8 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: return cls.model_validate(obj) _obj = cls.model_validate({ - "namespace": obj.get("namespace"), + "name": obj.get("name"), + "parent": obj.get("parent"), "properties": obj.get("properties") }) return _obj diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_table_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_table_response.py index 0c8a35cfa..fad45e781 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_table_response.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/get_table_response.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -24,12 +24,13 @@ class GetTableResponse(BaseModel): """ - Result used when a table is successfully loaded. + GetTableResponse """ # noqa: E501 name: StrictStr + namespace: List[StrictStr] location: StrictStr properties: Optional[Dict[str, StrictStr]] = None - __properties: ClassVar[List[str]] = ["name", "location", "properties"] + __properties: ClassVar[List[str]] = ["name", "namespace", "location", "properties"] model_config = ConfigDict( populate_by_name=True, @@ -83,6 +84,7 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: _obj = cls.model_validate({ "name": obj.get("name"), + "namespace": obj.get("namespace"), "location": obj.get("location"), "properties": obj.get("properties") }) diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_catalogs_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_catalogs_response.py new file mode 100644 index 000000000..506d2dc58 --- /dev/null +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_catalogs_response.py @@ -0,0 +1,89 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog Specification** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance catalogs recursively. It is designed to encapsulates concepts like namespace, metastore, database, schema, etc. that could appear in other similar systems, so that it can better integrate with any system with any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those catalog services or building a custom catalog server in a standardized way. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +from __future__ import annotations +import pprint +import re # noqa: F401 +import json + +from pydantic import BaseModel, ConfigDict, Field, StrictStr +from typing import Any, ClassVar, Dict, List, Optional +from typing import Optional, Set +from typing_extensions import Self + +class ListCatalogsResponse(BaseModel): + """ + ListCatalogsResponse + """ # noqa: E501 + catalogs: List[StrictStr] = Field(description="An array of child catalog names in the catalog.") + next_page_token: Optional[StrictStr] = Field(default=None, description="An opaque token that allows pagination for list APIs (e.g. ListCatalogs). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results.", alias="nextPageToken") + __properties: ClassVar[List[str]] = ["catalogs", "nextPageToken"] + + model_config = ConfigDict( + populate_by_name=True, + validate_assignment=True, + protected_namespaces=(), + ) + + + def to_str(self) -> str: + """Returns the string representation of the model using alias""" + return pprint.pformat(self.model_dump(by_alias=True)) + + def to_json(self) -> str: + """Returns the JSON representation of the model using alias""" + # TODO: pydantic v2: use .model_dump_json(by_alias=True, exclude_unset=True) instead + return json.dumps(self.to_dict()) + + @classmethod + def from_json(cls, json_str: str) -> Optional[Self]: + """Create an instance of ListCatalogsResponse from a JSON string""" + return cls.from_dict(json.loads(json_str)) + + def to_dict(self) -> Dict[str, Any]: + """Return the dictionary representation of the model using alias. + + This has the following differences from calling pydantic's + `self.model_dump(by_alias=True)`: + + * `None` is only added to the output dict for nullable fields that + were set at model initialization. Other fields with value `None` + are ignored. + """ + excluded_fields: Set[str] = set([ + ]) + + _dict = self.model_dump( + by_alias=True, + exclude=excluded_fields, + exclude_none=True, + ) + return _dict + + @classmethod + def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: + """Create an instance of ListCatalogsResponse from a dict""" + if obj is None: + return None + + if not isinstance(obj, dict): + return cls.model_validate(obj) + + _obj = cls.model_validate({ + "catalogs": obj.get("catalogs"), + "nextPageToken": obj.get("nextPageToken") + }) + return _obj + + diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_namespaces_response.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_namespaces_response.py index 2424e15be..40b721b81 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_namespaces_response.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/list_namespaces_response.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -26,7 +26,7 @@ class ListNamespacesResponse(BaseModel): """ ListNamespacesResponse """ # noqa: E501 - namespaces: List[StrictStr] = Field(description="An array of namespace names in the catalog.") + namespaces: List[StrictStr] next_page_token: Optional[StrictStr] = Field(default=None, description="An opaque token that allows pagination for list APIs (e.g. ListNamespaces). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results.", alias="nextPageToken") __properties: ClassVar[List[str]] = ["namespaces", "nextPageToken"] diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/register_table_request.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/register_table_request.py index 0e337eeec..bf0e60510 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/register_table_request.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/models/register_table_request.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) @@ -27,8 +27,9 @@ class RegisterTableRequest(BaseModel): RegisterTableRequest """ # noqa: E501 name: StrictStr + namespace: List[StrictStr] location: StrictStr - __properties: ClassVar[List[str]] = ["name", "location"] + __properties: ClassVar[List[str]] = ["name", "namespace", "location"] model_config = ConfigDict( populate_by_name=True, @@ -82,6 +83,7 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: _obj = cls.model_validate({ "name": obj.get("name"), + "namespace": obj.get("namespace"), "location": obj.get("location") }) return _obj diff --git a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/rest.py b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/rest.py index 0ce823bda..357e57e50 100644 --- a/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/rest.py +++ b/python/lance_catalog_urllib3_client/lance_catalog_urllib3_client/rest.py @@ -1,9 +1,9 @@ # coding: utf-8 """ - Lance Catalog REST Specification + Lance REST Namespace Specification - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). The version of the OpenAPI document: 0.0.1 Generated by OpenAPI Generator (https://openapi-generator.tech) diff --git a/python/lance_catalog_urllib3_client/pyproject.toml b/python/lance_catalog_urllib3_client/pyproject.toml index be73010e6..79d2074d8 100644 --- a/python/lance_catalog_urllib3_client/pyproject.toml +++ b/python/lance_catalog_urllib3_client/pyproject.toml @@ -1,12 +1,12 @@ [tool.poetry] name = "lance_catalog_urllib3_client" version = "0.0.1" -description = "Lance Catalog REST Specification" +description = "Lance REST Namespace Specification" authors = ["OpenAPI Generator Community "] license = "Apache 2.0" readme = "README.md" repository = "https://github.com/GIT_USER_ID/GIT_REPO_ID" -keywords = ["OpenAPI", "OpenAPI-Generator", "Lance Catalog REST Specification"] +keywords = ["OpenAPI", "OpenAPI-Generator", "Lance REST Namespace Specification"] include = ["lance_catalog_urllib3_client/py.typed"] [tool.poetry.dependencies] diff --git a/python/lance_catalog_urllib3_client/test/test_catalog_api.py b/python/lance_catalog_urllib3_client/test/test_catalog_api.py new file mode 100644 index 000000000..3372bc0f0 --- /dev/null +++ b/python/lance_catalog_urllib3_client/test/test_catalog_api.py @@ -0,0 +1,66 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables. It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It is a generalized concept that is also called namespace, metastore, database, schema in other similar systems. A Lance catalog can either contain a list of tables, or contain a list of Lance catalogs recursively. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is a standardized OpenAPI protocol to read, write and manage Lance tables. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +import unittest + +from lance_catalog_urllib3_client.api.catalog_api import CatalogApi + + +class TestCatalogApi(unittest.TestCase): + """CatalogApi unit test stubs""" + + def setUp(self) -> None: + self.api = CatalogApi() + + def tearDown(self) -> None: + pass + + def test_catalog_exists(self) -> None: + """Test case for catalog_exists + + Check if a catalog exists + """ + pass + + def test_create_catalog(self) -> None: + """Test case for create_catalog + + Create a new catalog. A catalog can manage either a collection of child catalogs, or a collection of tables. There are three modes when trying to create a catalog to differentiate the behavior when a catalog of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing catalog is kept. * OVERWRITE: the existing catalog is dropped and a new empty catalog with this name is created. + """ + pass + + def test_drop_catalog(self) -> None: + """Test case for drop_catalog + + Drop a catalog. The catalog must be empty. + """ + pass + + def test_get_catalog(self) -> None: + """Test case for get_catalog + + Get information about a catalog + """ + pass + + def test_list_catalogs(self) -> None: + """Test case for list_catalogs + + List all direct child catalogs of the root catalog. + """ + pass + + +if __name__ == '__main__': + unittest.main() diff --git a/python/lance_catalog_urllib3_client/test/test_create_catalog_request.py b/python/lance_catalog_urllib3_client/test/test_create_catalog_request.py new file mode 100644 index 000000000..7e4f180a2 --- /dev/null +++ b/python/lance_catalog_urllib3_client/test/test_create_catalog_request.py @@ -0,0 +1,57 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables. It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It is a generalized concept that is also called namespace, metastore, database, schema in other similar systems. A Lance catalog can either contain a list of tables, or contain a list of Lance catalogs recursively. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is a standardized OpenAPI protocol to read, write and manage Lance tables. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +import unittest + +from lance_catalog_urllib3_client.models.create_catalog_request import CreateCatalogRequest + +class TestCreateCatalogRequest(unittest.TestCase): + """CreateCatalogRequest unit test stubs""" + + def setUp(self): + pass + + def tearDown(self): + pass + + def make_instance(self, include_optional) -> CreateCatalogRequest: + """Test CreateCatalogRequest + include_optional is a boolean, when False only required + params are included, when True both required and + optional params are included """ + # uncomment below to create an instance of `CreateCatalogRequest` + """ + model = CreateCatalogRequest() + if include_optional: + return CreateCatalogRequest( + name = '', + mode = 'CREATE', + options = { + 'key' : '' + } + ) + else: + return CreateCatalogRequest( + name = '', + mode = 'CREATE', + ) + """ + + def testCreateCatalogRequest(self): + """Test CreateCatalogRequest""" + # inst_req_only = self.make_instance(include_optional=False) + # inst_req_and_optional = self.make_instance(include_optional=True) + +if __name__ == '__main__': + unittest.main() diff --git a/python/lance_catalog_urllib3_client/test/test_create_catalog_response.py b/python/lance_catalog_urllib3_client/test/test_create_catalog_response.py new file mode 100644 index 000000000..5d98ddae9 --- /dev/null +++ b/python/lance_catalog_urllib3_client/test/test_create_catalog_response.py @@ -0,0 +1,53 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables. It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It is a generalized concept that is also called namespace, metastore, database, schema in other similar systems. A Lance catalog can either contain a list of tables, or contain a list of Lance catalogs recursively. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is a standardized OpenAPI protocol to read, write and manage Lance tables. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +import unittest + +from lance_catalog_urllib3_client.models.create_catalog_response import CreateCatalogResponse + +class TestCreateCatalogResponse(unittest.TestCase): + """CreateCatalogResponse unit test stubs""" + + def setUp(self): + pass + + def tearDown(self): + pass + + def make_instance(self, include_optional) -> CreateCatalogResponse: + """Test CreateCatalogResponse + include_optional is a boolean, when False only required + params are included, when True both required and + optional params are included """ + # uncomment below to create an instance of `CreateCatalogResponse` + """ + model = CreateCatalogResponse() + if include_optional: + return CreateCatalogResponse( + name = '', + properties = {created_at=1452120468} + ) + else: + return CreateCatalogResponse( + name = '', + ) + """ + + def testCreateCatalogResponse(self): + """Test CreateCatalogResponse""" + # inst_req_only = self.make_instance(include_optional=False) + # inst_req_and_optional = self.make_instance(include_optional=True) + +if __name__ == '__main__': + unittest.main() diff --git a/python/lance_catalog_urllib3_client/test/test_get_catalog_response.py b/python/lance_catalog_urllib3_client/test/test_get_catalog_response.py new file mode 100644 index 000000000..2e05704a0 --- /dev/null +++ b/python/lance_catalog_urllib3_client/test/test_get_catalog_response.py @@ -0,0 +1,53 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables. It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It is a generalized concept that is also called namespace, metastore, database, schema in other similar systems. A Lance catalog can either contain a list of tables, or contain a list of Lance catalogs recursively. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is a standardized OpenAPI protocol to read, write and manage Lance tables. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +import unittest + +from lance_catalog_urllib3_client.models.get_catalog_response import GetCatalogResponse + +class TestGetCatalogResponse(unittest.TestCase): + """GetCatalogResponse unit test stubs""" + + def setUp(self): + pass + + def tearDown(self): + pass + + def make_instance(self, include_optional) -> GetCatalogResponse: + """Test GetCatalogResponse + include_optional is a boolean, when False only required + params are included, when True both required and + optional params are included """ + # uncomment below to create an instance of `GetCatalogResponse` + """ + model = GetCatalogResponse() + if include_optional: + return GetCatalogResponse( + catalog = '', + properties = {owner=Ralph, created_at=1452120468} + ) + else: + return GetCatalogResponse( + catalog = '', + ) + """ + + def testGetCatalogResponse(self): + """Test GetCatalogResponse""" + # inst_req_only = self.make_instance(include_optional=False) + # inst_req_and_optional = self.make_instance(include_optional=True) + +if __name__ == '__main__': + unittest.main() diff --git a/python/lance_catalog_urllib3_client/test/test_list_catalogs_response.py b/python/lance_catalog_urllib3_client/test/test_list_catalogs_response.py new file mode 100644 index 000000000..c185f3ddb --- /dev/null +++ b/python/lance_catalog_urllib3_client/test/test_list_catalogs_response.py @@ -0,0 +1,57 @@ +# coding: utf-8 + +""" + Lance REST Catalog Specification + + **Lance Catalog** is an open specification on top of the storage-based Lance open table and data format to standardize access to a collection of Lance tables. It describes how a catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Catalog, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance. A Lance catalog is a centralized repository for discovering, organizing, and managing Lance tables. It is a generalized concept that is also called namespace, metastore, database, schema in other similar systems. A Lance catalog can either contain a list of tables, or contain a list of Lance catalogs recursively. In an enterprise environment, typically there is a requirement to store tables in a catalog service such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST catalog** is a standardized OpenAPI protocol to read, write and manage Lance tables. + + The version of the OpenAPI document: 0.0.1 + Generated by OpenAPI Generator (https://openapi-generator.tech) + + Do not edit the class manually. +""" # noqa: E501 + + +import unittest + +from lance_catalog_urllib3_client.models.list_catalogs_response import ListCatalogsResponse + +class TestListCatalogsResponse(unittest.TestCase): + """ListCatalogsResponse unit test stubs""" + + def setUp(self): + pass + + def tearDown(self): + pass + + def make_instance(self, include_optional) -> ListCatalogsResponse: + """Test ListCatalogsResponse + include_optional is a boolean, when False only required + params are included, when True both required and + optional params are included """ + # uncomment below to create an instance of `ListCatalogsResponse` + """ + model = ListCatalogsResponse() + if include_optional: + return ListCatalogsResponse( + catalogs = [ + 'accounting' + ], + next_page_token = '' + ) + else: + return ListCatalogsResponse( + catalogs = [ + 'accounting' + ], + ) + """ + + def testListCatalogsResponse(self): + """Test ListCatalogsResponse""" + # inst_req_only = self.make_instance(include_optional=False) + # inst_req_and_optional = self.make_instance(include_optional=True) + +if __name__ == '__main__': + unittest.main() diff --git a/rust/Makefile b/rust/Makefile index 9b7f946ba..c56cba476 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -17,7 +17,7 @@ clean-rust-reqwest-client: gen-rust-reqwest-client: clean-rust-reqwest-client openapi-generator-cli generate \ - -i ../spec/catalog.yaml \ + -i ../spec/rest.yaml \ -g rust \ -o lance-catalog-reqwest-client \ --additional-properties=packageName=lance-catalog-reqwest-client,packageVersion=$(VERSION),asyncApi=true,library=reqwest diff --git a/rust/lance-catalog-reqwest-client/.openapi-generator/FILES b/rust/lance-catalog-reqwest-client/.openapi-generator/FILES index 6ddedffdd..54e39762e 100644 --- a/rust/lance-catalog-reqwest-client/.openapi-generator/FILES +++ b/rust/lance-catalog-reqwest-client/.openapi-generator/FILES @@ -3,7 +3,6 @@ Cargo.toml README.md docs/CreateNamespaceRequest.md -docs/CreateNamespaceResponse.md docs/ErrorResponse.md docs/GetNamespaceResponse.md docs/GetTableResponse.md @@ -18,7 +17,6 @@ src/apis/namespace_api.rs src/apis/table_api.rs src/lib.rs src/models/create_namespace_request.rs -src/models/create_namespace_response.rs src/models/error_response.rs src/models/get_namespace_response.rs src/models/get_table_response.rs diff --git a/rust/lance-catalog-reqwest-client/Cargo.toml b/rust/lance-catalog-reqwest-client/Cargo.toml index dfa07889f..411c09ac9 100644 --- a/rust/lance-catalog-reqwest-client/Cargo.toml +++ b/rust/lance-catalog-reqwest-client/Cargo.toml @@ -2,7 +2,7 @@ name = "lance-catalog-reqwest-client" version = "0.0.1" authors = ["OpenAPI Generator team and contributors"] -description = "**Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. " +description = "**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). " license = "Apache 2.0" edition = "2021" diff --git a/rust/lance-catalog-reqwest-client/README.md b/rust/lance-catalog-reqwest-client/README.md index 905674065..edaf07789 100644 --- a/rust/lance-catalog-reqwest-client/README.md +++ b/rust/lance-catalog-reqwest-client/README.md @@ -1,11 +1,8 @@ # Rust API client for lance-catalog-reqwest-client -**Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. -It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. -to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, -which is a REST server that converts the Lance catalog requests to native requests against the catalog service. -Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, -and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. +**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. +A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. +In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). @@ -32,20 +29,19 @@ All URIs are relative to *http://localhost:2333* Class | Method | HTTP request | Description ------------ | ------------- | ------------- | ------------- -*NamespaceApi* | [**create_namespace**](docs/NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. -*NamespaceApi* | [**drop_namespace**](docs/NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace from the catalog. Namespace must be empty. -*NamespaceApi* | [**get_namespace**](docs/NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace -*NamespaceApi* | [**list_namespaces**](docs/NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List all namespaces in the catalog. -*NamespaceApi* | [**namespace_exists**](docs/NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists -*TableApi* | [**get_table**](docs/TableApi.md#get_table) | **GET** /v1/namespaces/{ns}/tables/{table} | Get a table from the catalog -*TableApi* | [**register_table**](docs/TableApi.md#register_table) | **POST** /v1/namespaces/{ns}/register | Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. -*TableApi* | [**table_exists**](docs/TableApi.md#table_exists) | **HEAD** /v1/namespaces/{ns}/tables/{table} | Check if a table exists +*NamespaceApi* | [**create_namespace**](docs/NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace +*NamespaceApi* | [**drop_namespace**](docs/NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{namespace} | Drop a namespace +*NamespaceApi* | [**get_namespace**](docs/NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{namespace} | Get information about a namespace +*NamespaceApi* | [**list_namespaces**](docs/NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List namespaces +*NamespaceApi* | [**namespace_exists**](docs/NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{namespace} | Check if a namespace exists +*TableApi* | [**get_table**](docs/TableApi.md#get_table) | **GET** /v1/tables/{table} | Get a table from the namespace +*TableApi* | [**register_table**](docs/TableApi.md#register_table) | **POST** /v1/table/register | Register a table to a namespace +*TableApi* | [**table_exists**](docs/TableApi.md#table_exists) | **HEAD** /v1/tables/{table} | Check if a table exists ## Documentation For Models - [CreateNamespaceRequest](docs/CreateNamespaceRequest.md) - - [CreateNamespaceResponse](docs/CreateNamespaceResponse.md) - [ErrorResponse](docs/ErrorResponse.md) - [GetNamespaceResponse](docs/GetNamespaceResponse.md) - [GetTableResponse](docs/GetTableResponse.md) diff --git a/rust/lance-catalog-reqwest-client/docs/CreateNamespaceRequest.md b/rust/lance-catalog-reqwest-client/docs/CreateNamespaceRequest.md index 251442401..5dc734aab 100644 --- a/rust/lance-catalog-reqwest-client/docs/CreateNamespaceRequest.md +++ b/rust/lance-catalog-reqwest-client/docs/CreateNamespaceRequest.md @@ -5,6 +5,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **name** | **String** | | +**parent** | Option<**Vec**> | | [optional] **mode** | **String** | | **options** | Option<**std::collections::HashMap**> | | [optional] diff --git a/rust/lance-catalog-reqwest-client/docs/CreateNamespaceResponse.md b/rust/lance-catalog-reqwest-client/docs/CreateNamespaceResponse.md deleted file mode 100644 index 135420e9d..000000000 --- a/rust/lance-catalog-reqwest-client/docs/CreateNamespaceResponse.md +++ /dev/null @@ -1,12 +0,0 @@ -# CreateNamespaceResponse - -## Properties - -Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- -**name** | **String** | | -**properties** | Option<**std::collections::HashMap**> | Properties stored on the namespace, if supported by the server. | [optional][default to {}] - -[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) - - diff --git a/rust/lance-catalog-reqwest-client/docs/GetNamespaceResponse.md b/rust/lance-catalog-reqwest-client/docs/GetNamespaceResponse.md index 460d5b82a..f93f21422 100644 --- a/rust/lance-catalog-reqwest-client/docs/GetNamespaceResponse.md +++ b/rust/lance-catalog-reqwest-client/docs/GetNamespaceResponse.md @@ -4,7 +4,8 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- -**namespace** | **String** | | +**name** | **String** | | +**parent** | Option<**Vec**> | | [optional] **properties** | Option<**std::collections::HashMap**> | Properties stored on the namespace, if supported by the server. If the server does not support namespace properties, it should return null for this field. If namespace properties are supported, but none are set, it should return an empty object. | [optional][default to {}] [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) diff --git a/rust/lance-catalog-reqwest-client/docs/GetTableResponse.md b/rust/lance-catalog-reqwest-client/docs/GetTableResponse.md index 3d9332af3..88b4bff43 100644 --- a/rust/lance-catalog-reqwest-client/docs/GetTableResponse.md +++ b/rust/lance-catalog-reqwest-client/docs/GetTableResponse.md @@ -5,6 +5,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **name** | **String** | | +**namespace** | **Vec** | | **location** | **String** | | **properties** | Option<**std::collections::HashMap**> | | [optional] diff --git a/rust/lance-catalog-reqwest-client/docs/ListNamespacesResponse.md b/rust/lance-catalog-reqwest-client/docs/ListNamespacesResponse.md index fc22e2d98..2274dfadb 100644 --- a/rust/lance-catalog-reqwest-client/docs/ListNamespacesResponse.md +++ b/rust/lance-catalog-reqwest-client/docs/ListNamespacesResponse.md @@ -4,7 +4,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- -**namespaces** | **Vec** | An array of namespace names in the catalog. | +**namespaces** | **Vec** | | **next_page_token** | Option<**String**> | An opaque token that allows pagination for list APIs (e.g. ListNamespaces). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results. | [optional] [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) diff --git a/rust/lance-catalog-reqwest-client/docs/NamespaceApi.md b/rust/lance-catalog-reqwest-client/docs/NamespaceApi.md index e1926d6e6..c494f8265 100644 --- a/rust/lance-catalog-reqwest-client/docs/NamespaceApi.md +++ b/rust/lance-catalog-reqwest-client/docs/NamespaceApi.md @@ -4,18 +4,20 @@ All URIs are relative to *http://localhost:2333* Method | HTTP request | Description ------------- | ------------- | ------------- -[**create_namespace**](NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. -[**drop_namespace**](NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{ns} | Drop a namespace from the catalog. Namespace must be empty. -[**get_namespace**](NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{ns} | Get information about a namespace -[**list_namespaces**](NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List all namespaces in the catalog. -[**namespace_exists**](NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{ns} | Check if a namespace exists +[**create_namespace**](NamespaceApi.md#create_namespace) | **POST** /v1/namespaces | Create a new namespace +[**drop_namespace**](NamespaceApi.md#drop_namespace) | **DELETE** /v1/namespaces/{namespace} | Drop a namespace +[**get_namespace**](NamespaceApi.md#get_namespace) | **GET** /v1/namespaces/{namespace} | Get information about a namespace +[**list_namespaces**](NamespaceApi.md#list_namespaces) | **GET** /v1/namespaces | List namespaces +[**namespace_exists**](NamespaceApi.md#namespace_exists) | **HEAD** /v1/namespaces/{namespace} | Check if a namespace exists ## create_namespace -> models::CreateNamespaceResponse create_namespace(create_namespace_request) -Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. There are three modes when trying to create a namespace: * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. +> models::GetNamespaceResponse create_namespace(create_namespace_request) +Create a new namespace + +Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. ### Parameters @@ -26,7 +28,7 @@ Name | Type | Description | Required | Notes ### Return type -[**models::CreateNamespaceResponse**](CreateNamespaceResponse.md) +[**models::GetNamespaceResponse**](GetNamespaceResponse.md) ### Authorization @@ -42,15 +44,18 @@ No authorization required ## drop_namespace -> drop_namespace(ns) -Drop a namespace from the catalog. Namespace must be empty. +> drop_namespace(namespace, delimiter) +Drop a namespace + +Drop a namespace. The namespace must be empty. ### Parameters Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- -**ns** | **String** | The name of the namespace. | [required] | +**namespace** | **String** | A string identifier of the namespace. | [required] | +**delimiter** | Option<**String**> | The delimiter for the identifier used in the context | | ### Return type @@ -70,17 +75,18 @@ No authorization required ## get_namespace -> models::GetNamespaceResponse get_namespace(ns) +> models::GetNamespaceResponse get_namespace(namespace, delimiter) Get information about a namespace -Return a detailed information for a given namespace +Return the detailed information for a given namespace ### Parameters Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- -**ns** | **String** | The name of the namespace. | [required] | +**namespace** | **String** | A string identifier of the namespace. | [required] | +**delimiter** | Option<**String**> | The delimiter for the identifier used in the context | | ### Return type @@ -100,8 +106,10 @@ No authorization required ## list_namespaces -> models::ListNamespacesResponse list_namespaces(page_token, page_size) -List all namespaces in the catalog. +> models::ListNamespacesResponse list_namespaces(page_token, page_size, parent, delimiter) +List namespaces + +List all child namespace names of the root namespace or a given parent namespace. ### Parameters @@ -110,6 +118,8 @@ Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- **page_token** | Option<**String**> | | | **page_size** | Option<**i32**> | An inclusive upper bound of the number of results that a client will receive. | | +**parent** | Option<**String**> | A string identifier of the parent namespace. | | +**delimiter** | Option<**String**> | The delimiter for the identifier used in the context | | ### Return type @@ -129,17 +139,18 @@ No authorization required ## namespace_exists -> namespace_exists(ns) +> namespace_exists(namespace, delimiter) Check if a namespace exists -Check if a namespace exists. The response does not contain a body. +Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. ### Parameters Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- -**ns** | **String** | The name of the namespace. | [required] | +**namespace** | **String** | A string identifier of the namespace. | [required] | +**delimiter** | Option<**String**> | The delimiter for the identifier used in the context | | ### Return type diff --git a/rust/lance-catalog-reqwest-client/docs/RegisterTableRequest.md b/rust/lance-catalog-reqwest-client/docs/RegisterTableRequest.md index 17c662e03..a9dc32785 100644 --- a/rust/lance-catalog-reqwest-client/docs/RegisterTableRequest.md +++ b/rust/lance-catalog-reqwest-client/docs/RegisterTableRequest.md @@ -5,6 +5,7 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **name** | **String** | | +**namespace** | **Vec** | | **location** | **String** | | [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md) diff --git a/rust/lance-catalog-reqwest-client/docs/TableApi.md b/rust/lance-catalog-reqwest-client/docs/TableApi.md index 715e51155..703da8ccd 100644 --- a/rust/lance-catalog-reqwest-client/docs/TableApi.md +++ b/rust/lance-catalog-reqwest-client/docs/TableApi.md @@ -4,26 +4,26 @@ All URIs are relative to *http://localhost:2333* Method | HTTP request | Description ------------- | ------------- | ------------- -[**get_table**](TableApi.md#get_table) | **GET** /v1/namespaces/{ns}/tables/{table} | Get a table from the catalog -[**register_table**](TableApi.md#register_table) | **POST** /v1/namespaces/{ns}/register | Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. -[**table_exists**](TableApi.md#table_exists) | **HEAD** /v1/namespaces/{ns}/tables/{table} | Check if a table exists +[**get_table**](TableApi.md#get_table) | **GET** /v1/tables/{table} | Get a table from the namespace +[**register_table**](TableApi.md#register_table) | **POST** /v1/table/register | Register a table to a namespace +[**table_exists**](TableApi.md#table_exists) | **HEAD** /v1/tables/{table} | Check if a table exists ## get_table -> models::GetTableResponse get_table(ns, table) -Get a table from the catalog +> models::GetTableResponse get_table(table, delimiter) +Get a table from the namespace -Get a table's detailed information under a specified namespace from the catalog. +Get a table's detailed information. ### Parameters Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- -**ns** | **String** | The name of the namespace. | [required] | -**table** | **String** | A table name. | [required] | +**table** | **String** | A string identifier of the table | [required] | +**delimiter** | Option<**String**> | The delimiter for the identifier used in the context | | ### Return type @@ -43,15 +43,16 @@ No authorization required ## register_table -> models::GetTableResponse register_table(ns, register_table_request) -Register a new table in the given namespace. A table represents a lance dataset. In Lance catalog, a table must be hosted in a namespace. +> models::GetTableResponse register_table(register_table_request) +Register a table to a namespace + +Register an existing table at a given storage location to a namespace. ### Parameters Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- -**ns** | **String** | The name of the namespace. | [required] | **register_table_request** | [**RegisterTableRequest**](RegisterTableRequest.md) | | [required] | ### Return type @@ -72,18 +73,18 @@ No authorization required ## table_exists -> table_exists(ns, table) +> table_exists(table, delimiter) Check if a table exists -Check if a table exists within a given namespace. +Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. ### Parameters Name | Type | Description | Required | Notes ------------- | ------------- | ------------- | ------------- | ------------- -**ns** | **String** | The name of the namespace. | [required] | -**table** | **String** | A table name. | [required] | +**table** | **String** | A string identifier of the table | [required] | +**delimiter** | Option<**String**> | The delimiter for the identifier used in the context | | ### Return type diff --git a/rust/lance-catalog-reqwest-client/src/apis/configuration.rs b/rust/lance-catalog-reqwest-client/src/apis/configuration.rs index 8657df1dc..6669873f5 100644 --- a/rust/lance-catalog-reqwest-client/src/apis/configuration.rs +++ b/rust/lance-catalog-reqwest-client/src/apis/configuration.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * diff --git a/rust/lance-catalog-reqwest-client/src/apis/namespace_api.rs b/rust/lance-catalog-reqwest-client/src/apis/namespace_api.rs index 082a97030..e289f9b87 100644 --- a/rust/lance-catalog-reqwest-client/src/apis/namespace_api.rs +++ b/rust/lance-catalog-reqwest-client/src/apis/namespace_api.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -83,7 +83,8 @@ pub enum NamespaceExistsError { } -pub async fn create_namespace(configuration: &configuration::Configuration, create_namespace_request: models::CreateNamespaceRequest) -> Result> { +/// Create a new namespace. A namespace can manage either a collection of child namespaces, or a collection of tables. There are three modes when trying to create a namespace, to differentiate the behavior when a namespace of the same name already exists: * CREATE: the operation fails with 400. * EXIST_OK: the operation succeeds and the existing namespace is kept. * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. +pub async fn create_namespace(configuration: &configuration::Configuration, create_namespace_request: models::CreateNamespaceRequest) -> Result> { // add a prefix to parameters to efficiently prevent name collisions let p_create_namespace_request = create_namespace_request; @@ -110,8 +111,8 @@ pub async fn create_namespace(configuration: &configuration::Configuration, crea let content = resp.text().await?; match content_type { ContentType::Json => serde_json::from_str(&content).map_err(Error::from), - ContentType::Text => return Err(Error::from(serde_json::Error::custom("Received `text/plain` content type response that cannot be converted to `models::CreateNamespaceResponse`"))), - ContentType::Unsupported(unknown_type) => return Err(Error::from(serde_json::Error::custom(format!("Received `{unknown_type}` content type response that cannot be converted to `models::CreateNamespaceResponse`")))), + ContentType::Text => return Err(Error::from(serde_json::Error::custom("Received `text/plain` content type response that cannot be converted to `models::GetNamespaceResponse`"))), + ContentType::Unsupported(unknown_type) => return Err(Error::from(serde_json::Error::custom(format!("Received `{unknown_type}` content type response that cannot be converted to `models::GetNamespaceResponse`")))), } } else { let content = resp.text().await?; @@ -120,13 +121,18 @@ pub async fn create_namespace(configuration: &configuration::Configuration, crea } } -pub async fn drop_namespace(configuration: &configuration::Configuration, ns: &str) -> Result<(), Error> { +/// Drop a namespace. The namespace must be empty. +pub async fn drop_namespace(configuration: &configuration::Configuration, namespace: &str, delimiter: Option<&str>) -> Result<(), Error> { // add a prefix to parameters to efficiently prevent name collisions - let p_ns = ns; + let p_namespace = namespace; + let p_delimiter = delimiter; - let uri_str = format!("{}/v1/namespaces/{ns}", configuration.base_path, ns=crate::apis::urlencode(p_ns)); + let uri_str = format!("{}/v1/namespaces/{namespace}", configuration.base_path, namespace=crate::apis::urlencode(p_namespace)); let mut req_builder = configuration.client.request(reqwest::Method::DELETE, &uri_str); + if let Some(ref param_value) = p_delimiter { + req_builder = req_builder.query(&[("delimiter", ¶m_value.to_string())]); + } if let Some(ref user_agent) = configuration.user_agent { req_builder = req_builder.header(reqwest::header::USER_AGENT, user_agent.clone()); } @@ -145,14 +151,18 @@ pub async fn drop_namespace(configuration: &configuration::Configuration, ns: &s } } -/// Return a detailed information for a given namespace -pub async fn get_namespace(configuration: &configuration::Configuration, ns: &str) -> Result> { +/// Return the detailed information for a given namespace +pub async fn get_namespace(configuration: &configuration::Configuration, namespace: &str, delimiter: Option<&str>) -> Result> { // add a prefix to parameters to efficiently prevent name collisions - let p_ns = ns; + let p_namespace = namespace; + let p_delimiter = delimiter; - let uri_str = format!("{}/v1/namespaces/{ns}", configuration.base_path, ns=crate::apis::urlencode(p_ns)); + let uri_str = format!("{}/v1/namespaces/{namespace}", configuration.base_path, namespace=crate::apis::urlencode(p_namespace)); let mut req_builder = configuration.client.request(reqwest::Method::GET, &uri_str); + if let Some(ref param_value) = p_delimiter { + req_builder = req_builder.query(&[("delimiter", ¶m_value.to_string())]); + } if let Some(ref user_agent) = configuration.user_agent { req_builder = req_builder.header(reqwest::header::USER_AGENT, user_agent.clone()); } @@ -182,10 +192,13 @@ pub async fn get_namespace(configuration: &configuration::Configuration, ns: &st } } -pub async fn list_namespaces(configuration: &configuration::Configuration, page_token: Option<&str>, page_size: Option) -> Result> { +/// List all child namespace names of the root namespace or a given parent namespace. +pub async fn list_namespaces(configuration: &configuration::Configuration, page_token: Option<&str>, page_size: Option, parent: Option<&str>, delimiter: Option<&str>) -> Result> { // add a prefix to parameters to efficiently prevent name collisions let p_page_token = page_token; let p_page_size = page_size; + let p_parent = parent; + let p_delimiter = delimiter; let uri_str = format!("{}/v1/namespaces", configuration.base_path); let mut req_builder = configuration.client.request(reqwest::Method::GET, &uri_str); @@ -196,6 +209,12 @@ pub async fn list_namespaces(configuration: &configuration::Configuration, page_ if let Some(ref param_value) = p_page_size { req_builder = req_builder.query(&[("pageSize", ¶m_value.to_string())]); } + if let Some(ref param_value) = p_parent { + req_builder = req_builder.query(&[("parent", ¶m_value.to_string())]); + } + if let Some(ref param_value) = p_delimiter { + req_builder = req_builder.query(&[("delimiter", ¶m_value.to_string())]); + } if let Some(ref user_agent) = configuration.user_agent { req_builder = req_builder.header(reqwest::header::USER_AGENT, user_agent.clone()); } @@ -225,14 +244,18 @@ pub async fn list_namespaces(configuration: &configuration::Configuration, page_ } } -/// Check if a namespace exists. The response does not contain a body. -pub async fn namespace_exists(configuration: &configuration::Configuration, ns: &str) -> Result<(), Error> { +/// Check if a namespace exists. This API should behave exactly like the GetNamespace API, except it does not contain a body. +pub async fn namespace_exists(configuration: &configuration::Configuration, namespace: &str, delimiter: Option<&str>) -> Result<(), Error> { // add a prefix to parameters to efficiently prevent name collisions - let p_ns = ns; + let p_namespace = namespace; + let p_delimiter = delimiter; - let uri_str = format!("{}/v1/namespaces/{ns}", configuration.base_path, ns=crate::apis::urlencode(p_ns)); + let uri_str = format!("{}/v1/namespaces/{namespace}", configuration.base_path, namespace=crate::apis::urlencode(p_namespace)); let mut req_builder = configuration.client.request(reqwest::Method::HEAD, &uri_str); + if let Some(ref param_value) = p_delimiter { + req_builder = req_builder.query(&[("delimiter", ¶m_value.to_string())]); + } if let Some(ref user_agent) = configuration.user_agent { req_builder = req_builder.header(reqwest::header::USER_AGENT, user_agent.clone()); } diff --git a/rust/lance-catalog-reqwest-client/src/apis/table_api.rs b/rust/lance-catalog-reqwest-client/src/apis/table_api.rs index a39efd749..6871503bf 100644 --- a/rust/lance-catalog-reqwest-client/src/apis/table_api.rs +++ b/rust/lance-catalog-reqwest-client/src/apis/table_api.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -56,15 +56,18 @@ pub enum TableExistsError { } -/// Get a table's detailed information under a specified namespace from the catalog. -pub async fn get_table(configuration: &configuration::Configuration, ns: &str, table: &str) -> Result> { +/// Get a table's detailed information. +pub async fn get_table(configuration: &configuration::Configuration, table: &str, delimiter: Option<&str>) -> Result> { // add a prefix to parameters to efficiently prevent name collisions - let p_ns = ns; let p_table = table; + let p_delimiter = delimiter; - let uri_str = format!("{}/v1/namespaces/{ns}/tables/{table}", configuration.base_path, ns=crate::apis::urlencode(p_ns), table=crate::apis::urlencode(p_table)); + let uri_str = format!("{}/v1/tables/{table}", configuration.base_path, table=crate::apis::urlencode(p_table)); let mut req_builder = configuration.client.request(reqwest::Method::GET, &uri_str); + if let Some(ref param_value) = p_delimiter { + req_builder = req_builder.query(&[("delimiter", ¶m_value.to_string())]); + } if let Some(ref user_agent) = configuration.user_agent { req_builder = req_builder.header(reqwest::header::USER_AGENT, user_agent.clone()); } @@ -94,12 +97,12 @@ pub async fn get_table(configuration: &configuration::Configuration, ns: &str, t } } -pub async fn register_table(configuration: &configuration::Configuration, ns: &str, register_table_request: models::RegisterTableRequest) -> Result> { +/// Register an existing table at a given storage location to a namespace. +pub async fn register_table(configuration: &configuration::Configuration, register_table_request: models::RegisterTableRequest) -> Result> { // add a prefix to parameters to efficiently prevent name collisions - let p_ns = ns; let p_register_table_request = register_table_request; - let uri_str = format!("{}/v1/namespaces/{ns}/register", configuration.base_path, ns=crate::apis::urlencode(p_ns)); + let uri_str = format!("{}/v1/table/register", configuration.base_path); let mut req_builder = configuration.client.request(reqwest::Method::POST, &uri_str); if let Some(ref user_agent) = configuration.user_agent { @@ -132,15 +135,18 @@ pub async fn register_table(configuration: &configuration::Configuration, ns: &s } } -/// Check if a table exists within a given namespace. -pub async fn table_exists(configuration: &configuration::Configuration, ns: &str, table: &str) -> Result<(), Error> { +/// Check if a table exists. This API should behave exactly like the GetTable API, except it does not contain a body. +pub async fn table_exists(configuration: &configuration::Configuration, table: &str, delimiter: Option<&str>) -> Result<(), Error> { // add a prefix to parameters to efficiently prevent name collisions - let p_ns = ns; let p_table = table; + let p_delimiter = delimiter; - let uri_str = format!("{}/v1/namespaces/{ns}/tables/{table}", configuration.base_path, ns=crate::apis::urlencode(p_ns), table=crate::apis::urlencode(p_table)); + let uri_str = format!("{}/v1/tables/{table}", configuration.base_path, table=crate::apis::urlencode(p_table)); let mut req_builder = configuration.client.request(reqwest::Method::HEAD, &uri_str); + if let Some(ref param_value) = p_delimiter { + req_builder = req_builder.query(&[("delimiter", ¶m_value.to_string())]); + } if let Some(ref user_agent) = configuration.user_agent { req_builder = req_builder.header(reqwest::header::USER_AGENT, user_agent.clone()); } diff --git a/rust/lance-catalog-reqwest-client/src/models/create_namespace_request.rs b/rust/lance-catalog-reqwest-client/src/models/create_namespace_request.rs index c8d755c22..dd2a615bd 100644 --- a/rust/lance-catalog-reqwest-client/src/models/create_namespace_request.rs +++ b/rust/lance-catalog-reqwest-client/src/models/create_namespace_request.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -15,6 +15,8 @@ use serde::{Deserialize, Serialize}; pub struct CreateNamespaceRequest { #[serde(rename = "name")] pub name: String, + #[serde(rename = "parent", skip_serializing_if = "Option::is_none")] + pub parent: Option>, #[serde(rename = "mode")] pub mode: Mode, #[serde(rename = "options", skip_serializing_if = "Option::is_none")] @@ -25,6 +27,7 @@ impl CreateNamespaceRequest { pub fn new(name: String, mode: Mode) -> CreateNamespaceRequest { CreateNamespaceRequest { name, + parent: None, mode, options: None, } diff --git a/rust/lance-catalog-reqwest-client/src/models/create_namespace_response.rs b/rust/lance-catalog-reqwest-client/src/models/create_namespace_response.rs deleted file mode 100644 index 32cbb83de..000000000 --- a/rust/lance-catalog-reqwest-client/src/models/create_namespace_response.rs +++ /dev/null @@ -1,31 +0,0 @@ -/* - * Lance Catalog REST Specification - * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. - * - * The version of the OpenAPI document: 0.0.1 - * - * Generated by: https://openapi-generator.tech - */ - -use crate::models; -use serde::{Deserialize, Serialize}; - -#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)] -pub struct CreateNamespaceResponse { - #[serde(rename = "name")] - pub name: String, - /// Properties stored on the namespace, if supported by the server. - #[serde(rename = "properties", skip_serializing_if = "Option::is_none")] - pub properties: Option>, -} - -impl CreateNamespaceResponse { - pub fn new(name: String) -> CreateNamespaceResponse { - CreateNamespaceResponse { - name, - properties: None, - } - } -} - diff --git a/rust/lance-catalog-reqwest-client/src/models/error_response.rs b/rust/lance-catalog-reqwest-client/src/models/error_response.rs index a62a08a5e..487363e6a 100644 --- a/rust/lance-catalog-reqwest-client/src/models/error_response.rs +++ b/rust/lance-catalog-reqwest-client/src/models/error_response.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * diff --git a/rust/lance-catalog-reqwest-client/src/models/get_namespace_response.rs b/rust/lance-catalog-reqwest-client/src/models/get_namespace_response.rs index 2e533cebc..a947f7401 100644 --- a/rust/lance-catalog-reqwest-client/src/models/get_namespace_response.rs +++ b/rust/lance-catalog-reqwest-client/src/models/get_namespace_response.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -13,17 +13,20 @@ use serde::{Deserialize, Serialize}; #[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)] pub struct GetNamespaceResponse { - #[serde(rename = "namespace")] - pub namespace: String, + #[serde(rename = "name")] + pub name: String, + #[serde(rename = "parent", skip_serializing_if = "Option::is_none")] + pub parent: Option>, /// Properties stored on the namespace, if supported by the server. If the server does not support namespace properties, it should return null for this field. If namespace properties are supported, but none are set, it should return an empty object. #[serde(rename = "properties", skip_serializing_if = "Option::is_none")] pub properties: Option>, } impl GetNamespaceResponse { - pub fn new(namespace: String) -> GetNamespaceResponse { + pub fn new(name: String) -> GetNamespaceResponse { GetNamespaceResponse { - namespace, + name, + parent: None, properties: None, } } diff --git a/rust/lance-catalog-reqwest-client/src/models/get_table_response.rs b/rust/lance-catalog-reqwest-client/src/models/get_table_response.rs index 0dd38ebfa..4e6c14293 100644 --- a/rust/lance-catalog-reqwest-client/src/models/get_table_response.rs +++ b/rust/lance-catalog-reqwest-client/src/models/get_table_response.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -11,11 +11,12 @@ use crate::models; use serde::{Deserialize, Serialize}; -/// GetTableResponse : Result used when a table is successfully loaded. #[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)] pub struct GetTableResponse { #[serde(rename = "name")] pub name: String, + #[serde(rename = "namespace")] + pub namespace: Vec, #[serde(rename = "location")] pub location: String, #[serde(rename = "properties", skip_serializing_if = "Option::is_none")] @@ -23,10 +24,10 @@ pub struct GetTableResponse { } impl GetTableResponse { - /// Result used when a table is successfully loaded. - pub fn new(name: String, location: String) -> GetTableResponse { + pub fn new(name: String, namespace: Vec, location: String) -> GetTableResponse { GetTableResponse { name, + namespace, location, properties: None, } diff --git a/rust/lance-catalog-reqwest-client/src/models/list_namespaces_response.rs b/rust/lance-catalog-reqwest-client/src/models/list_namespaces_response.rs index 94326de46..bd5549cf4 100644 --- a/rust/lance-catalog-reqwest-client/src/models/list_namespaces_response.rs +++ b/rust/lance-catalog-reqwest-client/src/models/list_namespaces_response.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -13,7 +13,6 @@ use serde::{Deserialize, Serialize}; #[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)] pub struct ListNamespacesResponse { - /// An array of namespace names in the catalog. #[serde(rename = "namespaces")] pub namespaces: Vec, /// An opaque token that allows pagination for list APIs (e.g. ListNamespaces). For an initial client request for a list API, if the server cannot return all items in one response, or if there are more items than the `pageSize` specified in the client request, the server must return a `nextPageToken` in the response indicating there are more results available. After the initial request, the value of `nextPageToken` from each response must be used by the client as the `pageToken` parameter value for the next request. Clients must interpret either `null`, missing value or empty string value of `nextPageToken` from a server response as the end of the listing results. diff --git a/rust/lance-catalog-reqwest-client/src/models/mod.rs b/rust/lance-catalog-reqwest-client/src/models/mod.rs index 8f04e336f..a56ec5fbb 100644 --- a/rust/lance-catalog-reqwest-client/src/models/mod.rs +++ b/rust/lance-catalog-reqwest-client/src/models/mod.rs @@ -1,7 +1,5 @@ pub mod create_namespace_request; pub use self::create_namespace_request::CreateNamespaceRequest; -pub mod create_namespace_response; -pub use self::create_namespace_response::CreateNamespaceResponse; pub mod error_response; pub use self::error_response::ErrorResponse; pub mod get_namespace_response; diff --git a/rust/lance-catalog-reqwest-client/src/models/register_table_request.rs b/rust/lance-catalog-reqwest-client/src/models/register_table_request.rs index 99a03fc19..ffb9b7731 100644 --- a/rust/lance-catalog-reqwest-client/src/models/register_table_request.rs +++ b/rust/lance-catalog-reqwest-client/src/models/register_table_request.rs @@ -1,7 +1,7 @@ /* - * Lance Catalog REST Specification + * Lance REST Namespace Specification * - * **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, which is a REST server that converts the Lance catalog requests to native requests against the catalog service. Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. + * **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format to standardize access to a collection of Lance tables (a.k.a. Lance datasets). It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. should store and use Lance tables, as well as how ML/AI tools and analytics compute engines (will together be called _\"tools\"_ in this document) should integrate with Lance tables. A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. It can either contain a collection of tables, or a collection of Lance namespaces recursively. It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. In an enterprise environment, typically there is a requirement to store tables in a metadata service such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. for more advanced governance features around access control, auditing, lineage tracking, etc. **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables by connecting those metadata services or building a custom metadata server in a standardized way. The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). * * The version of the OpenAPI document: 0.0.1 * @@ -15,14 +15,17 @@ use serde::{Deserialize, Serialize}; pub struct RegisterTableRequest { #[serde(rename = "name")] pub name: String, + #[serde(rename = "namespace")] + pub namespace: Vec, #[serde(rename = "location")] pub location: String, } impl RegisterTableRequest { - pub fn new(name: String, location: String) -> RegisterTableRequest { + pub fn new(name: String, namespace: Vec, location: String) -> RegisterTableRequest { RegisterTableRequest { name, + namespace, location, } } diff --git a/spec/catalog-layout.png b/spec/catalog-layout.png new file mode 100644 index 000000000..148c9e4e8 Binary files /dev/null and b/spec/catalog-layout.png differ diff --git a/spec/layout.png b/spec/layout.png new file mode 100644 index 000000000..b45252688 Binary files /dev/null and b/spec/layout.png differ diff --git a/spec/catalog.yaml b/spec/rest.yaml similarity index 78% rename from spec/catalog.yaml rename to spec/rest.yaml index c8f29a870..7746c6094 100644 --- a/spec/catalog.yaml +++ b/spec/rest.yaml @@ -13,19 +13,29 @@ --- openapi: 3.1.1 info: - title: Lance Catalog REST Specification + title: Lance REST Namespace Specification license: name: Apache 2.0 url: https://www.apache.org/licenses/LICENSE-2.0.html version: 0.0.1 - description: | - **Lance Catalog** is an OpenAPI specification on top of the storage-based Lance format. - It provides an integration point for catalog service like Apache Hive MetaStore (HMS), Apache Gravitino, etc. - to store and use Lance tables. To integrate, the catalog service implements a **Lance Catalog Adapter**, - which is a REST server that converts the Lance catalog requests to native requests against the catalog service. - Different tools can integrate with Lance Catalog using the generated OpenAPI clients in various languages, - and invoke operations in Lance Catalog to read, write and manage Lance tables in the integrated catalog services. - + description: > + **Lance Namespace Specification** is an open specification on top of the storage-based Lance data format + to standardize access to a collection of Lance tables (a.k.a. Lance datasets). + It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. + should store and use Lance tables, as well as how ML/AI tools and analytics compute engines + (will together be called _"tools"_ in this document) should integrate with Lance tables. + + A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. + It can either contain a collection of tables, or a collection of Lance namespaces recursively. + It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. + that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. + + In an enterprise environment, typically there is a requirement to store tables in a metadata service + such as Apache Hive MetaStore, Apache Gravitino, Unity Namespace, etc. + for more advanced governance features around access control, auditing, lineage tracking, etc. + **Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables + by connecting those metadata services or building a custom metadata server in a standardized way. + The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). servers: - url: "{scheme}://{host}:{port}/{basePath}" description: Generic server URL with all parts configurable @@ -56,13 +66,18 @@ paths: /v1/namespaces: post: tags: [ Namespace ] - summary: > - Create a new namespace. A catalog can manage one or more namespaces. A namespace is used to manage one or more tables. - There are three modes when trying to create a namespace: - * CREATE: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation fails with 400. - * EXIST_OK: Create the namespace if it does not exist. If a namespace of the same name already exists, the operation succeeds and the existing namespace is kept. - * OVERWRITE: Create the namespace if it does not exist. If a namespace of the same name already exists, the existing namespace is dropped and a new namespace with this name with no table is created. + summary: Create a new namespace operationId: CreateNamespace + description: > + Create a new namespace. + + A namespace can manage either a collection of child namespaces, or a collection of tables. + + There are three modes when trying to create a namespace, + to differentiate the behavior when a namespace of the same name already exists: + * CREATE: the operation fails with 400. + * EXIST_OK: the operation succeeds and the existing namespace is kept. + * OVERWRITE: the existing namespace is dropped and a new empty namespace with this name is created. requestBody: required: true content: @@ -71,7 +86,7 @@ paths: $ref: '#/components/schemas/CreateNamespaceRequest' responses: 200: - $ref: '#/components/responses/CreateNamespaceResponse' + $ref: '#/components/responses/GetNamespaceResponse' 400: $ref: '#/components/responses/BadRequestErrorResponse' 401: @@ -89,12 +104,15 @@ paths: get: tags: [ Namespace ] - summary: > - List all namespaces in the catalog. + summary: List namespaces operationId: ListNamespaces + description: > + List all child namespace names of the root namespace or a given parent namespace. parameters: - $ref: '#/components/parameters/pageToken' - $ref: '#/components/parameters/pageSize' + - $ref: '#/components/parameters/parent' + - $ref: '#/components/parameters/delimiter' responses: 200: $ref: '#/components/responses/ListNamespacesResponse' @@ -111,15 +129,17 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' - /v1/namespaces/{ns}: + /v1/namespaces/{namespace}: parameters: - $ref: '#/components/parameters/namespace' + - $ref: '#/components/parameters/delimiter' get: tags: [ Namespace ] summary: Get information about a namespace operationId: GetNamespace - description: Return a detailed information for a given namespace + description: > + Return the detailed information for a given namespace responses: 200: $ref: '#/components/responses/GetNamespaceResponse' @@ -138,8 +158,10 @@ paths: delete: tags: [ Namespace ] - summary: Drop a namespace from the catalog. Namespace must be empty. + summary: Drop a namespace operationId: DropNamespace + description: > + Drop a namespace. The namespace must be empty. responses: 204: description: Success, no content @@ -162,8 +184,10 @@ paths: tags: [ Namespace ] summary: Check if a namespace exists operationId: NamespaceExists - description: - Check if a namespace exists. The response does not contain a body. + description: > + Check if a namespace exists. + + This API should behave exactly like the GetNamespace API, except it does not contain a body. responses: 200: description: Success, no content @@ -180,15 +204,13 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' - /v1/namespaces/{ns}/register: + /v1/table/register: post: tags: [ Table ] - summary: > - Register a new table in the given namespace. A table represents a lance dataset. - In Lance catalog, a table must be hosted in a namespace. + summary: Register a table to a namespace operationId: RegisterTable - parameters: - - $ref: '#/components/parameters/namespace' + description: > + Register an existing table at a given storage location to a namespace. requestBody: required: true content: @@ -213,17 +235,17 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' - /v1/namespaces/{ns}/tables/{table}: + /v1/tables/{table}: parameters: - - $ref: '#/components/parameters/namespace' - $ref: '#/components/parameters/table' + - $ref: '#/components/parameters/delimiter' get: tags: [ Table ] - summary: Get a table from the catalog + summary: Get a table from the namespace operationId: GetTable - description: - Get a table's detailed information under a specified namespace from the catalog. + description: > + Get a table's detailed information. responses: 200: $ref: '#/components/responses/GetTableResponse' @@ -244,8 +266,10 @@ paths: tags: [ Table ] summary: Check if a table exists operationId: TableExists - description: - Check if a table exists within a given namespace. + description: > + Check if a table exists. + + This API should behave exactly like the GetTable API, except it does not contain a body. responses: 200: description: Success, no content @@ -265,11 +289,17 @@ paths: components: parameters: namespace: - name: ns + name: namespace in: path required: true - description: - The name of the namespace. + description: A string identifier of the namespace. + schema: + type: string + + parent: + name: parent + in: query + description: A string identifier of the parent namespace. schema: type: string @@ -284,8 +314,7 @@ components: pageSize: name: pageSize in: query - description: - An inclusive upper bound of the number of results that a client will receive. + description: An inclusive upper bound of the number of results that a client will receive. required: false schema: type: integer @@ -294,12 +323,19 @@ components: table: name: table in: path - description: - A table name. + description: A string identifier of the table required: true schema: type: string + delimiter: + name: delimiter + in: query + description: The delimiter for the identifier used in the context + required: false + schema: + type: string + schemas: ErrorResponse: type: object @@ -332,30 +368,26 @@ components: CreateNamespaceRequest: type: object - required: [ name, mode ] + required: + - name + - mode properties: - name: { type: string } + name: + type: string + parent: + type: array + items: + type: string mode: type: string - enum: [ CREATE, EXIST_OK, OVERWRITE ] + enum: + - CREATE + - EXIST_OK + - OVERWRITE options: - type: object - additionalProperties: { type: string } - - CreateNamespaceResponse: - type: object - required: - - name - properties: - name: { type: string } - properties: type: object additionalProperties: type: string - description: - Properties stored on the namespace, if supported by the server. - example: { "created_at": "1452120468" } - default: { } ListNamespacesResponse: type: object @@ -363,7 +395,6 @@ components: - namespaces properties: namespaces: - description: An array of namespace names in the catalog. type: array uniqueItems: true items: @@ -375,10 +406,14 @@ components: GetNamespaceResponse: type: object required: - - namespace + - name properties: - namespace: + name: type: string + parent: + type: array + items: + type: string properties: type: object description: @@ -411,23 +446,31 @@ components: type: object required: - name + - namespace - location properties: name: type: string + namespace: + type: array + items: + type: string location: type: string GetTableResponse: - description: | - Result used when a table is successfully loaded. type: object required: - name + - namespace - location properties: name: type: string + namespace: + type: array + items: + type: string location: type: string properties: @@ -436,21 +479,6 @@ components: type: string responses: - CreateNamespaceResponse: - description: - Represents a successful call to create a namespace. - Returns the namespace created, as well as any properties that were stored for the namespace, - including those the server might have added. Implementations are not required to support namespace - properties. - content: - application/json: - schema: - $ref: '#/components/schemas/CreateNamespaceResponse' - example: { - "namespace": [ "lance.ns1" ], - "properties": { "created_at": "1452120468" } - } - ListNamespacesResponse: description: A list of namespaces content: @@ -539,7 +567,7 @@ components: "title": "Not found Error", "status": 404, "detail": "", - "instance": "/v1/namespaces/{ns}" + "instance": "/v1/namespaces/{namespace}" } UnsupportedOperationErrorResponse: @@ -567,7 +595,7 @@ components: "title": "The namespace has been concurrently modified", "status": 409, "detail": "", - "instance": "/v1/namespaces/{ns}" + "instance": "/v1/namespaces/{namespace}" } ServiceUnavailableErrorResponse: diff --git a/spec/spec.md b/spec/spec.md new file mode 100644 index 000000000..bbfa89bd5 --- /dev/null +++ b/spec/spec.md @@ -0,0 +1,196 @@ +# Lance Namespace Specification + +**Lance Namespace Specification** is an open specification on top of the storage-based Lance data format +to standardize access to a collection of Lance tables (a.k.a. Lance datasets). +It describes how a metadata service like Apache Hive MetaStore (HMS), Apache Gravitino, Unity Namespace, etc. +should store and use Lance tables, as well as how ML/AI tools and analytics compute engines +(will together be called _"tools"_ in this document) should integrate with Lance tables. + +## Concepts + +### Namespace Definition + +A Lance namespace is a centralized repository for discovering, organizing, and managing Lance tables. +It can either contain a collection of tables, or a collection of Lance namespaces recursively. +It is designed to encapsulates concepts including namespace, metastore, database, namespace, schema, etc. +that frequently appear in other similar data systems to allow easy integration with any system of any type of object hierarchy. + +Here is an example layout of a Lance namespace: + +![Lance namespace layout](./layout.png) + +### Parent & Child + +We use the term **parent** and **child** to describe relationship between 2 objects. +If namespace A directly contains B, then A is the parent namespace of B, i.e. B is a child of A. +For examples: + +- Namespace `ns1` contains 3 **child namespaces** `ns2`, `ns3` and `ns4`, i.e. `ns1` is the **parent namespace** of `ns2`, `ns3` and `ns4`. +- Namespace `ns2` contains a **child namespace** `ns5`. i.e. `ns2` is the **parent namespace** of `ns5`. +- Namespace `ns3` contains a **child table** `t2`, i.e. `t2` belongs to **parent namespace** `ns3`. + +### Root Namespace + +A root namespace is a namespace that has no parent. +In the example layout, `ns1` is a root namespace. + +### Object Name + +The **name** of an object is a string that uniquely identifies the object within the namespace it belongs to. +The name of any object must be unique among all other objects that share the same parent namespace. +For examples: + +- `cat2`, `cat3` and `cat4` are all unique names under `cat1` +- `t3` and `t4` are both unique names under `cat4` + +### Object Identifier + +The **identifier** of an object uniquely identifies the object within the root namespace it belongs to. +The identifier of any object must be unique among all other objects that share the same root namespace. + +Based on the uniqueness property of an object name, +an object identifier is the list of object names starting from (not including) the root namespace to the object itself. +This is also called the **list identifier**. +For examples: + +- the list identifier of `cat5` is `[cat2, cat5]` +- the list identifier of `t1` is `[cat2, cat5, t1]` + +The dot (`.`) symbol is typically used as the delimiter to join all the names to form an **string identifier**, +but other symbols could also be used if dot is used in the object name. +For examples: + +- the string identifier of `cat5` is `cat2.cat5` +- the string identifier of `t1` is `cat2.cat5.t1` +- the string identifier of `t3` is `cat4$t3` when using delimiter `$` + +## Name and Identifier for Root Namespace + +The root namespace itself has no name and identifier. +When represented in code, its name and string identifier is represented by an empty or null string, +and its list identifier is represented by an empty or null list. + +The actual name and identifier of the root namespace is typically +assigned by users through some configuration when used in a tool. +For example, a root namespace can be called `cat1` in Ray, but `cat2` in Apache Spark, +but they are both configured to connect to the same server with URI `https://mylancenamespace.com`. + +## Namespace Types + +There are 2 types of Lance namespaces: directory and REST. + +### Lance Directory Namespace + +**Lance directory namespace** is a lightweight and simple Lance namespace that only contains a list of tables. +People can easily get started with creating and using Lance tables directly on top of any +local or remote storage system with a Lance directory namespace. + +A directory namespace maps to a directory on storage, we call such directory a **namespace directory**. +A Lance table corresponds to a subdirectory in the namespace directory. +We call such a subdirectories **table directory**. +Consider the following example namespace directory layout: + +``` +. +└── /my/dir1/ + ├── table1/ + │ ├── data/ + │ │ ├── 0aa36d91-8293-406b-958c-faf9e7547938.lance + │ │ └── ed7af55d-b064-4442-bcb5-47b524e98d0e.lance + │ ├── _versions/ + │ │ └── 9223372036854775707.manifest + │ ├── _indices/ + │ │ └── 85814508-ed9a-41f2-b939-2050bb7a0ed5-fts/ + │ │ └── index.idx + │ └── _deletions/ + │ └── 75c69434-cde5-4c80-9fe1-e79a6d952fbf.bin + ├── table2 + └── table3 +``` + +This describes a Lance directory namespace with the namespace directory at `/my/dir1/`. +It contains tables `table1`, `table2`, `table3` sitting at table directories +`/my/dirs/table1`, `/my/dirs/table2`, `/my/dirs/table3` respectively. + +#### Directory Path + +There are 3 ways to specify a directory path: + +1. **URI**: a URI that follows the [RFC 3986 specification](https://datatracker.ietf.org/doc/html/rfc3986), e.g. `s3://mu-bucket/prefix`. +2. **Absolute POSIX storage path**: an absolute file path in a POSIX standard storage, e.g. `/my/dir`. +3. **Relative POSIX storage path**: a relative file path in a POSIX standard storage, e.g. `my/dir2`, `./my/dir3`. + The absolute path of the directory should be based on the current directory of the running process. + +#### Table Existence + +A table exists in a Lance directory namespace if a table directory of the specific name exists. +This is true even if the directory is empty or the contents in the directory does not follow the Lance table format spec. +For such cases, an operation that lists all tables in the directory should show the specific table, +and an operation that checks if a table exists should return true. +However, an operation that loads the Lance table metadata should fail with error +indicating the content in the folder is not compliant with the Lance table format spec. + +### Lance REST Namespace + +In an enterprise environment, typically there is a requirement to store tables in a metadata service +such as Apache Hive MetaStore, Apache Gravitino, Unity Catalog, etc. +for more advanced governance features around access control, auditing, lineage tracking, etc. +**Lance REST Namespace** is an OpenAPI protocol that enables reading, writing and managing Lance tables +by connecting those metadata services or building a custom metadata server in a standardized way. +The detailed OpenAPI specification content can be found in [rest.yaml](./rest.yaml). + +#### Namespace Server and Adapter + +Any REST HTTP server that implements this OpenAPI protocol is called a **Lance Namespace Server**. +If the main purpose of this server is to be a proxy on top of an existing metadata service, +converting back and forth between Lance REST API models and native API models of the metadata service, +then this Lance namespace server is called a **Lance Namespace Adapter**. + +#### Server and Storage + +Under the REST protocol, a Lance table exists both in the storage and the server. +There are 2 possible ways to manage a Lance table under such setting. +A Lance namespace server can choose to support one or both ways. + +##### Server Managed Table + +A server managed Lance table is a table that is fully managed by the Lance namespace server. +The server must maintain information about the latest version of the Lance table. +Any modifications to the table must happen through the server. +If a user directly modifies the underlying table in the storage bypassing server, +the server must not reflect the changes in the table to the namespace users. + +This mode ensures the namespace service is aware of all activities in the table, +and can thus fully enforce any governance and management features for the table. + +##### Storage Managed Table + +A storage managed Lance table is a table that is fully managed by the storage +with a metadata definition in the Lance namespace server. +The server only contains information about the table directory location. +It is expected that a tool finds the latest version of the Lance table based on the contents +in the table directory according to the Lance format specification. +A modification to the table can happen either directly against the storage, +or happen as a request to the server, where the server is responsible for applying the corresponding +change to the underlying storage according to the Lance format specification. + +This mode is more flexible for real world ML/AI workflows +but the server loses full visibility and control over the actions performed against the table, +so it will be harder to enforce any governance and management features for storage managed tables. + +## Tool Integration Guidelines + +The following are guidelines for tools to integrate with Lance namespaces. +Note that these are recommendations rather than hard requirements. +The goal of these guidelines is to offer a consistent user experience across different tools. + +### Configurations + +We recommend tools to offer the following configurations in some form or shape +for users to configure connection to a Lance namespace: + +| Config Key | Description | Required? | +|------------|-------------------------------------------------------------------------------------------------|-------------------------------| +| type | The type of the namespace, either `dir` for Lance directory, or `rest` for Lance REST Namespace | Yes | +| path | The path to the Lance directory namespace | Yes for `dir` namespace type | +| uri | The HTTP URI for the Lance REST Namespace | Yes for `rest` namespace type |