feat: more fixes (lance-format#5)

Author: jackye1995

docs/src/glue.md

@@ -6,18 +6,19 @@ implements the Lance Namespace client spec.
 ## Background
 
 AWS Glue Data Catalog is a fully managed metadata repository that stores structural and operational metadata for data assets. 
-It is compatible with the Apache Hive Metastore API and can be used as a central metadata repository for data lakes. 
+It is based on the Apache Hive Metastore API, but uses JSON RPC instead of Apache Thrift for request response.
+It can be used as a central metadata repository for data lakes. 
 For details on AWS Glue, see the [AWS Glue Data Catalog Documentation](https://docs.aws.amazon.com/glue/latest/dg/manage-catalog.html).
 
 ## Namespace Implementation Configuration Properties
 
 The Lance Glue namespace implementation accepts the following configuration properties:
 
-The **catalog_id** property is optional and specifies the Catalog ID of the Glue catalog (defaults to AWS account ID).
+The **catalog_id** property is optional and specifies the Catalog ID of the Glue catalog to use as the starting point. When not specified, it is resolved to the caller's AWS account ID.
 
 The **endpoint** property is optional and specifies a custom Glue service endpoint for API compatible metastores.
 
-The **region** property is optional and specifies the AWS region for all Glue operations.
+The **region** property is optional and specifies the AWS region for all Glue operations. When not specified, it is resolved to the default AWS region in the caller's environment.
 
 The **access_key_id** property is optional and specifies the AWS access key ID for static credentials.
 
@@ -51,7 +52,7 @@ The Glue namespace supports multiple authentication methods:
 AWS Glue Data Catalog supports a recursive catalog structure through the [GetCatalog](https://docs.aws.amazon.com/glue/latest/webapi/API_GetCatalog.html) and [GetCatalogs](https://docs.aws.amazon.com/glue/latest/webapi/API_GetCatalogs.html) APIs. 
 This allows for multi-level namespace hierarchies.
 
-The **root namespace** is represented by the default AWS Glue Data Catalog, which has a catalog ID of `None` or equal to the caller's AWS account ID.
+The **root namespace** is represented by the default AWS Glue Data Catalog. When the `catalog_id` property is not specified or set to `None`, it is resolved to the caller's AWS account ID. Users can specify a different `catalog_id` to use another AWS account's Glue catalog as the starting point.
 
 A **child catalog** within the root catalog forms a child namespace. The [GetCatalogs](https://docs.aws.amazon.com/glue/latest/webapi/API_GetCatalogs.html) API supports `ParentCatalogId` parameter to traverse the catalog hierarchy.
 

docs/src/gravitino.md

@@ -1,4 +1,4 @@
-# Lance Gravitino Namespace
+# Apache Gravitino
 
 **Apache Gravitino** is a high-performance, geo-distributed, and federated metadata lake.
 It manages metadata directly in different sources, types, and regions, providing unified metadata access for data and AI assets.

docs/src/hive2.md

@@ -1,4 +1,4 @@
-# Lance Hive 2.x Namespace Implementation Spec
+# Apache Hive 2.X MetaStore Lance Namespace Implementation Spec
 
 This document describes how the Hive 2.x MetaStore implements the Lance Namespace client spec.
 
@@ -14,8 +14,6 @@ The **client.pool-size** property is optional and specifies the size of the HMS
 
 The **root** property is optional and specifies the storage root location of the lakehouse on Hive catalog. Default value is the current working directory.
 
-The **storage.*** prefix properties are optional and specify additional storage configurations to access tables (e.g., `storage.region=us-west-2`).
-
 ## Object Mapping
 
 ### Namespace
@@ -56,7 +54,9 @@ The implementation:
 
 **Error Handling:**
 
-If the namespace already exists and mode is CREATE, return error code `2` (NamespaceAlreadyExists). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the namespace already exists and mode is CREATE, return error code `2` (NamespaceAlreadyExists).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### ListNamespaces
 
@@ -83,7 +83,9 @@ The implementation:
 
 **Error Handling:**
 
-If the namespace does not exist, return error code `1` (NamespaceNotFound). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### DropNamespace
 
@@ -113,13 +115,17 @@ The implementation:
 1. Parse the table identifier to extract database and table name
 2. Verify the parent namespace exists
 3. Create an HMS Table object with `tableType=EXTERNAL_TABLE`
-4. Set the storage descriptor with the specified or default location
+4. Set the storage descriptor with the specified or default location. When location is not specified, it defaults to `{root}/{database}.db/{table}`
 5. Add `table_type=lance` to the table parameters
 6. Register the table in HMS
 
 **Error Handling:**
 
-If the parent namespace does not exist, return error code `1` (NamespaceNotFound). If the table already exists, return error code `5` (TableAlreadyExists). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the parent namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the table already exists, return error code `5` (TableAlreadyExists).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### ListTables
 
@@ -135,18 +141,20 @@ The implementation:
 
 **Error Handling:**
 
-If the namespace does not exist, return error code `1` (NamespaceNotFound). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### DescribeTable
 
-Retrieves metadata for a Lance table. Only `load_detailed_metadata=false` is supported. When `load_detailed_metadata=false`, only the table location and storage_options are returned; other fields (version, table_uri, schema, stats) are null.
+Retrieves metadata for a Lance table. Only `load_detailed_metadata=false` is supported. When `load_detailed_metadata=false`, only the table location is returned; other fields (version, table_uri, schema, stats) are null.
 
 The implementation:
 
 1. Parse the table identifier
 2. Retrieve the Table object from HMS
 3. Validate that it is a Lance table (check `table_type=lance`)
-4. Return the table location from `storageDescriptor.location` and storage_options from `parameters`
+4. Return the table location from `storageDescriptor.location`
 
 **Error Handling:**
 
@@ -168,4 +176,8 @@ The implementation:
 
 **Error Handling:**
 
-If the table does not exist, return error code `4` (TableNotFound). If the table is not a Lance table, return error code `13` (InvalidInput). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the table does not exist, return error code `4` (TableNotFound).
+
+If the table is not a Lance table, return error code `13` (InvalidInput).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).

docs/src/hive3.md

@@ -1,4 +1,4 @@
-# Lance Hive 3.x Namespace Implementation Spec
+# Apache Hive 3+.X MetaStore Lance Namespace Implementation Spec
 
 This document describes how the Hive 3.x MetaStore implements the Lance Namespace client spec.
 
@@ -8,14 +8,12 @@ Apache Hive MetaStore (HMS) is a centralized metadata repository for Apache Hive
 
 ## Namespace Implementation Configuration Properties
 
-The Lance Hive 3.x namespace implementation accepts the following configuration properties:
+The Lance Hive 3+.x namespace implementation accepts the following configuration properties:
 
 The **client.pool-size** property is optional and specifies the size of the HMS client connection pool. Default value is `3`.
 
 The **root** property is optional and specifies the storage root location of the lakehouse on Hive catalog. Default value is the current working directory.
 
-The **storage.*** prefix properties are optional and specify additional storage configurations to access tables (e.g., `storage.region=us-west-2`).
-
 ## Object Mapping
 
 ### Namespace
@@ -57,7 +55,11 @@ The implementation:
 
 **Error Handling:**
 
-If the namespace already exists and mode is CREATE, return error code `2` (NamespaceAlreadyExists). If the parent catalog does not exist when creating a database, return error code `1` (NamespaceNotFound). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the namespace already exists and mode is CREATE, return error code `2` (NamespaceAlreadyExists).
+
+If the parent catalog does not exist when creating a database, return error code `1` (NamespaceNotFound).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### ListNamespaces
 
@@ -72,7 +74,9 @@ The implementation:
 
 **Error Handling:**
 
-If the parent namespace does not exist, return error code `1` (NamespaceNotFound). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the parent namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### DescribeNamespace
 
@@ -86,7 +90,9 @@ The implementation:
 
 **Error Handling:**
 
-If the namespace does not exist, return error code `1` (NamespaceNotFound). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### DropNamespace
 
@@ -116,13 +122,17 @@ The implementation:
 1. Parse the table identifier to extract catalog, database, and table name
 2. Verify the parent namespace exists
 3. Create an HMS Table object with `tableType=EXTERNAL_TABLE`
-4. Set the storage descriptor with the specified or default location
+4. Set the storage descriptor with the specified or default location. When location is not specified, it defaults to `{root}/{database}.db/{table}` for the default `hive` catalog (hive2-compatible), or `{root}/{catalog}/{database}.db/{table}` for other catalogs
 5. Add `table_type=lance` to the table parameters
 6. Register the table in HMS
 
 **Error Handling:**
 
-If the parent namespace does not exist, return error code `1` (NamespaceNotFound). If the table already exists, return error code `5` (TableAlreadyExists). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the parent namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the table already exists, return error code `5` (TableAlreadyExists).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### ListTables
 
@@ -138,18 +148,20 @@ The implementation:
 
 **Error Handling:**
 
-If the namespace does not exist, return error code `1` (NamespaceNotFound). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the namespace does not exist, return error code `1` (NamespaceNotFound).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).
 
 ### DescribeTable
 
-Retrieves metadata for a Lance table. Only `load_detailed_metadata=false` is supported. When `load_detailed_metadata=false`, only the table location and storage_options are returned; other fields (version, table_uri, schema, stats) are null.
+Retrieves metadata for a Lance table. Only `load_detailed_metadata=false` is supported. When `load_detailed_metadata=false`, only the table location is returned; other fields (version, table_uri, schema, stats) are null.
 
 The implementation:
 
 1. Parse the table identifier
 2. Retrieve the Table object from HMS
 3. Validate that it is a Lance table (check `table_type=lance`)
-4. Return the table location from `storageDescriptor.location` and storage_options from `parameters`
+4. Return the table location from `storageDescriptor.location`
 
 **Error Handling:**
 
@@ -171,4 +183,8 @@ The implementation:
 
 **Error Handling:**
 
-If the table does not exist, return error code `4` (TableNotFound). If the table is not a Lance table, return error code `13` (InvalidInput). If the HMS connection fails, return error code `17` (ServiceUnavailable).
+If the table does not exist, return error code `4` (TableNotFound).
+
+If the table is not a Lance table, return error code `13` (InvalidInput).
+
+If the HMS connection fails, return error code `17` (ServiceUnavailable).