feat: add CreateTable REST storage options and properties and remove CreateEmptyTable (#330)

## Summary
- add `CreateTableRequest.storage_options` and map both `properties` and
`storage_options` through the REST `CreateTable` API as JSON-encoded
query parameters
- remove the deprecated `CreateEmptyTable` REST API, models, and
hand-written core interface methods in generated and source docs/code
- document the `CreateTable` REST translation more clearly, add `409
Conflict`, and fix the Arrow stream content type in the impl spec

Author: jackye1995

docs/src/client/operations/errors.md

@@ -71,7 +71,6 @@ All operations may return the following errors:
 | TableExists               | 1 (NamespaceNotFound), 4 (TableNotFound)                                                                                         |
 | DropTable                 | 1 (NamespaceNotFound), 4 (TableNotFound)                                                                                         |
 | DeregisterTable           | 1 (NamespaceNotFound), 4 (TableNotFound)                                                                                         |
-| CreateEmptyTable          | 1 (NamespaceNotFound), 5 (TableAlreadyExists), 14 (ConcurrentModification)                                                       |
 | RestoreTable              | 1 (NamespaceNotFound), 4 (TableNotFound), 11 (TableVersionNotFound), 14 (ConcurrentModification)                                 |
 | RenameTable               | 1 (NamespaceNotFound), 4 (TableNotFound), 5 (TableAlreadyExists), 14 (ConcurrentModification)                                    |
 | GetTableStats             | 1 (NamespaceNotFound), 4 (TableNotFound)                                                                                         |

docs/src/client/operations/index.md

@@ -61,7 +61,6 @@ See [REST Routes](../../rest/catalog-spec.md#rest-routes) for more details.
 | CountTableRows            | 1               |           | ✓     |       |          | ✓    |             |
 | CreateTable               | 1               |           | ✓     |       |          | ✓    |             |
 | DeclareTable              | 1               |           | ✓     |       | ✓        |      |             |
-| CreateEmptyTable          | 1 (deprecated)  |           | ✓     |       | ✓        |      |             |
 | CreateTableIndex          | 1               |           | ✓     | ✓     | ✓        |      |             |
 | CreateTableScalarIndex    | 1               |           | ✓     | ✓     | ✓        |      |             |
 | ListTableIndices          | 1               |           | ✓     | ✓     | ✓        |      |             |

docs/src/client/operations/models/CreateEmptyTableRequest.md

@@ -2,7 +2,7 @@
 
 # CreateTableRequest
 
-Request for creating a table, excluding the Arrow IPC stream. 
+Request for creating a table, excluding the Arrow IPC stream. The table location and any credential vending behavior are determined by the implementation and returned in the response, rather than specified in this request. 
 
 ## Properties
 
@@ -12,7 +12,8 @@ Request for creating a table, excluding the Arrow IPC stream.
 |**context** | **Map<String, String>** | Arbitrary context for a request as key-value pairs. How to use the context is custom to the specific implementation.  REST NAMESPACE ONLY Context entries are passed via HTTP headers using the naming convention `x-lance-ctx-<key>: <value>`. For example, a context entry `{\"trace_id\": \"abc123\"}` would be sent as the header `x-lance-ctx-trace_id: abc123`.  |  [optional] |
 |**id** | **List<String>** |  |  [optional] |
 |**mode** | **String** | There are three modes when trying to create a table, to differentiate the behavior when a table of the same name already exists. Case insensitive, supports both PascalCase and snake_case. Valid values are:   * Create: the operation fails with 409.   * ExistOk: the operation succeeds and the existing table is kept.   * Overwrite: the existing table is dropped and a new table with this name is created.  |  [optional] |
-|**properties** | **Map<String, String>** | Properties stored on the table, if supported by the implementation.  |  [optional] |
+|**properties** | **Map<String, String>** | Business logic properties stored and managed by the namespace implementation outside Lance context, if supported by the implementation.  |  [optional] |
+|**storageOptions** | **Map<String, String>** | Storage options that configure overrides for writing table data and metadata during table creation. These are passed to Lance for the write path.  |  [optional] |
 
 
 

docs/src/client/operations/models/CreateEmptyTableResponse.md

@@ -11,7 +11,7 @@
 |**location** | **String** |  |  [optional] |
 |**version** | **Long** |  |  [optional] |
 |**storageOptions** | **Map<String, String>** | Configuration options to be used to access storage. The available options depend on the type of storage in use. These will be passed directly to Lance to initialize storage access.  |  [optional] |
-|**properties** | **Map<String, String>** | If the implementation does not support table properties, it should return null for this field. Otherwise it should return the properties.  |  [optional] |
+|**properties** | **Map<String, String>** | Business logic properties stored and managed by the namespace implementation outside Lance context. If the implementation does not support table properties, it should return null for this field.  |  [optional] |
 
 
 

docs/src/client/operations/models/CreateTableRequest.md

@@ -13,7 +13,7 @@ Request for declaring a table.
 |**id** | **List<String>** |  |  [optional] |
 |**location** | **String** | Optional storage location for the table. If not provided, the namespace implementation should determine the table location.  |  [optional] |
 |**vendCredentials** | **Boolean** | Whether to include vended credentials in the response `storage_options`. When true, the implementation should provide vended credentials for accessing storage. When not set, the implementation can decide whether to return vended credentials.  |  [optional] |
-|**properties** | **Map<String, String>** | Properties stored on the table, if supported by the server.  |  [optional] |
+|**properties** | **Map<String, String>** | Business logic properties stored and managed by the namespace implementation outside Lance context, if supported by the implementation.  |  [optional] |
 
 
 

docs/src/client/operations/models/CreateTableResponse.md

@@ -1321,6 +1321,25 @@ paths:
         required: false
         schema:
           type: string
+      - name: "properties"
+        in: query
+        required: false
+        schema:
+          type: string
+        description: |
+          Business logic properties managed by the namespace implementation outside Lance context.
+          The map is translated to a single JSON-encoded query parameter such as
+          `properties={"user":"alice","team":"eng"}`.
+      - name: "storage_options"
+        in: query
+        required: false
+        schema:
+          type: string
+        description: |
+          Storage options that configure overrides for writing table data and metadata during
+          table creation. These are passed to Lance for the write path.
+          The map is translated to a single JSON-encoded query parameter such as
+          `storage_options={"aws_region":"us-east-1","timeout":"30s"}`.
     post:
       tags:
         - Table
@@ -1338,6 +1357,12 @@ paths:
         It passes in the `CreateTableRequest` information in the following way:
         - `id`: pass through path parameter of the same name
         - `mode`: pass through query parameter of the same name
+        - `properties`: serialize as a single JSON-encoded query parameter such as
+          `properties={"user":"alice","team":"eng"}`; these are business logic properties
+          managed by the namespace implementation outside Lance context
+        - `storage_options`: serialize as a single JSON-encoded query parameter such as
+          `storage_options={"aws_region":"us-east-1","timeout":"30s"}`; these configure
+          write-time overrides for data and metadata written during table creation
       requestBody:
         description: Arrow IPC data
         content:
@@ -1357,6 +1382,8 @@ paths:
           $ref: "#/components/responses/ForbiddenErrorResponse"
         404:
           $ref: "#/components/responses/NotFoundErrorResponse"
+        409:
+          $ref: "#/components/responses/ConflictErrorResponse"
         503:
           $ref: "#/components/responses/ServiceUnavailableErrorResponse"
         5XX:
@@ -1792,49 +1819,6 @@ paths:
         5XX:
           $ref: "#/components/responses/ServerErrorResponse"
 
-  /v1/table/{id}/create-empty:
-    parameters:
-      - $ref: "#/components/parameters/id"
-      - $ref: "#/components/parameters/delimiter"
-    post:
-      tags:
-        - Table
-        - Metadata
-      summary: Create an empty table
-      operationId: CreateEmptyTable
-      deprecated: true
-      description: |
-        Create an empty table with the given name without touching storage.
-        This is a metadata-only operation that records the table existence and sets up aspects like access control.
-
-        For DirectoryNamespace implementation, this creates a `.lance-reserved` file in the table directory
-        to mark the table's existence without creating actual Lance data files.
-
-        **Deprecated**: Use `DeclareTable` instead.
-      requestBody:
-        required: true
-        content:
-          application/json:
-            schema:
-              $ref: "#/components/schemas/CreateEmptyTableRequest"
-      responses:
-        200:
-          $ref: "#/components/responses/CreateEmptyTableResponse"
-        400:
-          $ref: "#/components/responses/BadRequestErrorResponse"
-        401:
-          $ref: "#/components/responses/UnauthorizedErrorResponse"
-        403:
-          $ref: "#/components/responses/ForbiddenErrorResponse"
-        404:
-          $ref: "#/components/responses/NotFoundErrorResponse"
-        409:
-          $ref: "#/components/responses/ConflictErrorResponse"
-        503:
-          $ref: "#/components/responses/ServiceUnavailableErrorResponse"
-        5XX:
-          $ref: "#/components/responses/ServerErrorResponse"
-
   /v1/table/{id}/tags/create:
     parameters:
       - $ref: "#/components/parameters/id"
@@ -3163,6 +3147,8 @@ components:
       type: object
       description: |
         Request for creating a table, excluding the Arrow IPC stream.
+        The table location and any credential vending behavior are determined by the implementation
+        and returned in the response, rather than specified in this request.
       properties:
         identity:
           $ref: "#/components/schemas/Identity"
@@ -3184,87 +3170,30 @@ components:
         properties:
           type: object
           description: |
-            Properties stored on the table, if supported by the implementation.
+            Business logic properties stored and managed by the namespace implementation outside
+            Lance context, if supported by the implementation.
           additionalProperties:
             type: string
-
-    CreateTableResponse:
-      type: object
-      properties:
-        transaction_id:
-          type: string
-          description: Optional transaction identifier
-        location:
-          type: string
-        version:
-          type: integer
-          format: int64
-          minimum: 0
         storage_options:
           type: object
           description: |
-            Configuration options to be used to access storage. The available
-            options depend on the type of storage in use. These will be
-            passed directly to Lance to initialize storage access.
+            Storage options that configure overrides for writing table data and metadata during
+            table creation. These are passed to Lance for the write path.
           additionalProperties:
             type: string
-        properties:
-          type: object
-          description: |
-            If the implementation does not support table properties, it should return null for this field. Otherwise it should return the properties.
-          additionalProperties:
-            type: string
-          example: { "owner": "Ralph", "created_at": "1452120468" }
-          default: {}
-          nullable: true
-
-    CreateEmptyTableRequest:
-      type: object
-      deprecated: true
-      description: |
-        Request for creating an empty table.
 
-        **Deprecated**: Use `DeclareTableRequest` instead.
-      properties:
-        identity:
-          $ref: "#/components/schemas/Identity"
-        context:
-          $ref: "#/components/schemas/Context"
-        id:
-          type: array
-          items:
-            type: string
-        location:
-          type: string
-          description: |
-            Optional storage location for the table.
-            If not provided, the namespace implementation should determine the table location.
-        vend_credentials:
-          description: |
-            Whether to include vended credentials in the response `storage_options`.
-            When true, the implementation should provide vended credentials for accessing storage.
-            When not set, the implementation can decide whether to return vended credentials.
-          type: boolean
-        properties:
-          type: object
-          description: |
-            Properties stored on the table, if supported by the server.
-          additionalProperties:
-            type: string
-
-    CreateEmptyTableResponse:
+    CreateTableResponse:
       type: object
-      deprecated: true
-      description: |
-        Response for creating an empty table.
-
-        **Deprecated**: Use `DeclareTableResponse` instead.
       properties:
         transaction_id:
           type: string
           description: Optional transaction identifier
         location:
           type: string
+        version:
+          type: integer
+          format: int64
+          minimum: 0
         storage_options:
           type: object
           description: |
@@ -3276,7 +3205,9 @@ components:
         properties:
           type: object
           description: |
-            If the implementation does not support table properties, it should return null for this field. Otherwise it should return the properties.
+            Business logic properties stored and managed by the namespace implementation outside
+            Lance context. If the implementation does not support table properties, it should
+            return null for this field.
           additionalProperties:
             type: string
           example: { "owner": "Ralph", "created_at": "1452120468" }
@@ -3310,7 +3241,8 @@ components:
         properties:
           type: object
           description: |
-            Properties stored on the table, if supported by the server.
+            Business logic properties stored and managed by the namespace implementation outside
+            Lance context, if supported by the implementation.
           additionalProperties:
             type: string
 
@@ -5205,13 +5137,6 @@ components:
           schema:
             $ref: "#/components/schemas/DeclareTableResponse"
 
-    CreateEmptyTableResponse:
-      description: Table properties result when creating an empty table
-      content:
-        application/json:
-          schema:
-            $ref: "#/components/schemas/CreateEmptyTableResponse"
-
     # Error Responses
 
     BadRequestErrorResponse:

docs/src/client/operations/models/DeclareTableRequest.md

@@ -654,22 +654,37 @@ Content-Type: application/json
 
 Creates a new table with initial data.
 
+For REST namespace, `CreateTableRequest` fields are passed as follows:
+
+- `id`: path parameter
+- `mode`: query parameter
+- `properties`: a single JSON-encoded query parameter such as
+  `properties={"user":"alice","team":"eng"}`; these are business logic properties managed
+  by the namespace implementation outside Lance context
+- `storage_options`: a single JSON-encoded query parameter such as
+  `storage_options={"aws_region":"us-east-1","timeout":"30s"}`; these configure write-time
+  overrides for data and metadata written during table creation
+
 **HTTP Request:**
 
 ```
 POST /v1/table/{id}/create
-Content-Type: application/json
+Content-Type: application/vnd.apache.arrow.stream
 ```
 
-### CreateEmptyTable
-
-Creates an empty table with a specified schema.
-
-**HTTP Request:**
+**Response:**
 
-```
-POST /v1/table/{id}/create-empty
-Content-Type: application/json
+```json
+{
+  "location": "s3://bucket/data/users.lance",
+  "version": 1,
+  "storage_options": {
+    "aws_region": "us-east-1"
+  },
+  "properties": {
+    "user": "alice"
+  }
+}
 ```
 
 ### GetTableStats