Skip to content

Commit 7375095

Browse files
jackye1995claude
andauthored
feat: add backfill columns and refresh materialized view operations (#335)
## Summary - Update `add_columns` and `alter_columns` schemas: remove `identity`, `context`, `id` from requests, remove `transaction_id` from responses, rename `NewColumnTransform` to `AddColumnsEntry`, adopt OpenAPI 3.1 nullable style - Add `POST /v1/table/{id}/backfill_column` (`AlterTableBackfillColumns`) — trigger async backfill job for a computed column, returns 202 with job_id - Add `POST /v1/table/{id}/refresh` (`RefreshMaterializedView`) — trigger async materialized view refresh, returns 202 with job_id - Add new operations to `LanceNamespace` interfaces (Python, Java sync, Java async) - Regenerate all clients (Rust, Python, Java) - Update operation docs and REST impl spec ## Test plan - [x] YAML validates successfully - [x] All `$ref` targets resolve - [x] Rust client compiles - [x] Java builds with mvnw - [x] Python imports verified --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent e470831 commit 7375095

155 files changed

Lines changed: 14082 additions & 3787 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/src/client/operations/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,9 @@ See [REST Routes](../../rest/catalog-spec.md#rest-routes) for more details.
7676
| AnalyzeTableQueryPlan | 1 | || | || |
7777
| AlterTableAddColumns | 1 | || | || |
7878
| AlterTableAlterColumns | 1 | || || | |
79+
| AlterTableBackfillColumns | 1 | || | || |
7980
| AlterTableDropColumns | 1 | || || | |
81+
| RefreshMaterializedView | 1 | || | || |
8082
| UpdateTableSchemaMetadata | 1 | || || | |
8183
| GetTableStats | 1 | || || | |
8284
| ListTableTags | 1 | || || | |
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
2+
3+
# AddColumnsEntry
4+
5+
6+
## Properties
7+
8+
| Name | Type | Description | Notes |
9+
|------------ | ------------- | ------------- | -------------|
10+
|**name** | **String** | Name of the new column | |
11+
|**expression** | **String** | SQL expression for the column (optional if virtual_column is specified) | [optional] |
12+
|**virtualColumn** | [**AddVirtualColumnEntry**](AddVirtualColumnEntry.md) | | [optional] |
13+
14+
15+

docs/src/client/operations/models/AlterColumnsEntry.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
|**dataType** | **Object** | New data type for the column using JSON representation (optional) | |
1212
|**rename** | **String** | New name for the column (optional) | [optional] |
1313
|**nullable** | **Boolean** | Whether the column should be nullable (optional) | [optional] |
14-
|**virtualColumn** | [**AlterVirtualColumnEntry**](AlterVirtualColumnEntry.md) | Virtual column alterations (optional) | [optional] |
14+
|**virtualColumn** | [**AlterVirtualColumnEntry**](AlterVirtualColumnEntry.md) | | [optional] |
1515

1616

1717

docs/src/client/operations/models/AlterTableAddColumnsRequest.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,7 @@
77

88
| Name | Type | Description | Notes |
99
|------------ | ------------- | ------------- | -------------|
10-
|**identity** | [**Identity**](Identity.md) | | [optional] |
11-
|**context** | **Map&lt;String, String&gt;** | Arbitrary context for a request as key-value pairs. How to use the context is custom to the specific implementation. REST NAMESPACE ONLY Context entries are passed via HTTP headers using the naming convention &#x60;x-lance-ctx-&lt;key&gt;: &lt;value&gt;&#x60;. For example, a context entry &#x60;{\&quot;trace_id\&quot;: \&quot;abc123\&quot;}&#x60; would be sent as the header &#x60;x-lance-ctx-trace_id: abc123&#x60;. | [optional] |
12-
|**id** | **List&lt;String&gt;** | | [optional] |
13-
|**newColumns** | [**List&lt;NewColumnTransform&gt;**](NewColumnTransform.md) | List of new columns to add | |
10+
|**newColumns** | [**List&lt;AddColumnsEntry&gt;**](AddColumnsEntry.md) | List of new columns to add to the table | |
1411

1512

1613

docs/src/client/operations/models/AlterTableAddColumnsResponse.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,7 @@
77

88
| Name | Type | Description | Notes |
99
|------------ | ------------- | ------------- | -------------|
10-
|**transactionId** | **String** | Optional transaction identifier | [optional] |
11-
|**version** | **Long** | Version of the table after adding columns | |
10+
|**version** | **Long** | The commit version associated with the operation | |
1211

1312

1413

docs/src/client/operations/models/AlterTableAlterColumnsRequest.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,7 @@
77

88
| Name | Type | Description | Notes |
99
|------------ | ------------- | ------------- | -------------|
10-
|**identity** | [**Identity**](Identity.md) | | [optional] |
11-
|**context** | **Map&lt;String, String&gt;** | Arbitrary context for a request as key-value pairs. How to use the context is custom to the specific implementation. REST NAMESPACE ONLY Context entries are passed via HTTP headers using the naming convention &#x60;x-lance-ctx-&lt;key&gt;: &lt;value&gt;&#x60;. For example, a context entry &#x60;{\&quot;trace_id\&quot;: \&quot;abc123\&quot;}&#x60; would be sent as the header &#x60;x-lance-ctx-trace_id: abc123&#x60;. | [optional] |
12-
|**id** | **List&lt;String&gt;** | | [optional] |
13-
|**alterations** | [**List&lt;AlterColumnsEntry&gt;**](AlterColumnsEntry.md) | List of column alterations to perform | |
10+
|**alterations** | [**List&lt;AlterColumnsEntry&gt;**](AlterColumnsEntry.md) | List of column alterations to apply to the table | |
1411

1512

1613

docs/src/client/operations/models/AlterTableAlterColumnsResponse.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,7 @@
77

88
| Name | Type | Description | Notes |
99
|------------ | ------------- | ------------- | -------------|
10-
|**transactionId** | **String** | Optional transaction identifier | [optional] |
11-
|**version** | **Long** | Version of the table after altering columns | |
10+
|**version** | **Long** | The commit version associated with the operation | |
1211

1312

1413

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
2+
3+
# AlterTableBackfillColumnsRequest
4+
5+
6+
## Properties
7+
8+
| Name | Type | Description | Notes |
9+
|------------ | ------------- | ------------- | -------------|
10+
|**column** | **String** | Column name to backfill | |
11+
|**where** | **String** | Optional WHERE clause filter | [optional] |
12+
|**concurrency** | **Integer** | Optional concurrency override | [optional] |
13+
|**intraApplierConcurrency** | **Integer** | Optional intra-applier concurrency override | [optional] |
14+
|**minCheckpointSize** | **Integer** | Optional minimum checkpoint size | [optional] |
15+
|**maxCheckpointSize** | **Integer** | Optional maximum checkpoint size | [optional] |
16+
|**batchCheckpointFlushIntervalSeconds** | **BigDecimal** | Optional batch checkpoint flush interval in seconds | [optional] |
17+
|**readVersion** | **Integer** | Optional table version to read from | [optional] |
18+
|**taskSize** | **Integer** | Optional task size | [optional] |
19+
|**numFrags** | **Integer** | Optional number of fragments | [optional] |
20+
|**checkpointSize** | **Integer** | Optional checkpoint size | [optional] |
21+
|**commitGranularity** | **Integer** | Optional commit granularity | [optional] |
22+
|**cluster** | **String** | Optional cluster name | [optional] |
23+
|**manifest** | **String** | Optional manifest name | [optional] |
24+
25+
26+
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
2+
3+
# AlterTableBackfillColumnsResponse
4+
5+
6+
## Properties
7+
8+
| Name | Type | Description | Notes |
9+
|------------ | ------------- | ------------- | -------------|
10+
|**jobId** | **String** | The job ID for tracking the backfill job | |
11+
12+
13+

docs/src/client/operations/models/NewColumnTransform.md

Lines changed: 0 additions & 15 deletions
This file was deleted.

0 commit comments

Comments
 (0)