Merge pull request #84 from hotdata-dev/openapi-update-28839841740

feat: async table loads + database-scoped results/query runs

Author: zfarrell

CHANGELOG.md

@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed
+
+- feat: support async table loads and append mode
+- **Breaking:** results and query runs are now scoped to a database via the
+  required `X-Database-Id` header. The ergonomic wrappers gain a `database_id`
+  argument to match: `Client::get_result`, `Client::list_results`,
+  `Client::list_query_runs`, `Client::await_result`, `Client::get_result_arrow`,
+  `Client::stream_result_arrow`, `Client::query_to_arrow`, and the
+  `results()` / `query_runs()` resource handles. `Client::query`'s truncation
+  auto-follow now forwards the query's database scope (the `X-Database-Id`
+  header, or the request-body `database_id` when no header is set) to the
+  follow-up result and query-run fetches.
+
 ## [0.7.0] - 2026-06-30
 
 ### Added

README.md

@@ -64,16 +64,22 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
         // .base_url("https://api.hotdata.dev")  // optional
         .build()?;
 
-    // Submit a query. `query` transparently retries HTTP 429 (server overload)
+    // Queries, results, and query runs are scoped to a database (the required
+    // `X-Database-Id` header), so pick one first.
+    let database_id = "your_database_id";
+
+    // Submit a query. `query_in` transparently retries HTTP 429 (server overload)
     // and auto-follows a truncated large result to its full row set. Rows come
     // back inline, plus a result_id persisted for later retrieval.
     let response = client
-        .query(QueryRequest::new("SELECT 1 AS n".to_string()))
+        .query_in(QueryRequest::new("SELECT 1 AS n".to_string()), database_id)
         .await?;
 
     if let Some(result_id) = response.result_id.flatten() {
         // Poll the persisted result to `ready` without hand-rolling a loop.
-        let result = client.await_result(&result_id, PollConfig::default()).await?;
+        let result = client
+            .await_result(&result_id, database_id, PollConfig::default())
+            .await?;
         println!("result {} is {}", result.result_id, result.status);
     }
 
@@ -91,7 +97,8 @@ ergonomic, workspace-scoped handles so you never pass a `Configuration` around:
 let connections = client.connections().list().await?;
 let connection  = client.connections().get(&connections.connections[0].id).await?;
 let secrets  = client.secrets().list().await?;
-let runs     = client.query_runs().list(Some(50), None, None, None).await?;
+// Query runs are database-scoped: pass the database id first.
+let runs     = client.query_runs().list(database_id, Some(50), None, None, None).await?;
 ```
 
 Handles exist for every resource — `connections`, `connection_types`,
@@ -101,7 +108,9 @@ Handles exist for every resource — `connections`, `connection_types`,
 operations also have flat shortcuts directly on `Client` (`query` — with
 `query_in` to scope to a database, `query_preview` to skip auto-follow, and
 `query_with` for a per-call `QueryConfig` — plus `get_result`, `list_results`,
-`list_query_runs`, `list_workspaces`).
+`list_query_runs`, `list_workspaces`). The result and query-run operations take
+a `database_id` argument — they are scoped to a database via the required
+`X-Database-Id` header.
 
 For anything not yet wrapped, the full generated surface is one call away via
 `client.configuration()`:
@@ -120,12 +129,12 @@ them with the typed [`ResultStatus`] / [`QueryRunStatus`] enums via the
 ```rust
 use hotdata::prelude::*;
 
-let result = client.await_result(&result_id, PollConfig::default()).await?;
+let result = client.await_result(&result_id, database_id, PollConfig::default()).await?;
 if result.result_status().is_ready() {
     // ...
 }
 
-let run = client.query_runs().get(&query_run_id).await?;
+let run = client.query_runs().get(&query_run_id, database_id).await?;
 if run.run_status().is_terminal() { /* ... */ }
 ```
 
@@ -183,16 +192,19 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
         .workspace_id("your_workspace_id")
         .build()?;
 
+    // Results are database-scoped (the required `X-Database-Id` header).
+    let database_id = "your_database_id";
+
     // Buffered: decodes every batch into a Vec<RecordBatch>.
-    let result = client.get_result_arrow(&result_id, None, None).await?;
+    let result = client.get_result_arrow(&result_id, database_id, None, None).await?;
     println!("schema: {:?}", result.schema);
     println!("total rows: {:?}", result.total_row_count);
     for batch in &result.batches {
         // work with each arrow_array::RecordBatch
     }
 
     // Streaming: yields batches lazily without holding them all at once.
-    let mut stream = client.stream_result_arrow(&result_id, None, None).await?;
+    let mut stream = client.stream_result_arrow(&result_id, database_id, None, None).await?;
     for batch in stream.by_ref() {
         let batch = batch?;
         // ...
@@ -202,7 +214,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
 }
 ```
 
-Both methods accept `offset` and `limit` for pagination, and both honor the transparent JWT exchange. They return `ArrowError::NotReady` if the result is still pending or processing — poll `client.get_result(result_id)` until its status is `ready` first. `ArrowResult` also surfaces the `X-Total-Row-Count` header (`total_row_count`) and the `rel="next"` pagination `Link` (`next_link`).
+Both methods accept `offset` and `limit` for pagination, and both honor the transparent JWT exchange. They return `ArrowError::NotReady` if the result is still pending or processing — poll `client.get_result(result_id, database_id)` until its status is `ready` first. `ArrowResult` also surfaces the `X-Total-Row-Count` header (`total_row_count`) and the `rel="next"` pagination `Link` (`next_link`).
 
 To run a query and get its result as Arrow in a single call — submit, await
 `ready`, and decode — use `query_to_arrow`:
@@ -211,6 +223,7 @@ To run a query and get its result as Arrow in a single call — submit, await
 let arrow = client
     .query_to_arrow(
         QueryRequest::new("SELECT * FROM big_table".to_string()),
+        database_id,
         PollConfig::default(),
         None, // offset
         None, // limit

docs/AsyncQueryResponse.md

@@ -7,7 +7,7 @@ Name | Type | Description | Notes
 **query_run_id** | **String** | Unique identifier for the query run. | 
 **reason** | Option<**String**> | Human-readable reason why the query went async (e.g., caching tables for the first time). | [optional]
 **status** | **String** | Current status of the query run. | 
-**status_url** | **String** | URL to poll for query run status. | 
+**status_url** | **String** | URL to poll for query run status. Requires the same `X-Database-Id` header used to submit the query. | 
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)
 

docs/ConnectionsApi.md

@@ -298,7 +298,7 @@ This endpoint does not need any parameter.
 > models::LoadManagedTableResponse load_managed_table(connection_id, schema, table, load_managed_table_request)
 Load managed table from upload
 
-Publish a previously-uploaded file as the new contents of a managed table. CSV, JSON, and Parquet uploads are supported; the format is auto-detected from the upload's `Content-Type` and file contents, or set explicitly via the `format` field. Only `mode = \"replace\"` is supported. Concurrent loads against the same upload return 409.
+Publish a previously-uploaded file as the new contents of a managed table. CSV, JSON, and Parquet uploads are supported; the format is auto-detected from the upload's `Content-Type` and file contents, or set explicitly via the `format` field. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the upload is applied: `replace` overwrites the table's contents, `append` inserts the uploaded rows on top of the existing data. Concurrent loads against the same upload return 409. Set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID.
 
 ### Parameters
 

docs/CreateIndexRequest.md

@@ -5,6 +5,7 @@
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **r#async** | Option<**bool**> | When true, create the index as a background job and return a job ID for polling. | [optional]
+**async_after_ms** | Option<**i32**> | If set (requires `async` = true), wait up to this many milliseconds for the index build to finish: if it completes in time the index is returned (201), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional]
 **columns** | **Vec<String>** | Columns to index. Required for all index types. | 
 **description** | Option<**String**> | User-facing description of the embedding (e.g., \"product descriptions\"). | [optional]
 **dimensions** | Option<**i32**> | Output vector dimensions. Some models support multiple dimension sizes (e.g., OpenAI text-embedding-3-small supports 512 or 1536). If omitted, the model's default dimensions are used | [optional]

docs/DatabasesApi.md

@@ -259,7 +259,7 @@ This endpoint does not need any parameter.
 > models::LoadManagedTableResponse load_database_table(database_id, schema, table, load_managed_table_request)
 Load database table from upload
 
-Publish a previously-uploaded file as the new contents of a table on the database's default catalog. The database-scoped equivalent of the connection-scoped managed-table load — addressed by `database_id`, so no `default_connection_id` is needed. CSV, JSON, and Parquet uploads are supported; the format is auto-detected or set via `format`. Only `mode = \"replace\"` is supported. Concurrent loads against the same upload return 409.
+Publish a previously-uploaded file as the new contents of a table on the database's default catalog. The database-scoped equivalent of the connection-scoped managed-table load — addressed by `database_id`, so no `default_connection_id` is needed. CSV, JSON, and Parquet uploads are supported; the format is auto-detected or set via `format`. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the upload is applied: `replace` overwrites the table's contents, `append` inserts the uploaded rows on top of the existing data. Concurrent loads against the same upload return 409. Set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID.
 
 ### Parameters
 

docs/JobResult.md

@@ -6,6 +6,7 @@
 |---- | -----|
 | ConnectionRefreshResult | Job-specific result payload. The shape depends on the job type. Null while the job is pending or running. |
 | IndexInfoResponse | Job-specific result payload. The shape depends on the job type. Null while the job is pending or running. |
+| LoadManagedTableResponse | Job-specific result payload. The shape depends on the job type. Null while the job is pending or running. |
 | TableRefreshResult | Job-specific result payload. The shape depends on the job type. Null while the job is pending or running. |
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

docs/LoadManagedTableRequest.md

@@ -4,8 +4,10 @@
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
+**r#async** | Option<**bool**> | When true, run the load as a background job and return a job ID to poll instead of blocking until it finishes. Recommended for large uploads, which can take longer than an HTTP request should stay open. | [optional]
+**async_after_ms** | Option<**i32**> | If set (requires `async` = true), wait up to this many milliseconds for the load to finish: if it completes in time the full result is returned (200), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional]
 **format** | Option<**String**> | File format of the upload: `\"csv\"`, `\"json\"`, or `\"parquet\"`. Optional — when omitted, the format is auto-detected from the upload's `Content-Type` and, failing that, from the file contents. Provide it explicitly to override detection or when the contents are ambiguous. `\"json\"` expects newline-delimited JSON (one object per line), not a JSON array. | [optional]
-**mode** | **String** | Load mode. Only `\"replace\"` is supported in this release. | 
+**mode** | **String** | How the upload is applied: `\"replace\"` overwrites the table's contents, `\"append\"` inserts the uploaded rows on top of the existing data. | 
 **upload_id** | **String** | ID of a previously-staged upload (see `POST /v1/files`). The upload is claimed atomically; concurrent loads against the same `upload_id` return 409. | 
 
 [[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)

docs/LoadManagedTableResponse.md

@@ -6,7 +6,7 @@ Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **arrow_schema_json** | **String** | Schema of the loaded table, as JSON. | 
 **connection_id** | **String** |  | 
-**row_count** | **i64** | Total rows in the published parquet file. | 
+**row_count** | **i64** | Total number of rows in the table after the load. | 
 **schema_name** | **String** |  | 
 **table_name** | **String** |  | 
 

docs/QueryRequest.md

@@ -5,7 +5,7 @@
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **r#async** | Option<**bool**> | When true, execute the query asynchronously and return a query run ID for polling via GET /query-runs/{id}. The query results can be retrieved via GET /results/{id} once the query run status is \"succeeded\". | [optional]
-**async_after_ms** | Option<**i32**> | If set with async=true, wait up to this many milliseconds for the query to complete synchronously before returning an async response. Minimum 1000ms. Ignored if async is false. | [optional]
+**async_after_ms** | Option<**i32**> | If set (requires `async` = true), first attempt the query synchronously and wait up to this many milliseconds: if it finishes in time the full result is returned, otherwise an async response (a run id to poll) is returned. Must be at least 1000 and at most the server's configured maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional]
 **database_id** | Option<**String**> | Database to scope the query to (its id). Alternative to the `X-Database-Id` header — exactly one source must be provided. If both this field and the header are set and they disagree, the request is rejected with a 400. | [optional]
 **default_catalog** | Option<**String**> | Catalog that unqualified table references resolve against within the query's database scope. Must name a catalog visible in the database (`default`, an attached catalog alias, or a system catalog). Defaults to `default` when omitted. | [optional]
 **default_schema** | Option<**String**> | Schema that unqualified table references resolve against within the query's database scope. Defaults to `main` when omitted. Existence is not validated up front — an unknown schema surfaces as a \"table not found\" error at planning time. | [optional]