diff --git a/docs/CreateDatabaseRequest.md b/docs/CreateDatabaseRequest.md index c4759d2..cd8c89b 100644 --- a/docs/CreateDatabaseRequest.md +++ b/docs/CreateDatabaseRequest.md @@ -6,9 +6,10 @@ Request body for POST /databases Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- -**expires_at** | **str** | When this database expires. Accepts either an RFC 3339 timestamp (e.g. `\"2026-06-01T00:00:00Z\"`) or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days) — for example `\"24h\"`, `\"48h\"`, or `\"7d\"`. Defaults to `\"24h\"` when omitted. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. | [optional] +**expires_at** | **str** | When this database expires. Accepts either an RFC 3339 timestamp (e.g. `\"2026-06-01T00:00:00Z\"`) or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days) — for example `\"24h\"`, `\"48h\"`, or `\"7d\"`. Omitted (or empty) means the database never expires. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. | [optional] **name** | **str** | Optional free-form display label (for UIs/CLIs). Not unique. Not an identifier — databases are always addressed by `id`. Accepts the legacy `description` key as an alias so clients that predate the rename keep populating this field. | [optional] **schemas** | [**List[DatabaseDefaultSchemaDecl]**](DatabaseDefaultSchemaDecl.md) | Optional schemas/tables to declare on the database's auto-created `default` catalog. Mirrors the `config.schemas` field of a managed `POST /v1/connections`. Tables declared here can be loaded via the standard managed-table load endpoint targeting `default_connection_id`. Omitted or empty means the default catalog starts empty. | [optional] +**storage_backend** | **str** | Physical storage backend for the database's auto-created `default` catalog. `\"parquet\"` (default) uses the versioned parquet cache. `\"ducklake\"` stores data in a DuckLake catalog in the shared metadata DB configured via `ducklake.metadata_pg_url`, which must be configured for that value to be accepted. Omitted means `\"parquet\"`. | [optional] ## Example diff --git a/docs/DatabasesApi.md b/docs/DatabasesApi.md index b2ffa65..b141f59 100644 --- a/docs/DatabasesApi.md +++ b/docs/DatabasesApi.md @@ -105,7 +105,7 @@ void (empty response body) Create database -Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. Defaults to `24h` when omitted. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. +Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. When omitted, the database never expires. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. Optional `storage_backend` selects the physical backend for the default catalog — `parquet` (default) or `ducklake` (requires `ducklake.metadata_pg_url` to be configured). ### Example diff --git a/docs/DatasetsApi.md b/docs/DatasetsApi.md index 412eaee..566e49f 100644 --- a/docs/DatasetsApi.md +++ b/docs/DatasetsApi.md @@ -13,11 +13,13 @@ Method | HTTP request | Description # **create_dataset** -> CreateDatasetResponse create_dataset(create_dataset_request) +> CreateDatasetResponse create_dataset(create_dataset_request, x_database_id=x_database_id) Create dataset -Create a new dataset from an uploaded file or inline data. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. +Create a new dataset from an uploaded file, inline data, a URL, or a SQL/saved query. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. + +For `sql_query` / `saved_query` sources the dataset materializes by running that SQL, so the `X-Database-Id` header is required and the query sees only that database's catalogs (the scope is also reused on refresh). Upload/url/inline sources ignore the header. ### Example @@ -65,10 +67,11 @@ with hotdata.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = hotdata.DatasetsApi(api_client) create_dataset_request = hotdata.CreateDatasetRequest() # CreateDatasetRequest | + x_database_id = 'x_database_id_example' # str | Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404. (optional) try: # Create dataset - api_response = api_instance.create_dataset(create_dataset_request) + api_response = api_instance.create_dataset(create_dataset_request, x_database_id=x_database_id) print("The response of DatasetsApi->create_dataset:\n") pprint(api_response) except Exception as e: @@ -83,6 +86,7 @@ with hotdata.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **create_dataset_request** | [**CreateDatasetRequest**](CreateDatasetRequest.md)| | + **x_database_id** | **str**| Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404. | [optional] ### Return type @@ -102,7 +106,8 @@ Name | Type | Description | Notes | Status code | Description | Response headers | |-------------|-------------|------------------| **201** | Dataset created | - | -**400** | Invalid request | - | +**400** | Invalid request (query-backed dataset without X-Database-Id) | - | +**404** | Database (from X-Database-Id) not found | - | **409** | Dataset already exists | - | [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) diff --git a/docs/QueryApi.md b/docs/QueryApi.md index 60e285e..fad7fc1 100644 --- a/docs/QueryApi.md +++ b/docs/QueryApi.md @@ -12,12 +12,10 @@ Method | HTTP request | Description Execute SQL query -Execute a SQL query against all registered connections and datasets. Use standard Postgres-compatible SQL to reference tables from any connection using the format `connection_name.schema.table`. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. +Execute a SQL query scoped to a database. A database is the only window into catalogs: the query sees only that database's auto `default` catalog plus any catalogs explicitly attached to it. Select the database with EITHER the `X-Database-Id` header OR the `database_id` body field (exactly one must be given; if both are sent and disagree, that's a 400). Use standard Postgres-compatible SQL; reference the default catalog as `default..
` (or just `.
` / `
`) and attached catalogs by their alias. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. -Set the `X-Database-Id` header to scope the query to a specific database. Inside a database scope the query only sees that database's auto `default` catalog plus any catalogs explicitly attached to it; workspace catalogs are invisible. - ### Example * Api Key Authentication (WorkspaceId): @@ -64,7 +62,7 @@ with hotdata.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = hotdata.QueryApi(api_client) query_request = hotdata.QueryRequest() # QueryRequest | - x_database_id = 'x_database_id_example' # str | Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. (optional) + x_database_id = 'x_database_id_example' # str | Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. (optional) try: # Execute SQL query @@ -83,7 +81,7 @@ with hotdata.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **query_request** | [**QueryRequest**](QueryRequest.md)| | - **x_database_id** | **str**| Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. | [optional] + **x_database_id** | **str**| Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. | [optional] ### Return type @@ -104,8 +102,8 @@ Name | Type | Description | Notes |-------------|-------------|------------------| **200** | Query executed successfully | - | **202** | Query submitted asynchronously | - | -**400** | Invalid request | - | -**404** | Database (from X-Database-Id) not found | - | +**400** | Invalid request (no database specified, or header/body database_id conflict) | - | +**404** | Database not found | - | **500** | Internal server error | - | [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) diff --git a/docs/QueryRequest.md b/docs/QueryRequest.md index 00ed64e..6853b86 100644 --- a/docs/QueryRequest.md +++ b/docs/QueryRequest.md @@ -8,8 +8,9 @@ Name | Type | Description | Notes ------------ | ------------- | ------------- | ------------- **var_async** | **bool** | When true, execute the query asynchronously and return a query run ID for polling via GET /query-runs/{id}. The query results can be retrieved via GET /results/{id} once the query run status is \"succeeded\". | [optional] **async_after_ms** | **int** | If set with async=true, wait up to this many milliseconds for the query to complete synchronously before returning an async response. Minimum 1000ms. Ignored if async is false. | [optional] -**default_catalog** | **str** | Catalog that unqualified table references resolve against. Only honored inside an `X-Database-Id` scope; sending it without that header is a 400. Must name a catalog visible in the database (`default`, an attached catalog alias, or a system catalog). Defaults to `default` when omitted. | [optional] -**default_schema** | **str** | Schema that unqualified table references resolve against. Only honored inside an `X-Database-Id` scope; sending it without that header is a 400. Defaults to `main` when omitted. Existence is not validated up front — an unknown schema surfaces as a \"table not found\" error at planning time. | [optional] +**database_id** | **str** | Database to scope the query to (its id). Alternative to the `X-Database-Id` header — exactly one source must be provided. If both this field and the header are set and they disagree, the request is rejected with a 400. | [optional] +**default_catalog** | **str** | Catalog that unqualified table references resolve against within the query's database scope. Must name a catalog visible in the database (`default`, an attached catalog alias, or a system catalog). Defaults to `default` when omitted. | [optional] +**default_schema** | **str** | Schema that unqualified table references resolve against within the query's database scope. Defaults to `main` when omitted. Existence is not validated up front — an unknown schema surfaces as a \"table not found\" error at planning time. | [optional] **sql** | **str** | | ## Example diff --git a/docs/SavedQueriesApi.md b/docs/SavedQueriesApi.md index add490a..3db5782 100644 --- a/docs/SavedQueriesApi.md +++ b/docs/SavedQueriesApi.md @@ -182,11 +182,11 @@ void (empty response body) [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) # **execute_saved_query** -> QueryResponse execute_saved_query(id, execute_saved_query_request=execute_saved_query_request) +> QueryResponse execute_saved_query(id, x_database_id, execute_saved_query_request=execute_saved_query_request) Execute saved query -Execute a saved query. By default runs the latest version. Optionally specify a version number to execute a previous version. Returns the same response format as POST /v1/query. +Execute a saved query, scoped to a database (required `X-Database-Id` header). By default runs the latest version. Optionally specify a version number to execute a previous version. The SQL runs inside the given database scope, the same way POST /v1/query does. Returns the same response format as POST /v1/query. ### Example @@ -227,11 +227,12 @@ with hotdata.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = hotdata.SavedQueriesApi(api_client) id = 'id_example' # str | Saved query ID + x_database_id = 'x_database_id_example' # str | Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404. execute_saved_query_request = hotdata.ExecuteSavedQueryRequest() # ExecuteSavedQueryRequest | Optional version to execute (optional) try: # Execute saved query - api_response = api_instance.execute_saved_query(id, execute_saved_query_request=execute_saved_query_request) + api_response = api_instance.execute_saved_query(id, x_database_id, execute_saved_query_request=execute_saved_query_request) print("The response of SavedQueriesApi->execute_saved_query:\n") pprint(api_response) except Exception as e: @@ -246,6 +247,7 @@ with hotdata.ApiClient(configuration) as api_client: Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **id** | **str**| Saved query ID | + **x_database_id** | **str**| Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404. | **execute_saved_query_request** | [**ExecuteSavedQueryRequest**](ExecuteSavedQueryRequest.md)| Optional version to execute | [optional] ### Return type @@ -266,7 +268,8 @@ Name | Type | Description | Notes | Status code | Description | Response headers | |-------------|-------------|------------------| **200** | Query executed | - | -**404** | Saved query not found | - | +**400** | Invalid request (including a missing X-Database-Id header) | - | +**404** | Saved query or database not found | - | [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md) diff --git a/hotdata/api/databases_api.py b/hotdata/api/databases_api.py index b27b407..fc316de 100644 --- a/hotdata/api/databases_api.py +++ b/hotdata/api/databases_api.py @@ -360,7 +360,7 @@ def create_database( ) -> CreateDatabaseResponse: """Create database - Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. Defaults to `24h` when omitted. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. + Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. When omitted, the database never expires. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. Optional `storage_backend` selects the physical backend for the default catalog — `parquet` (default) or `ducklake` (requires `ducklake.metadata_pg_url` to be configured). :param create_database_request: (required) :type create_database_request: CreateDatabaseRequest @@ -429,7 +429,7 @@ def create_database_with_http_info( ) -> ApiResponse[CreateDatabaseResponse]: """Create database - Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. Defaults to `24h` when omitted. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. + Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. When omitted, the database never expires. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. Optional `storage_backend` selects the physical backend for the default catalog — `parquet` (default) or `ducklake` (requires `ducklake.metadata_pg_url` to be configured). :param create_database_request: (required) :type create_database_request: CreateDatabaseRequest @@ -498,7 +498,7 @@ def create_database_without_preload_content( ) -> RESTResponseType: """Create database - Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. Defaults to `24h` when omitted. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. + Create a new database (a metadata-only grouping). A managed default catalog is auto-created and addressable inside the database as `default`, with a `main` schema pre-declared so `default.main.
` works out of the box. The optional `name` is a free-form display label and is not required to be unique. Optional `schemas` declares additional schemas/tables on the default catalog at create time; declared tables can be loaded via the standard managed-tables-load endpoint targeting `default_connection_id`. Optional `expires_at` sets when the database expires — accepts either an RFC 3339 timestamp or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days), e.g. `24h`, `48h`, `90m`, `7d`. When omitted, the database never expires. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp. Optional `storage_backend` selects the physical backend for the default catalog — `parquet` (default) or `ducklake` (requires `ducklake.metadata_pg_url` to be configured). :param create_database_request: (required) :type create_database_request: CreateDatabaseRequest diff --git a/hotdata/api/datasets_api.py b/hotdata/api/datasets_api.py index 2c3545b..cccf150 100644 --- a/hotdata/api/datasets_api.py +++ b/hotdata/api/datasets_api.py @@ -49,6 +49,7 @@ def __init__(self, api_client=None) -> None: def create_dataset( self, create_dataset_request: CreateDatasetRequest, + x_database_id: Annotated[Optional[StrictStr], Field(description="Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404.")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -64,10 +65,12 @@ def create_dataset( ) -> CreateDatasetResponse: """Create dataset - Create a new dataset from an uploaded file or inline data. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. + Create a new dataset from an uploaded file, inline data, a URL, or a SQL/saved query. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. For `sql_query` / `saved_query` sources the dataset materializes by running that SQL, so the `X-Database-Id` header is required and the query sees only that database's catalogs (the scope is also reused on refresh). Upload/url/inline sources ignore the header. :param create_dataset_request: (required) :type create_dataset_request: CreateDatasetRequest + :param x_database_id: Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404. + :type x_database_id: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -92,6 +95,7 @@ def create_dataset( _param = self._create_dataset_serialize( create_dataset_request=create_dataset_request, + x_database_id=x_database_id, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -101,6 +105,7 @@ def create_dataset( _response_types_map: Dict[str, Optional[str]] = { '201': "CreateDatasetResponse", '400': "ApiErrorResponse", + '404': "ApiErrorResponse", '409': "ApiErrorResponse", } response_data = self.api_client.call_api( @@ -118,6 +123,7 @@ def create_dataset( def create_dataset_with_http_info( self, create_dataset_request: CreateDatasetRequest, + x_database_id: Annotated[Optional[StrictStr], Field(description="Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404.")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -133,10 +139,12 @@ def create_dataset_with_http_info( ) -> ApiResponse[CreateDatasetResponse]: """Create dataset - Create a new dataset from an uploaded file or inline data. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. + Create a new dataset from an uploaded file, inline data, a URL, or a SQL/saved query. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. For `sql_query` / `saved_query` sources the dataset materializes by running that SQL, so the `X-Database-Id` header is required and the query sees only that database's catalogs (the scope is also reused on refresh). Upload/url/inline sources ignore the header. :param create_dataset_request: (required) :type create_dataset_request: CreateDatasetRequest + :param x_database_id: Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404. + :type x_database_id: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -161,6 +169,7 @@ def create_dataset_with_http_info( _param = self._create_dataset_serialize( create_dataset_request=create_dataset_request, + x_database_id=x_database_id, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -170,6 +179,7 @@ def create_dataset_with_http_info( _response_types_map: Dict[str, Optional[str]] = { '201': "CreateDatasetResponse", '400': "ApiErrorResponse", + '404': "ApiErrorResponse", '409': "ApiErrorResponse", } response_data = self.api_client.call_api( @@ -187,6 +197,7 @@ def create_dataset_with_http_info( def create_dataset_without_preload_content( self, create_dataset_request: CreateDatasetRequest, + x_database_id: Annotated[Optional[StrictStr], Field(description="Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404.")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -202,10 +213,12 @@ def create_dataset_without_preload_content( ) -> RESTResponseType: """Create dataset - Create a new dataset from an uploaded file or inline data. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. + Create a new dataset from an uploaded file, inline data, a URL, or a SQL/saved query. The dataset becomes a queryable table under the `datasets` schema (e.g., `SELECT * FROM datasets.my_table`). Supports CSV, JSON, and Parquet formats. Optionally specify explicit column types. For `sql_query` / `saved_query` sources the dataset materializes by running that SQL, so the `X-Database-Id` header is required and the query sees only that database's catalogs (the scope is also reused on refresh). Upload/url/inline sources ignore the header. :param create_dataset_request: (required) :type create_dataset_request: CreateDatasetRequest + :param x_database_id: Required for query-backed datasets (sql_query / saved_query): the database whose catalogs the materializing query runs against. An unknown id is a 404. + :type x_database_id: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request timeout. It can also be a pair (tuple) of @@ -230,6 +243,7 @@ def create_dataset_without_preload_content( _param = self._create_dataset_serialize( create_dataset_request=create_dataset_request, + x_database_id=x_database_id, _request_auth=_request_auth, _content_type=_content_type, _headers=_headers, @@ -239,6 +253,7 @@ def create_dataset_without_preload_content( _response_types_map: Dict[str, Optional[str]] = { '201': "CreateDatasetResponse", '400': "ApiErrorResponse", + '404': "ApiErrorResponse", '409': "ApiErrorResponse", } response_data = self.api_client.call_api( @@ -251,6 +266,7 @@ def create_dataset_without_preload_content( def _create_dataset_serialize( self, create_dataset_request, + x_database_id, _request_auth, _content_type, _headers, @@ -274,6 +290,8 @@ def _create_dataset_serialize( # process the path parameters # process the query parameters # process the header parameters + if x_database_id is not None: + _header_params['X-Database-Id'] = x_database_id # process the form parameters # process the body parameter if create_dataset_request is not None: diff --git a/hotdata/api/query_api.py b/hotdata/api/query_api.py index ad61416..6500446 100644 --- a/hotdata/api/query_api.py +++ b/hotdata/api/query_api.py @@ -44,7 +44,7 @@ def __init__(self, api_client=None) -> None: def query( self, query_request: QueryRequest, - x_database_id: Annotated[Optional[StrictStr], Field(description="Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404.")] = None, + x_database_id: Annotated[Optional[StrictStr], Field(description="Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404.")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -60,11 +60,11 @@ def query( ) -> QueryResponse: """Execute SQL query - Execute a SQL query against all registered connections and datasets. Use standard Postgres-compatible SQL to reference tables from any connection using the format `connection_name.schema.table`. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. Set the `X-Database-Id` header to scope the query to a specific database. Inside a database scope the query only sees that database's auto `default` catalog plus any catalogs explicitly attached to it; workspace catalogs are invisible. + Execute a SQL query scoped to a database. A database is the only window into catalogs: the query sees only that database's auto `default` catalog plus any catalogs explicitly attached to it. Select the database with EITHER the `X-Database-Id` header OR the `database_id` body field (exactly one must be given; if both are sent and disagree, that's a 400). Use standard Postgres-compatible SQL; reference the default catalog as `default..
` (or just `.
` / `
`) and attached catalogs by their alias. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. :param query_request: (required) :type query_request: QueryRequest - :param x_database_id: Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. + :param x_database_id: Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. :type x_database_id: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request @@ -119,7 +119,7 @@ def query( def query_with_http_info( self, query_request: QueryRequest, - x_database_id: Annotated[Optional[StrictStr], Field(description="Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404.")] = None, + x_database_id: Annotated[Optional[StrictStr], Field(description="Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404.")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -135,11 +135,11 @@ def query_with_http_info( ) -> ApiResponse[QueryResponse]: """Execute SQL query - Execute a SQL query against all registered connections and datasets. Use standard Postgres-compatible SQL to reference tables from any connection using the format `connection_name.schema.table`. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. Set the `X-Database-Id` header to scope the query to a specific database. Inside a database scope the query only sees that database's auto `default` catalog plus any catalogs explicitly attached to it; workspace catalogs are invisible. + Execute a SQL query scoped to a database. A database is the only window into catalogs: the query sees only that database's auto `default` catalog plus any catalogs explicitly attached to it. Select the database with EITHER the `X-Database-Id` header OR the `database_id` body field (exactly one must be given; if both are sent and disagree, that's a 400). Use standard Postgres-compatible SQL; reference the default catalog as `default..
` (or just `.
` / `
`) and attached catalogs by their alias. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. :param query_request: (required) :type query_request: QueryRequest - :param x_database_id: Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. + :param x_database_id: Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. :type x_database_id: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request @@ -194,7 +194,7 @@ def query_with_http_info( def query_without_preload_content( self, query_request: QueryRequest, - x_database_id: Annotated[Optional[StrictStr], Field(description="Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404.")] = None, + x_database_id: Annotated[Optional[StrictStr], Field(description="Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404.")] = None, _request_timeout: Union[ None, Annotated[StrictFloat, Field(gt=0)], @@ -210,11 +210,11 @@ def query_without_preload_content( ) -> RESTResponseType: """Execute SQL query - Execute a SQL query against all registered connections and datasets. Use standard Postgres-compatible SQL to reference tables from any connection using the format `connection_name.schema.table`. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. Set the `X-Database-Id` header to scope the query to a specific database. Inside a database scope the query only sees that database's auto `default` catalog plus any catalogs explicitly attached to it; workspace catalogs are invisible. + Execute a SQL query scoped to a database. A database is the only window into catalogs: the query sees only that database's auto `default` catalog plus any catalogs explicitly attached to it. Select the database with EITHER the `X-Database-Id` header OR the `database_id` body field (exactly one must be given; if both are sent and disagree, that's a 400). Use standard Postgres-compatible SQL; reference the default catalog as `default..
` (or just `.
` / `
`) and attached catalogs by their alias. Results are returned inline and a `result_id` is provided for later retrieval via the Results API. Set `async: true` to execute asynchronously — returns a query run ID for polling. Optionally set `async_after_ms` to attempt synchronous execution first, falling back to async if the query exceeds the timeout. :param query_request: (required) :type query_request: QueryRequest - :param x_database_id: Scope the query to a specific database (its id). When set, only the database's attached catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. + :param x_database_id: Database id to scope the query to. Required unless the `database_id` body field is set; if both are present they must match. Only that database's catalogs are visible during planning. A malformed value is a 400; an unknown database id is a 404. :type x_database_id: str :param _request_timeout: timeout setting for this request. If one number provided, it will be total request diff --git a/hotdata/api/saved_queries_api.py b/hotdata/api/saved_queries_api.py index 331a851..aaeff93 100644 --- a/hotdata/api/saved_queries_api.py +++ b/hotdata/api/saved_queries_api.py @@ -589,6 +589,7 @@ def _delete_saved_query_serialize( def execute_saved_query( self, id: Annotated[StrictStr, Field(description="Saved query ID")], + x_database_id: Annotated[StrictStr, Field(description="Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404.")], execute_saved_query_request: Annotated[Optional[ExecuteSavedQueryRequest], Field(description="Optional version to execute")] = None, _request_timeout: Union[ None, @@ -605,10 +606,12 @@ def execute_saved_query( ) -> QueryResponse: """Execute saved query - Execute a saved query. By default runs the latest version. Optionally specify a version number to execute a previous version. Returns the same response format as POST /v1/query. + Execute a saved query, scoped to a database (required `X-Database-Id` header). By default runs the latest version. Optionally specify a version number to execute a previous version. The SQL runs inside the given database scope, the same way POST /v1/query does. Returns the same response format as POST /v1/query. :param id: Saved query ID (required) :type id: str + :param x_database_id: Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404. (required) + :type x_database_id: str :param execute_saved_query_request: Optional version to execute :type execute_saved_query_request: ExecuteSavedQueryRequest :param _request_timeout: timeout setting for this request. If one @@ -635,6 +638,7 @@ def execute_saved_query( _param = self._execute_saved_query_serialize( id=id, + x_database_id=x_database_id, execute_saved_query_request=execute_saved_query_request, _request_auth=_request_auth, _content_type=_content_type, @@ -644,6 +648,7 @@ def execute_saved_query( _response_types_map: Dict[str, Optional[str]] = { '200': "QueryResponse", + '400': "ApiErrorResponse", '404': "ApiErrorResponse", } response_data = self.api_client.call_api( @@ -661,6 +666,7 @@ def execute_saved_query( def execute_saved_query_with_http_info( self, id: Annotated[StrictStr, Field(description="Saved query ID")], + x_database_id: Annotated[StrictStr, Field(description="Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404.")], execute_saved_query_request: Annotated[Optional[ExecuteSavedQueryRequest], Field(description="Optional version to execute")] = None, _request_timeout: Union[ None, @@ -677,10 +683,12 @@ def execute_saved_query_with_http_info( ) -> ApiResponse[QueryResponse]: """Execute saved query - Execute a saved query. By default runs the latest version. Optionally specify a version number to execute a previous version. Returns the same response format as POST /v1/query. + Execute a saved query, scoped to a database (required `X-Database-Id` header). By default runs the latest version. Optionally specify a version number to execute a previous version. The SQL runs inside the given database scope, the same way POST /v1/query does. Returns the same response format as POST /v1/query. :param id: Saved query ID (required) :type id: str + :param x_database_id: Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404. (required) + :type x_database_id: str :param execute_saved_query_request: Optional version to execute :type execute_saved_query_request: ExecuteSavedQueryRequest :param _request_timeout: timeout setting for this request. If one @@ -707,6 +715,7 @@ def execute_saved_query_with_http_info( _param = self._execute_saved_query_serialize( id=id, + x_database_id=x_database_id, execute_saved_query_request=execute_saved_query_request, _request_auth=_request_auth, _content_type=_content_type, @@ -716,6 +725,7 @@ def execute_saved_query_with_http_info( _response_types_map: Dict[str, Optional[str]] = { '200': "QueryResponse", + '400': "ApiErrorResponse", '404': "ApiErrorResponse", } response_data = self.api_client.call_api( @@ -733,6 +743,7 @@ def execute_saved_query_with_http_info( def execute_saved_query_without_preload_content( self, id: Annotated[StrictStr, Field(description="Saved query ID")], + x_database_id: Annotated[StrictStr, Field(description="Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404.")], execute_saved_query_request: Annotated[Optional[ExecuteSavedQueryRequest], Field(description="Optional version to execute")] = None, _request_timeout: Union[ None, @@ -749,10 +760,12 @@ def execute_saved_query_without_preload_content( ) -> RESTResponseType: """Execute saved query - Execute a saved query. By default runs the latest version. Optionally specify a version number to execute a previous version. Returns the same response format as POST /v1/query. + Execute a saved query, scoped to a database (required `X-Database-Id` header). By default runs the latest version. Optionally specify a version number to execute a previous version. The SQL runs inside the given database scope, the same way POST /v1/query does. Returns the same response format as POST /v1/query. :param id: Saved query ID (required) :type id: str + :param x_database_id: Required. Scope execution to this database (its id). A missing or malformed value is a 400; an unknown database id is a 404. (required) + :type x_database_id: str :param execute_saved_query_request: Optional version to execute :type execute_saved_query_request: ExecuteSavedQueryRequest :param _request_timeout: timeout setting for this request. If one @@ -779,6 +792,7 @@ def execute_saved_query_without_preload_content( _param = self._execute_saved_query_serialize( id=id, + x_database_id=x_database_id, execute_saved_query_request=execute_saved_query_request, _request_auth=_request_auth, _content_type=_content_type, @@ -788,6 +802,7 @@ def execute_saved_query_without_preload_content( _response_types_map: Dict[str, Optional[str]] = { '200': "QueryResponse", + '400': "ApiErrorResponse", '404': "ApiErrorResponse", } response_data = self.api_client.call_api( @@ -800,6 +815,7 @@ def execute_saved_query_without_preload_content( def _execute_saved_query_serialize( self, id, + x_database_id, execute_saved_query_request, _request_auth, _content_type, @@ -826,6 +842,8 @@ def _execute_saved_query_serialize( _path_params['id'] = id # process the query parameters # process the header parameters + if x_database_id is not None: + _header_params['X-Database-Id'] = x_database_id # process the form parameters # process the body parameter if execute_saved_query_request is not None: diff --git a/hotdata/api_client.py b/hotdata/api_client.py index b1bc64b..4c1cabd 100644 --- a/hotdata/api_client.py +++ b/hotdata/api_client.py @@ -91,7 +91,7 @@ def __init__( self.default_headers[header_name] = header_value self.cookie = cookie # Set default User-Agent. - self.user_agent = 'OpenAPI-Generator/0.2.5/python' + self.user_agent = 'OpenAPI-Generator/0.2.6/python' self.client_side_validation = configuration.client_side_validation def __enter__(self): diff --git a/hotdata/models/create_database_request.py b/hotdata/models/create_database_request.py index 5d0e07c..51a8ed8 100644 --- a/hotdata/models/create_database_request.py +++ b/hotdata/models/create_database_request.py @@ -28,10 +28,11 @@ class CreateDatabaseRequest(BaseModel): """ Request body for POST /databases """ # noqa: E501 - expires_at: Optional[StrictStr] = Field(default=None, description="When this database expires. Accepts either an RFC 3339 timestamp (e.g. `\"2026-06-01T00:00:00Z\"`) or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days) — for example `\"24h\"`, `\"48h\"`, or `\"7d\"`. Defaults to `\"24h\"` when omitted. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp.") + expires_at: Optional[StrictStr] = Field(default=None, description="When this database expires. Accepts either an RFC 3339 timestamp (e.g. `\"2026-06-01T00:00:00Z\"`) or a relative duration suffixed with `h` (hours), `m` (minutes), or `d` (days) — for example `\"24h\"`, `\"48h\"`, or `\"7d\"`. Omitted (or empty) means the database never expires. Expiry is best-effort: the database will not be deleted before `expires_at`, but cleanup may run later than the exact timestamp.") name: Optional[StrictStr] = Field(default=None, description="Optional free-form display label (for UIs/CLIs). Not unique. Not an identifier — databases are always addressed by `id`. Accepts the legacy `description` key as an alias so clients that predate the rename keep populating this field.") schemas: Optional[List[DatabaseDefaultSchemaDecl]] = Field(default=None, description="Optional schemas/tables to declare on the database's auto-created `default` catalog. Mirrors the `config.schemas` field of a managed `POST /v1/connections`. Tables declared here can be loaded via the standard managed-table load endpoint targeting `default_connection_id`. Omitted or empty means the default catalog starts empty.") - __properties: ClassVar[List[str]] = ["expires_at", "name", "schemas"] + storage_backend: Optional[StrictStr] = Field(default=None, description="Physical storage backend for the database's auto-created `default` catalog. `\"parquet\"` (default) uses the versioned parquet cache. `\"ducklake\"` stores data in a DuckLake catalog in the shared metadata DB configured via `ducklake.metadata_pg_url`, which must be configured for that value to be accepted. Omitted means `\"parquet\"`.") + __properties: ClassVar[List[str]] = ["expires_at", "name", "schemas", "storage_backend"] model_config = ConfigDict( populate_by_name=True, @@ -89,6 +90,11 @@ def to_dict(self) -> Dict[str, Any]: if self.name is None and "name" in self.model_fields_set: _dict['name'] = None + # set to None if storage_backend (nullable) is None + # and model_fields_set contains the field + if self.storage_backend is None and "storage_backend" in self.model_fields_set: + _dict['storage_backend'] = None + return _dict @classmethod @@ -103,7 +109,8 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: _obj = cls.model_validate({ "expires_at": obj.get("expires_at"), "name": obj.get("name"), - "schemas": [DatabaseDefaultSchemaDecl.from_dict(_item) for _item in obj["schemas"]] if obj.get("schemas") is not None else None + "schemas": [DatabaseDefaultSchemaDecl.from_dict(_item) for _item in obj["schemas"]] if obj.get("schemas") is not None else None, + "storage_backend": obj.get("storage_backend") }) return _obj diff --git a/hotdata/models/query_request.py b/hotdata/models/query_request.py index a956105..abebade 100644 --- a/hotdata/models/query_request.py +++ b/hotdata/models/query_request.py @@ -30,10 +30,11 @@ class QueryRequest(BaseModel): """ # noqa: E501 var_async: Optional[StrictBool] = Field(default=None, description="When true, execute the query asynchronously and return a query run ID for polling via GET /query-runs/{id}. The query results can be retrieved via GET /results/{id} once the query run status is \"succeeded\".", alias="async") async_after_ms: Optional[Annotated[int, Field(strict=True, ge=1000)]] = Field(default=None, description="If set with async=true, wait up to this many milliseconds for the query to complete synchronously before returning an async response. Minimum 1000ms. Ignored if async is false.") - default_catalog: Optional[StrictStr] = Field(default=None, description="Catalog that unqualified table references resolve against. Only honored inside an `X-Database-Id` scope; sending it without that header is a 400. Must name a catalog visible in the database (`default`, an attached catalog alias, or a system catalog). Defaults to `default` when omitted.") - default_schema: Optional[StrictStr] = Field(default=None, description="Schema that unqualified table references resolve against. Only honored inside an `X-Database-Id` scope; sending it without that header is a 400. Defaults to `main` when omitted. Existence is not validated up front — an unknown schema surfaces as a \"table not found\" error at planning time.") + database_id: Optional[StrictStr] = Field(default=None, description="Database to scope the query to (its id). Alternative to the `X-Database-Id` header — exactly one source must be provided. If both this field and the header are set and they disagree, the request is rejected with a 400.") + default_catalog: Optional[StrictStr] = Field(default=None, description="Catalog that unqualified table references resolve against within the query's database scope. Must name a catalog visible in the database (`default`, an attached catalog alias, or a system catalog). Defaults to `default` when omitted.") + default_schema: Optional[StrictStr] = Field(default=None, description="Schema that unqualified table references resolve against within the query's database scope. Defaults to `main` when omitted. Existence is not validated up front — an unknown schema surfaces as a \"table not found\" error at planning time.") sql: StrictStr - __properties: ClassVar[List[str]] = ["async", "async_after_ms", "default_catalog", "default_schema", "sql"] + __properties: ClassVar[List[str]] = ["async", "async_after_ms", "database_id", "default_catalog", "default_schema", "sql"] model_config = ConfigDict( populate_by_name=True, @@ -79,6 +80,11 @@ def to_dict(self) -> Dict[str, Any]: if self.async_after_ms is None and "async_after_ms" in self.model_fields_set: _dict['async_after_ms'] = None + # set to None if database_id (nullable) is None + # and model_fields_set contains the field + if self.database_id is None and "database_id" in self.model_fields_set: + _dict['database_id'] = None + # set to None if default_catalog (nullable) is None # and model_fields_set contains the field if self.default_catalog is None and "default_catalog" in self.model_fields_set: @@ -103,6 +109,7 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]: _obj = cls.model_validate({ "async": obj.get("async"), "async_after_ms": obj.get("async_after_ms"), + "database_id": obj.get("database_id"), "default_catalog": obj.get("default_catalog"), "default_schema": obj.get("default_schema"), "sql": obj.get("sql")