Skip to content

Commit d5b53e0

Browse files
izeigermanizeigerman
authored
Chore: Document the 'allow_partials' attribute in the model definition (#1485)
1 parent fb0fb4e commit d5b53e0

2 files changed

Lines changed: 23 additions & 19 deletions

File tree

‎docs/concepts/models/overview.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,9 @@ MODEL (
129129

130130
```
131131

132+
### allow_partials
133+
- Indicates that this model can be executed for partial (incomplete) data intervals. Setting this to `true` means that the model may process a segment of input data that is missing some of the data points. Please note that setting this attribute to `true` results in the disregard of the [cron](#cron) attribute.
134+
132135
## Incremental Model Properties
133136

134137
For models that are incremental, the following parameters can be specified in the `kind`'s definition.

‎docs/reference/model_configuration.md‎

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -8,25 +8,26 @@ Learn more about specifying SQLMesh model properties in the [model concepts over
88

99
Configuration options for SQLMesh model properties supported by all model kinds.
1010

11-
| Option | Description | Type | Required |
12-
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------: | :------: |
13-
| `name` | The model name. Must include at least a qualifying schema (`<schema>.<model>`) and may include a catalog (`<catalog>.<schema>.<model>`). If any project model name includes a catalog, all model names must include a catalog. | str | Y |
14-
| `kind` | The default model kind ([Additional Details](../guides/configuration.md#model-kind)) (Default: `VIEW`) | str \| dict | N |
15-
| `dialect` | The SQL dialect in which the model's query is written. All SQL dialects [supported by the SQLGlot library](https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/dialect.py) are allowed. | str | N |
16-
| `owner` | The owner of a model; may be used for notification purposes | str | N |
17-
| `stamp` | Arbitrary string used to indicate a model's version without changing the model name | str | N |
18-
| `tags` | Arbitrary strings used to organize or classify a model | array[str] | N |
19-
| `cron` | The default cron expression specifying how often the model should be refreshed. (Default: `@daily`) | str | N |
20-
| `interval_unit` | The temporal granularity of the model's data intervals. Supported values: `year`, `month`, `day`, `hour`, `half_hour`, `quarter_hour`, `five_minute`. (Default: inferred from `cron`) | str | N |
21-
| `start` | The date/time that determines the earliest date interval that should be processed by a model. Can be a datetime string, epoch time in milliseconds, or a relative datetime such as `1 year ago`. | str \| int | N |
22-
| `batch_size` | The maximum number of intervals that can be evaluated in a single backfill task. If this is `None`, all intervals will be processed as part of a single task. If this is set, a model's backfill will be chunked such that each individual task only contains jobs with the maximum of `batch_size` intervals. (Default: `None`) | int | N |
23-
| `grains` | The column(s) whose combination uniquely identifies each row in the model | str \| array[str] | N |
24-
| `references` | The model column(s) used to join to other models' grains | str \| array[str] | N |
25-
| `depends_on` | Models on which this model depends. (Default: dependencies inferred from model code) | array[str] | N |
26-
| `storage_format` | The storage format that should be used to store physical tables; only applicable to engines such as Spark | str | N |
27-
| `partitioned_by` | The column(s) used to partition the model's physical table; only applicable to engines that support partitioning | str | N |
28-
| `clustered_by` | The column(s) used to cluster the model's physical table; only applicable to engines that support clustering | str | N |
29-
| `table_properties` | Arbitrary table properties specific to the target engine. Specified as key-value pairs (`key = value`) | dict | N |
11+
| Option | Description | Type | Required |
12+
|--------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------:|:--------:|
13+
| `name` | The model name. Must include at least a qualifying schema (`<schema>.<model>`) and may include a catalog (`<catalog>.<schema>.<model>`). If any project model name includes a catalog, all model names must include a catalog. | str | Y |
14+
| `kind` | The default model kind ([Additional Details](../guides/configuration.md#model-kind)) (Default: `VIEW`) | str \| dict | N |
15+
| `dialect` | The SQL dialect in which the model's query is written. All SQL dialects [supported by the SQLGlot library](https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/dialect.py) are allowed. | str | N |
16+
| `owner` | The owner of a model; may be used for notification purposes | str | N |
17+
| `stamp` | Arbitrary string used to indicate a model's version without changing the model name | str | N |
18+
| `tags` | Arbitrary strings used to organize or classify a model | array[str] | N |
19+
| `cron` | The default cron expression specifying how often the model should be refreshed. (Default: `@daily`) | str | N |
20+
| `interval_unit` | The temporal granularity of the model's data intervals. Supported values: `year`, `month`, `day`, `hour`, `half_hour`, `quarter_hour`, `five_minute`. (Default: inferred from `cron`) | str | N |
21+
| `start` | The date/time that determines the earliest date interval that should be processed by a model. Can be a datetime string, epoch time in milliseconds, or a relative datetime such as `1 year ago`. | str \| int | N |
22+
| `batch_size` | The maximum number of intervals that can be evaluated in a single backfill task. If this is `None`, all intervals will be processed as part of a single task. If this is set, a model's backfill will be chunked such that each individual task only contains jobs with the maximum of `batch_size` intervals. (Default: `None`) | int | N |
23+
| `grains` | The column(s) whose combination uniquely identifies each row in the model | str \| array[str] | N |
24+
| `references` | The model column(s) used to join to other models' grains | str \| array[str] | N |
25+
| `depends_on` | Models on which this model depends. (Default: dependencies inferred from model code) | array[str] | N |
26+
| `storage_format` | The storage format that should be used to store physical tables; only applicable to engines such as Spark | str | N |
27+
| `partitioned_by` | The column(s) used to partition the model's physical table; only applicable to engines that support partitioning | str | N |
28+
| `clustered_by` | The column(s) used to cluster the model's physical table; only applicable to engines that support clustering | str | N |
29+
| `table_properties` | Arbitrary table properties specific to the target engine. Specified as key-value pairs (`key = value`) | dict | N |
30+
| `allow_partials` | Whether this model can process partial (incomplete) data intervals | bool | N |
3031

3132
### Model defaults
3233

0 commit comments

Comments
 (0)