Skip to content

Commit e1d667b

Browse files
jochenchristjochenchrist
authored
Merge pull request #111 from datacontract/develop/1.2.0
v1.2.0
2 parents e7a0259 + 78f938a commit e1d667b

13 files changed

Lines changed: 6220 additions & 29 deletions

‎CHANGELOG.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,13 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8-
## [Unreleased]
8+
## [1.2.0] - 2024-01-22
9+
10+
### Added
11+
- Support for models.additionalFields (#99)
12+
13+
### Changed
14+
- server.type changed from enum to simple string to support custom types (#107)
915

1016
## [1.1.0] - 2024-10-30
1117

‎README.md‎

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -371,12 +371,12 @@ This object _MAY_ be extended with [Specification Extensions](#specification-ext
371371

372372
The fields are dependent on the defined type.
373373

374-
| Field | Type | Description |
375-
|-------------|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
376-
| type | `string` | REQUIRED. The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `s3`, `glue`, `redshift`, `azure`, `sqlserver`, `snowflake`, `databricks`, `postgres`, `oracle`, `kafka`, `pubsub`, `sftp`, `kinesis`, `trino`, `local` |
377-
| description | `string` | An optional string describing the server. |
378-
| environment | `string` | An optional string describing the environment, e.g., prod, sit, stg. |
379-
| roles | Array of [Server Role Object](#server-role-object) | An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data. |
374+
| Field | Type | Description |
375+
|-------------|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
376+
| type | `string` | REQUIRED. The type of the data product technology that implements the data contract. Well-known server types are: `bigquery`, `clickhouse`, `s3`, `glue`, `redshift`, `azure`, `sqlserver`, `snowflake`, `databricks`, `postgres`, `oracle`, `kafka`, `pubsub`, `sftp`, `kinesis`, `trino`, `local` |
377+
| description | `string` | An optional string describing the server. |
378+
| environment | `string` | An optional string describing the environment, e.g., prod, sit, stg. |
379+
| roles | Array of [Server Role Object](#server-role-object) | An optional array of roles that are available and can be requested to access the server for role-based access control. E.g. separate roles for different regions or sensitive data. |
380380

381381
This object _MAY_ be extended with [Specification Extensions](#specification-extensions).
382382

@@ -631,16 +631,17 @@ The Model Object describes the structure and semantics of a data model, such as
631631

632632
The name of the data model (table name) is defined by the key that refers to this Model Object.
633633

634-
| Field | Type | Description |
635-
|-------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|
636-
| type | `string` | The type of the model. Examples: `table`, `view`, `object`. Default: `table`. |
637-
| description | `string` | An optional string describing the data model. |
638-
| title | `string` | An optional string for the title of the data model. Especially useful if the name of the model is cryptic or contains abbreviations. |
639-
| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |
640-
| primaryKey | Array of `string` | If the primary key is a compound key, list the field names that constitute the primary key. Alternative to field-level `primaryKey`. |
641-
| quality | Array of [Quality Object](#quality-object) | Specifies the quality attributes on model level. |
642-
| examples | Array of `Any` | Specifies example data sets for the model. |
643-
| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |
634+
| Field | Type | Description |
635+
|------------------|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|
636+
| type | `string` | The type of the model. Examples: `table`, `view`, `object`. Default: `table`. |
637+
| description | `string` | An optional string describing the data model. |
638+
| title | `string` | An optional string for the title of the data model. Especially useful if the name of the model is cryptic or contains abbreviations. |
639+
| fields | Map[`string`, [Field Object](#field-object)] | The fields (e.g. columns) of the data model. |
640+
| primaryKey | Array of `string` | If the primary key is a compound key, list the field names that constitute the primary key. Alternative to field-level `primaryKey`. |
641+
| quality | Array of [Quality Object](#quality-object) | Specifies the quality attributes on model level. |
642+
| examples | Array of `Any` | Specifies example data sets for the model. |
643+
| additionalFields | `Boolean` | Specify, if the model can have additional fields that are not defined in the contract. Default: `false`. |
644+
| config | [Config Object](#config-object) | Any additional key-value pairs that might be useful for further tooling. |
644645

645646

646647
This object _MAY_ be extended with [Specification Extensions](#specification-extensions).
@@ -1159,10 +1160,13 @@ The following data types are supported for model fields and definitions:
11591160
- Timestamp with timezone: `timestamp`, `timestamp_tz`
11601161
- Timestamp with no timezone: `timestamp_ntz`
11611162
- Date with no time information: `date`
1163+
- Time with no date information: `time`
11621164
- Array: `array`
11631165
- Map: `map` (may not be supported by some server types)
11641166
- Sequence of 8-bit unsigned bytes: `bytes`
11651167
- Complex type: `object`, `record`, `struct`
1168+
- Semi-structured data: `variant` (may not be supported by some server types)
1169+
- JSON data: `json` (may not be supported by some server types)
11661170
- No value: `null`
11671171

11681172
### Specification Extensions

‎datacontract.schema.json‎

Lines changed: 66 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
"type": "string",
88
"title": "DataContractSpecificationVersion",
99
"enum": [
10+
"1.2.0",
1011
"1.1.0",
1112
"0.9.3",
1213
"0.9.2",
@@ -336,6 +337,21 @@
336337
"$ref": "#/$defs/TrinoServer"
337338
}
338339
},
340+
{
341+
"if": {
342+
"properties": {
343+
"type": {
344+
"const": "clickhouse"
345+
}
346+
},
347+
"required": [
348+
"type"
349+
]
350+
},
351+
"then": {
352+
"$ref": "#/$defs/ClickhouseServer"
353+
}
354+
},
339355
{
340356
"if": {
341357
"properties": {
@@ -689,6 +705,11 @@
689705
"examples": {
690706
"type": "array"
691707
},
708+
"additionalFields": {
709+
"type": "boolean",
710+
"description": " Specify, if the model can have additional fields that are not defined in the contract. ",
711+
"default": false
712+
},
692713
"config": {
693714
"type": "object",
694715
"description": "Additional metadata for model configuration.",
@@ -1092,12 +1113,15 @@
10921113
"timestamp_tz",
10931114
"timestamp_ntz",
10941115
"date",
1116+
"time",
10951117
"array",
10961118
"map",
10971119
"object",
10981120
"record",
10991121
"struct",
11001122
"bytes",
1123+
"variant",
1124+
"json",
11011125
"null"
11021126
]
11031127
},
@@ -1115,25 +1139,26 @@
11151139
"type": {
11161140
"type": "string",
11171141
"description": "The type of the data product technology that implements the data contract.",
1118-
"enum": [
1142+
"examples": [
1143+
"azure",
11191144
"bigquery",
11201145
"BigQuery",
1121-
"s3",
1122-
"sftp",
1123-
"redshift",
1124-
"azure",
1125-
"sqlserver",
1126-
"snowflake",
1146+
"clickhouse",
11271147
"databricks",
11281148
"dataframe",
11291149
"glue",
1130-
"postgres",
1131-
"oracle",
11321150
"kafka",
1133-
"pubsub",
11341151
"kinesis",
1135-
"trino",
1136-
"local"
1152+
"local",
1153+
"oracle",
1154+
"postgres",
1155+
"pubsub",
1156+
"redshift",
1157+
"sftp",
1158+
"sqlserver",
1159+
"snowflake",
1160+
"s3",
1161+
"trino"
11371162
]
11381163
},
11391164
"roles": {
@@ -1663,6 +1688,35 @@
16631688
"schema"
16641689
]
16651690
},
1691+
"ClickhouseServer": {
1692+
"type": "object",
1693+
"title": "ClickhouseServer",
1694+
"properties": {
1695+
"host": {
1696+
"type": "string",
1697+
"description": "The host to the database server",
1698+
"examples": [
1699+
"localhost"
1700+
]
1701+
},
1702+
"port": {
1703+
"type": "integer",
1704+
"description": "The port to the database server."
1705+
},
1706+
"database": {
1707+
"type": "string",
1708+
"description": "The name of the database.",
1709+
"examples": [
1710+
"postgres"
1711+
]
1712+
}
1713+
},
1714+
"required": [
1715+
"host",
1716+
"port",
1717+
"database"
1718+
]
1719+
},
16661720
"LocalServer": {
16671721
"type": "object",
16681722
"title": "LocalServer",

0 commit comments

Comments
 (0)