You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(docs): casing of files and folders; remove closed issues from fusion limitations (#447)
* fix casing and separators, missing _category_.json
* Remove closed issues from limitation page
* fix: rename reference table site to match old url-pattern
* fix: add missing _category_.jsons
Copy file name to clipboardExpand all lines: docs/01_macro-instructions/06_hubs/06_standard-hub.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ title: Standard Hub
8
8
9
9
---
10
10
11
-
This macro creates a standard Hub entity based on one or more stage models. The macro requires an input source model similar to the output of the stage macro. So by default the stage models would be used as source models for hubs. If a Hub is loaded by multiple sources, each source needs to have the same number of Business Key columns. Additionally, a multi-source hub needs a [rsrc_static Attribute](../../26_General_Usage_Notes/32_The_rsrc_static_Attribute/32_the_rsrc_static_attribute.md) defined for each source.
11
+
This macro creates a standard Hub entity based on one or more stage models. The macro requires an input source model similar to the output of the stage macro. So by default the stage models would be used as source models for hubs. If a Hub is loaded by multiple sources, each source needs to have the same number of Business Key columns. Additionally, a multi-source hub needs a [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md) defined for each source.
| hashkey | string | mandatory | – | Name of the hashkey column inside the stage, that should be used as PK of the Hub. |
25
25
| business_keys | string \| list of strings | mandatory | – | Name(s) of the business key columns that should be loaded into the hub and are the input of the hashkey column. Needs to be available inside the stage model. If the names differ between multiple sources, you should define here how the business keys should be called inside the final hub model. The actual input column names need to be defined inside the `source_model` parameter then. |
26
-
| source_models | string \| list of dictionaries \| dictionary | mandatory | – | If single source, just a string holding the name of the stage model is required. For multi source Hubs, a list of dictionaries with information about each source is required. For more information see [this](../../26_General_Usage_Notes/30_Multi-Source_Entities/30_multi-source_entities.md) page! The inner dictionaries need to have `name` as a key, and optionally the keys `rsrc_static`, `hk_column` and `bk_columns`. For further information about the `rsrc_static` attribute, please visit the following page: [rsrc_static Attribute](../../26_General_Usage_Notes/32_The_rsrc_static_Attribute/32_the_rsrc_static_attribute.md)|
26
+
| source_models | string \| list of dictionaries \| dictionary | mandatory | – | If single source, just a string holding the name of the stage model is required. For multi source Hubs, a list of dictionaries with information about each source is required. For more information see [this](../../26_general-usage-notes/30_multi-source-entities/30_multi-source-entities.md) page! The inner dictionaries need to have `name` as a key, and optionally the keys `rsrc_static`, `hk_column` and `bk_columns`. For further information about the `rsrc_static` attribute, please visit the following page: [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md)|
27
27
28
28
### OPTIONAL PARAMETERS
29
29
@@ -55,4 +55,4 @@ source_models: stage_account
55
55
-**hashkey:** This hashkey column was created before inside the corresponding staging area, using the stage macro.
56
56
-**business_keys:** This hub has two business keys which are both defined here. Need to equal the input columns for the hashkey column.
57
57
-**source_models:** This would create a hub loaded from only one source, which is not uncommon. It uses the model `stage_account` and since no `bk_columns` are specified, the same columns as defined in `business_keys` will be selected from the source.
58
-
- The `rsrc_static` attribute is not set, because it is not required for single source entities. For more information see [rsrc_static Attribute](/docs/General_Usage_Notes/The_rsrc_static_Attribute).
58
+
- The `rsrc_static` attribute is not set, because it is not required for single source entities. For more information see [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md).
Copy file name to clipboardExpand all lines: docs/01_macro-instructions/07_links/07_standard-link.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ title: Standard Link
8
8
9
9
---
10
10
11
-
This macro creates a link entity, connecting two or more entities, or an entity with itself. It can be loaded by one or more source staging tables, if multiple sources share the same business definitions. Typically a link would only be loaded by multiple sources, if those multiple sources also share the business definitions of the hubs, and therefore load the connected hubs together as well. If multiple sources are used, it is required that they all have the same number of foreign keys inside, otherwise they would not share the same business definition of that link. Additionally, a multi-source link needs a [rsrc_static Attribute](../../26_General_Usage_Notes/32_The_rsrc_static_Attribute/32_the_rsrc_static_attribute.md) defined for each source.
11
+
This macro creates a link entity, connecting two or more entities, or an entity with itself. It can be loaded by one or more source staging tables, if multiple sources share the same business definitions. Typically a link would only be loaded by multiple sources, if those multiple sources also share the business definitions of the hubs, and therefore load the connected hubs together as well. If multiple sources are used, it is required that they all have the same number of foreign keys inside, otherwise they would not share the same business definition of that link. Additionally, a multi-source link needs a [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md) defined for each source.
| link_hashkey | string | mandatory | – | Name of the link hashkey column inside the stage. Should get calculated out of all business keys inside the link. |
25
25
| foreign_hashkeys | list of strings | mandatory | – | List of all hashkey columns inside the link, that refer to other hub entities. All hashkey columns must be available inside the stage area. |
26
-
| source_models | string \| list of dictionaries \| dictionary | mandatory | – | For a single source entity, a string with the name of the source staging model is required. For multi source entities, a list of dictionaries with information about the source models is required. For more information see [this](../../26_General_Usage_Notes/30_Multi-Source_Entities/30_multi-source_entities.md) page! The dictionaries need to have the keys `name` and optionally the keys `rsrc_static`, `hk_column` and `fk_columns`. For further information about the `rsrc_static` attribute, please visit the following page: [rsrc_static Attribute](../../26_General_Usage_Notes/32_The_rsrc_static_Attribute/32_the_rsrc_static_attribute.md)|
26
+
| source_models | string \| list of dictionaries \| dictionary | mandatory | – | For a single source entity, a string with the name of the source staging model is required. For multi source entities, a list of dictionaries with information about the source models is required. For more information see [this](../../26_general-usage-notes/30_multi-source-entities/30_multi-source-entities.md) page! The dictionaries need to have the keys `name` and optionally the keys `rsrc_static`, `hk_column` and `fk_columns`. For further information about the `rsrc_static` attribute, please visit the following page: [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md)|
27
27
28
28
### OPTIONAL PARAMETERS
29
29
@@ -59,7 +59,7 @@ With this example, a regular standard link is created. The created link represen
59
59
-**foreign_hashkeys:**
60
60
-[`hk_opportunity_h`, `hk_account_h`] The link between opportunity and account needs to contain both the hashkey of account and contact to enable joins the the corresponding hub entities.
61
61
-**source_models:**
62
-
- This would create a link loaded from only one source, which is not uncommon. It uses the model `stage_account`. The rsrc_static attribute is not set, because it is not required for single source entities. For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](/docs/General_Usage_Notes/The_rsrc_static_Attribute).
62
+
- This would create a link loaded from only one source, which is not uncommon. It uses the model `stage_account`. The rsrc_static attribute is not set, because it is not required for single source entities. For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md).
63
63
64
64
## EXAMPLE 2
65
65
@@ -95,14 +95,14 @@ With this example, a regular standard link is created. The created link represen
95
95
-[`hk_opportunity_h`, `hk_account_h`] The link between opportunity and account needs to contain both the hashkey of account and contact to enable joins the the corresponding hub entities.
96
96
-**source_models:**
97
97
- This would create a link loaded from two sources, which is also not uncommon. With “link_hk” and “fk_columns” defined differently for stage_account, a source mapping is enabled, that allows users to use different input columns for different source models.
98
-
- For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](/docs/General_Usage_Notes/The_rsrc_static_Attribute).
98
+
- For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md).
99
99
100
100
### DISABLING HIGH-WATER MARK
101
101
102
102
The High-Water Mark can be disabled safely, but typically would decrease the performance again.
103
103
104
104
We recommend to **try a bit what works best in your environment**. You basically have three options:
105
105
106
-
-**Keep HWM activated**, for multi-source Links this would require the [rsrc_static](/docs/General_Usage_Notes/The_rsrc_static_Attribute) to be defined for each source. For single source Links, nothing needs to be done, the HWM is activated automatically.
106
+
-**Keep HWM activated**, for multi-source Links this would require the [rsrc_static](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md) to be defined for each source. For single source Links, nothing needs to be done, the HWM is activated automatically.
107
107
-**Disable the HWM entirely.** For multi-source Links you just need to not specify the rsrc_static attribute. For single-source Links you need to add the parameter disable_hwm=true to your Link macro call.
108
108
-**Move the HWM to a previous layer.** First, you apply the previous step to disable the HWM in the Links. Then you implement some kind of mechanism in previous dbt layers to ensure that only records newer than what you already processed are available there. This could be especially effective when combining with different materializations of these previous layers.
Copy file name to clipboardExpand all lines: docs/01_macro-instructions/07_links/08_non-historized-link.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ title: Non-Historized Link
8
8
9
9
---
10
10
11
-
This macro creates a non-historized (former transactional) link entity, connecting two or more entities, or an entity with itself. It can be loaded by one or more source staging tables, if multiple sources share the same business definitions. If multiple sources are used, it is required that they all have the same number of foreign keys inside, otherwise they would not share the same business definition of that non-historized link. Additionally, a multi-source nh-link needs a [rsrc_static Attribute](../../26_General_Usage_Notes/32_The_rsrc_static_Attribute/32_the_rsrc_static_attribute.md) defined for each source.
11
+
This macro creates a non-historized (former transactional) link entity, connecting two or more entities, or an entity with itself. It can be loaded by one or more source staging tables, if multiple sources share the same business definitions. If multiple sources are used, it is required that they all have the same number of foreign keys inside, otherwise they would not share the same business definition of that non-historized link. Additionally, a multi-source nh-link needs a [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md) defined for each source.
12
12
13
13
In the background a non-historized link uses exactly the same loading logic as a regular link, but adds the descriptive attributes as additional payload.
| link_hashkey | string | mandatory | – | Name of the non-historized link hashkey column inside the stage. Should be calculated out of all business keys inside the link. |
27
27
| payload | list of strings | mandatory | – | A list of all the descriptive attributes that should be the payload of this non-historized link. If the names differ between source models, this list will define how the columns are named inside the result non historized link. The mapping which columns to use from which source model then need to be defined inside the parameter `payload` inside the variable `source_models`. |
28
-
| source_models | string \| list of dictionaries \| dictionary | mandatory | – | For a single source entity, a string with the name of the source staging model is required. For multi source entities, a list of dictionaries with information for each source is required. These dictionaries need to have the key `name`, and optionally the keys `rsrc_static`, `hk_column`, `fk_columns` and `payload`. Especially regarding multi-source metadata, please see [this](../../26_General_Usage_Notes/30_Multi-Source_Entities/30_multi-source_entities.md) page! For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](../../26_General_Usage_Notes/32_The_rsrc_static_Attribute/32_the_rsrc_static_attribute.md)|
28
+
| source_models | string \| list of dictionaries \| dictionary | mandatory | – | For a single source entity, a string with the name of the source staging model is required. For multi source entities, a list of dictionaries with information for each source is required. These dictionaries need to have the key `name`, and optionally the keys `rsrc_static`, `hk_column`, `fk_columns` and `payload`. Especially regarding multi-source metadata, please see [this](../../26_general-usage-notes/30_multi-source-entities/30_multi-source-entities.md) page! For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md)|
29
29
30
30
### OPTIONAL PARAMETERS
31
31
@@ -102,7 +102,7 @@ With this example, a non-historized link is created.
102
102
-**foreign_hashkeys:**
103
103
-[`hk_creditcard_h`] List of all hashkey columns inside the non-historized link, that refer to other hub entities. All hashkey columns must be available inside the stage area. This example contains only one foreign hashkey. As a result, there is only one hub connected to the non-historized link.
104
104
-**source_models:**
105
-
- This would create a link loaded from two sources, which is not uncommon. It uses the models `stage_creditcard_transactions` and `stage_purchases`. For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](/docs/General_Usage_Notes/The_rsrc_static_Attribute)
105
+
- This would create a link loaded from two sources, which is not uncommon. It uses the models `stage_creditcard_transactions` and `stage_purchases`. For further information about the rsrc_static attribute, please visit the following page: [rsrc_static Attribute](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md)
106
106
107
107
### PERFORMANCE BOOSTING
108
108
@@ -154,6 +154,6 @@ The High-Water Mark can be disabled safely, but typically would decrease the per
154
154
155
155
We recommend to **try a bit what works best in your environment**. You basically have three options:
156
156
157
-
-**Keep HWM activated**, for multi-source NH Links this would require the [rsrc_static](/docs/General_Usage_Notes/The_rsrc_static_Attribute) to be defined for each source. For single source NH Links, nothing needs to be done, the HWM is activated automatically.
157
+
-**Keep HWM activated**, for multi-source NH Links this would require the [rsrc_static](../../26_general-usage-notes/32_the-rsrc-static-attribute/32_the-rsrc-static-attribute.md) to be defined for each source. For single source NH Links, nothing needs to be done, the HWM is activated automatically.
158
158
-**Disable the HWM entirely.** For multi-source NH Links you just need to not specify the rsrc_static attribute. For single-source NH Links you need to add the parameter disable_hwm=true to your NH Link macro call.
159
159
-**Move the HWM to a previous layer.** First, you apply the previous step to disable the HWM in the NH Link. Then you implement some kind of mechanism in previous dbt layers to ensure that only records newer than what you already processed are available there. This could be especially effective when combining with different materializations of these previous layers.
0 commit comments