You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[2. Create a database in Snowflake](#2-create-a-database-in-snowflake)
27
-
-[3. Create a project in eccenca Corporate Memory](#3-create-a-project-in-eccenca-corporate-memory)
28
-
-[4. Create a transformation to build mapping rules](#4-create-a-transformation-to-build-mapping-rules)
29
-
-[5. Create a knowledge graph](#5-create-a-knowledge-graph)
25
+
-[1. Create a database in Snowflake](#1-create-a-database-in-snowflake)
26
+
-[2. Create a project in eccenca Corporate Memory](#2-create-a-project-in-eccenca-corporate-memory)
27
+
-[3. Create a transformation to build mapping rules](#3-create-a-transformation-to-build-mapping-rules)
28
+
-[4. Create a knowledge graph](#4-create-a-knowledge-graph)
30
29
31
30
## Sample material
32
31
33
32
The following material is used in this tutorial, you should download the files and have them at hand throughout the tutorial:
34
33
35
34
- The product data vocabulary [products_vocabulary.nt](products_vocabulary.nt)
36
35
37
-
## 1. Configure Custom JDBC Driver
38
-
39
-
To connect to the Snowflake cloud data warehouse a JDBC driver is required.
40
-
41
-
The SQL-Dataset of eccenca Corporate Memory can access any database that offers a (supported/tested) JDBC driver.
42
-
This happens partly via Apache Spark SQL but requires no Spark specific configuration for eccenca Corporate Memory.
43
-
For Snowflake supported (e.g. 3.13.34) JDBC drivers can be found at:
44
-
45
-
-[MVN Repository](https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/) ([direct jar download](https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/3.13.34/snowflake-jdbc-3.13.34.jar))
46
-
- to verify and build yourself: [github.com/snowflakedb/snowflake-jdbc](https://github.com/snowflakedb/snowflake-jdbc)
47
-
48
-
To use the driver it needs to be part of the classpath of eccenca Build (DataIntegration).
49
-
That can be achieved in multiple ways but it is recommended to register the driver via the `dataintegration.conf` configuration file.
50
-
51
-
There are 3 settings to specify:
52
-
53
-
1. The most important is to add the driver name to `spark.sql.options.jdbc.drivers` - a comma separated list of drivers.
54
-
The names in this list are the same as the database name in its JDBC-connection string (i.e. `snowflake` for its connection URL which looks like `jdbc:snowflake://<account_identifier>.snowflakecomputing.com/?<connection_params>`).
55
-
2. Snowflake specific property for the jar file location: `spark.sql.options.jdbc.snowflake.jar="/location/for/snowflake-*-jdbc.jar"`
56
-
3. Snowflake specific property for the driver class name: `spark.sql.options.jdbc.snowflake.name="com.snowflake.client.jdbc.SnowflakeDriver"`
- Click on scheme **products_vocabulary** on the left side of the page then click on **Create** on the right side of the page, then click on **Table**, then select then **Standard**.
- Click on schema name **products_vocabulary** on the left side of the page and type the **sql query** for creating a table in the center, then click on **Run** on the right side of the page.
104
67
105
-
You can create the _PRODUCT_ table with the following SQL query:
106
-
107
-
```sql
108
-
CREATETABLEproduct(
109
-
product_id VARCHAR(12) NOT NULLPRIMARY KEY
110
-
,product_name VARCHAR(50) NOT NULL
111
-
,height INTEGERNOT NULL
112
-
,width INTEGERNOT NULL
113
-
,depth INTEGERNOT NULL
114
-
,weigth INTEGERNOT NULL
115
-
,product_manager VARCHAR(50) NOT NULL
116
-
,price VARCHAR(10) NOT NULL
117
-
) ;
118
-
```
68
+
You can create the _PRODUCT_ table with the following SQL query:
69
+
70
+
```sql
71
+
CREATETABLEproduct(
72
+
product_idVARCHAR(12) NOT NULLPRIMARY KEY
73
+
, product_name VARCHAR(50) NOT NULL
74
+
, height INTEGERNOT NULL
75
+
, width INTEGERNOT NULL
76
+
, depth INTEGERNOT NULL
77
+
, weigth INTEGERNOT NULL
78
+
, product_manager VARCHAR(50) NOT NULL
79
+
, price VARCHAR(10) NOT NULL
80
+
) ;
81
+
```
119
82
120
83
- Type or copy the **SQL** query for creating a database in the table that is created, then click on**Run**.
In the **Worksheets** view (you might need to create a new worksheet), select the **product** database, the **products_vocabulary** schema and finally the **product** table.
125
88
Here you can populate some test data with the following SQL query:
@@ -1127,7 +1090,7 @@ Here you can populate some test data with the following SQL query:
- Type the name **product**in the **Label** field, in the **INPUT TASK Dataset**select**Product Table (JDBC)**andin the **Type** field select**table**.
-Select the **target property** according to transformation requirements, for example name, id, etc., then select the **value path** according to the target property as the product name, product id etc.
1268
1211
This step will help in mapping the data from the source to the target property.
0 commit comments