complete initial draft

Author: rpietzsch

data/integrations.yml

@@ -264,26 +264,24 @@ integrations:
   - name: PostgreSQL
     icon: ":simple-postgresql:"
     content: |
-      PostgreSQL can be accessed with the {{p.Jdbc}} dataset and a
-      [JDBC driver](https://central.sonatype.com/artifact/org.postgresql/postgresql/versions).
+      PostgreSQL can be accessed with the {{p.Jdbc}} dataset (JDBC driver included).
 
   - name: MariaDB
     icon: ":simple-mariadb:"
     content: |
       MariaDB can be accessed with the {{p.Jdbc}} dataset and a
-      [JDBC driver](https://central.sonatype.com/artifact/org.mariadb.jdbc/mariadb-java-client/overview).
+      [Custom JDBC driver](https://central.sonatype.com/artifact/org.mariadb.jdbc/mariadb-java-client/overview).
 
   - name: SQLite
     icon: ":simple-sqlite:"
     content: |
       SQLite can be accessed with the {{p.Jdbc}} dataset and a
-      [JDBC driver](https://central.sonatype.com/artifact/org.xerial/sqlite-jdbc).
+      [Custom JDBC driver](https://central.sonatype.com/artifact/org.xerial/sqlite-jdbc).
 
   - name: MySQL
     icon: ":simple-mysql:"
     content: |
-      MySQL can be accessed with the {{p.Jdbc}} dataset and a
-      [JDBC driver](https://central.sonatype.com/artifact/org.mariadb.jdbc/mariadb-java-client/overview).
+      MySQL can be accessed with the {{p.Jdbc}} dataset (JDBC driver included).
 
   - name: Hive
     icon: ":simple-apachehive:"
@@ -293,14 +291,12 @@ integrations:
   - name: Microsoft SQL
     icon: ":material-microsoft:"
     content: |
-      The Microsoft SQL Server can be accessed with the {{p.Jdbc}} dataset and a
-      [JDBC driver](https://central.sonatype.com/artifact/com.microsoft.sqlserver/mssql-jdbc).
+      The Microsoft SQL Server can be accessed with the {{p.Jdbc}} dataset (JDBC driver included).
 
   - name: Snowflake
     icon: ":simple-snowflake:"
     content: |
-      Snowflake can be accessed with the {{p.SnowflakeJdbc}} dataset and a
-      [JDBC driver](https://central.sonatype.com/artifact/net.snowflake/snowflake-jdbc).
+      Snowflake can be accessed with the {{p.SnowflakeJdbc}} dataset (JDBC driver included).
 
   - name: pgvector
     icon: ":black_large_square:"
@@ -312,13 +308,14 @@ integrations:
     icon: ":simple-trino:"
     content: |
       [Trino](https://github.com/trinodb/trino) can be access with the
-      {{p.Jdbc}} dataset and a [JDBC driver](https://trino.io/docs/current/client/jdbc.html).
+      {{p.Jdbc}} dataset and a [Custom JDBC driver](https://trino.io/docs/current/client/jdbc.html).
+      Please have a look at
+      [Setup and use of JDBC Drivers](../../deploy-and-configure/configuration/dataintegration/jdbc/)
 
   - name: Databricks
     icon: ":simple-databricks:"
     content: |
       [Databricks](http://databricks.com/) can be accessed with the {{p.Jdbc}} dataset and a
       [Custom JDBC driver](https://github.com/databricks/databricks-jdbc).
       Please have a look at
-      [Setup additional JDBC Drivers](../../deploy-and-configure/configuration/dataintegration/jdbc/).
-
+      [Setup and use of JDBC Drivers](../../deploy-and-configure/configuration/dataintegration/jdbc/).

docs/build/integrations/index.md

@@ -54,7 +54,7 @@ to interact with any [Azure AI Foundry provided Large Language Models](https://a
     [Databricks](http://databricks.com/) can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a
 [Custom JDBC driver](https://github.com/databricks/databricks-jdbc).
 Please have a look at
-[Setup additional JDBC Drivers](../../deploy-and-configure/configuration/dataintegration/jdbc/).
+[Setup and use of JDBC Drivers](../../deploy-and-configure/configuration/dataintegration/jdbc/).
 
 
 -   :material-email-outline:{ .lg .middle } eMail / SMTP
@@ -145,7 +145,7 @@ GraphDB can be used as the integrated Quad Store as well.
     ---
 
     MariaDB can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a
-[JDBC driver](https://central.sonatype.com/artifact/org.mariadb.jdbc/mariadb-java-client/overview).
+[Custom JDBC driver](https://central.sonatype.com/artifact/org.mariadb.jdbc/mariadb-java-client/overview).
 
 
 -   :simple-mattermost:{ .lg .middle } Mattermost
@@ -160,16 +160,14 @@ the [Send Mattermost messages](../../build/reference/customtask/cmem_plugin_matt
 
     ---
 
-    The Microsoft SQL Server can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a
-[JDBC driver](https://central.sonatype.com/artifact/com.microsoft.sqlserver/mssql-jdbc).
+    The Microsoft SQL Server can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset (JDBC driver included).
 
 
 -   :simple-mysql:{ .lg .middle } MySQL
 
     ---
 
-    MySQL can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a
-[JDBC driver](https://central.sonatype.com/artifact/org.mariadb.jdbc/mariadb-java-client/overview).
+    MySQL can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset (JDBC driver included).
 
 
 -   :simple-neo4j:{ .lg .middle } Neo4J
@@ -256,8 +254,7 @@ using the [Search Vector Embeddings](../../build/reference/customtask/cmem_plugi
 
     ---
 
-    PostgreSQL can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a
-[JDBC driver](https://central.sonatype.com/artifact/org.postgresql/postgresql/versions).
+    PostgreSQL can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset (JDBC driver included).
 
 
 -   :other-powerbi:{ .lg .middle } PowerBI
@@ -317,8 +314,7 @@ execute a [SOQL query (Salesforce)](../../build/reference/customtask/cmem_plugin
 
     ---
 
-    Snowflake can be accessed with the [Snowflake SQL endpoint](../../build/reference/dataset/SnowflakeJdbc.md) dataset and a
-[JDBC driver](https://central.sonatype.com/artifact/net.snowflake/snowflake-jdbc).
+    Snowflake can be accessed with the [Snowflake SQL endpoint](../../build/reference/dataset/SnowflakeJdbc.md) dataset (JDBC driver included).
 
 
 -   :simple-apachespark:{ .lg .middle } Spark
@@ -333,7 +329,7 @@ execute a [SOQL query (Salesforce)](../../build/reference/customtask/cmem_plugin
     ---
 
     SQLite can be accessed with the [Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a
-[JDBC driver](https://central.sonatype.com/artifact/org.xerial/sqlite-jdbc).
+[Custom JDBC driver](https://central.sonatype.com/artifact/org.xerial/sqlite-jdbc).
 
 
 -   :material-ssh:{ .lg .middle } SSH
@@ -360,7 +356,9 @@ Tentris can be used as the integrated Quad Store as well (beta).
     ---
 
     [Trino](https://github.com/trinodb/trino) can be access with the
-[Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a [JDBC driver](https://trino.io/docs/current/client/jdbc.html).
+[Remote SQL endpoint](../../build/reference/dataset/Jdbc.md) dataset and a [Custom JDBC driver](https://trino.io/docs/current/client/jdbc.html).
+Please have a look at
+[Setup and use of JDBC Drivers](../../deploy-and-configure/configuration/dataintegration/jdbc/)
 
 
 -   :black_large_square:{ .lg .middle } Virtuoso

docs/deploy-and-configure/configuration/dataintegration/jdbc/index.md

@@ -3,8 +3,73 @@ tags:
     - Configuration
     - JDBC
 ---
-# Setup additional JDBC Drivers
+# Setup and use of JDBC Drivers
 
-In addition to the shipped default JDBC drivers (MariaDB, PostgreSQL, ...), Corporate Memory can be extended with custom JDBC drivers.
-This page describes the steps how to configure and use these additional drivers.
+Corporate Memory allows to use JDBC to connect to Database Management Systems (DBMS).
+We ship some drivers as part of our platform.
+Custom drivers can be added and used, too.
 
+!!! info "References"
+
+    For more technical details also check the following reference pages:
+
+    -   [Remote SQL endpoint](../../../../build/reference/dataset/Jdbc.md) and
+    -   [Snowflake SQL endpoint](../../../../build/reference/dataset/SnowflakeJdbc.md).
+
+## Embedded JDBC Driver
+
+Our platform contains and ship the following JDBC drivers:
+
+-   PostgreSQL (`postgresql v42.7.10`)
+-   MariaDB (includes support for MySQL, `mariadb-java-client v3.5.7`)
+-   Microsoft SQL Server (`mssql-jdbc v13.2.1.jre11`)
+-   Snowflake (`snowflake-jdbc v3.28.0`)
+
+## Custom JDBC Driver
+
+In addition to the shipped default JDBC drivers custom JDBC drivers can be registered.
+The following steps describe how they are configured.
+
+### Download Custom JDBC Driver
+
+Download the respective JDBC driver for the Database Management Systems that need to be connected.
+[Integrations](../../../../build/integrations/index.md) provides links for well know system and at the same time lists those that are in active use with Corporate Memory.
+
+### Provide a Custom JDBC Driver
+
+Consult your solutions manager or DevOps specialist for options how to copy / inject additional files, namely the JDBC driver `jar`, into a Corporate Memory deployment.
+Depending on the specific deployment model used the suitable options differ, some possibilities:
+
+-   the docker compose package `cmem-orchestration` mounts the folder `./conf/dataintegration/plugin/` into the DataIntegration container (this location is assumed in the configuration snippets below and mapped to the container internal path `/opt/cmem/eccenca-DataIntegration/dist/etc/dataintegration/conf/plugin/`),
+-   a dedicated Build project into which the driver jar files are uploaded as project (file) resources,
+-   dedicated file or resource mounts in a docker compose or helm/kubernets configuration.
+
+## Driver Registration
+
+A custom JDBC driver needs to be registered in the DataIntegration configuration file, `dataintegration.conf`, `spark.sql.options` section.
+The following example demonstrates how to register the custom JDBC driver for the Databricks data lakehouse:
+
+```conf
+…
+spark.sql.options {
+  …
+  # driver name
+  jdbc.drivers = "databricks"
+  # path to the jar in the docker container
+  jdbc.databricks.jar =  "/opt/cmem/eccenca-DataIntegration/dist/etc/dataintegration/conf/plugin/DatabricksJDBC.jar"
+  # class name
+  jdbc.databricks.name = "com.databricks.client.jdbc.Driver"
+  …
+}
+…
+```
+
+## Use the driver
+
+JDBC drivers are used via the **Remote SQL endpoint** (or **Snowflake SQL endpoint**) dataset type.
+
+![](jdbc-dataset.png){ class="bordered" width="85%" }
+
+They can be configured in the configuration dialog of the dataset, consult the documentation of your DBMS / JDBC driver for details about the [JDBC connection string](https://www.baeldung.com/java-jdbc-url-format).
+
+![](jdbc-config-databricks.png){ class="bordered" width="100%" }