Skip to content

Commit f9adace

Browse files
rpietzschrpietzsch
committed
add DataIntegration
1 parent d6646c7 commit f9adace

1 file changed

Lines changed: 77 additions & 5 deletions

File tree

  • docs/release-notes/corporate-memory-25-3

‎docs/release-notes/corporate-memory-25-3/index.md‎

Lines changed: 77 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -42,19 +42,79 @@ More detailed information for this release is provided in the next sections.
4242

4343
## eccenca DataIntegration v25.3.0
4444

45-
We are excited to announce the release of DataIntegration v25.3.0, which introduces …
45+
We are excited to announce the release of DataIntegration v25.3.0, bringing explicit Databricks connectivity, richer dataset and plugin APIs, streamlined Python plugin tooling, and broad workflow and editor reliability improvements.
4646

4747
**v25.3.0 of DataIntegration adds the following new features:**
4848

49-
- …
49+
- **Data sources and workflow operators:**
50+
- Added explicit Databricks support.
51+
- Added an operator that allows arbitrary SQL statements as input for `SqlDataset`.
52+
- Added an endpoint for browsing files to simplify file selection.
53+
- Added a documentation link to the comparison task node to surface guidance in-context.
54+
- Added the `**` operator to the JSON dataset so all child nodes can be selected recursively.
55+
- **Text preparation:**
56+
- Introduced a new default stop word remover and improved the existing stop word transformers.
57+
- **Plugin APIs:**
58+
- Added the `pluginType` attribute to the `/core/plugins` endpoint response.
59+
- Added a `/pluginTypes` endpoint to list all plugin types.
60+
- Added the `backendType` attribute to plugin JSON payloads to highlight whether a plugin is native or Python-based.
61+
- **Python plugin development:**
62+
- Added `autocomplete_query` to `ParameterType`, providing a more flexible alternative to the legacy `autocomplete` function.
5063

5164
**v25.3.0 of DataIntegration introduces the following changes:**
5265

53-
- …
66+
- **Data connectors and storage workflows:**
67+
- Adapted the Google Drive Spreadsheet dataset tests to accommodate Google’s changed behavior.
68+
- Projects imported into a knowledge graph backend now delete the target graph before import to avoid stale data.
69+
- **Mapping Creator experience:**
70+
- Mapping rules opened from the Mapping Creator no longer need to be saved immediately.
71+
- Newly created mapping rules that are not yet persisted can now be edited directly in the Mapping Creator.
72+
- **UI and documentation improvements:**
73+
- Improved the formatting of operator examples for better readability.
74+
- Updated the application to the new color palette.
75+
- Made decoding underscores to spaces an explicit option in the **Strip URI prefix** transformer.
76+
- **Python subsystem and plugin lifecycle:**
77+
- Loading a Python plugin that reuses an existing identifier now records an error in the discovery result.
78+
- The `/listPlugins` and `/updatePlugins` endpoints now return HTTP 500 if at least one plugin fails to load (previously HTTP 200).
79+
- Upgraded the embedded Python subsystem:
80+
- Installation now relies on uv 0.9.2.
81+
- Updated the packaged Python distribution to Python 3.13.8.
82+
- Replaced the Red Hat UBI system Python with a uv-installed runtime.
83+
- Uses a read-only virtual environment from `/usr/local/venv`.
84+
- Plugin package installation now uses uv instead of pip.
5485

5586
**v25.3.0 of DataIntegration ships the following fixes:**
5687

57-
- …
88+
- **Workflow editor and execution:**
89+
- Edge arrows are displayed again in the workflow editor via React Flow v12.
90+
- Nested workflows no longer expose data input ports.
91+
- Workflow nodes loaded from the store retain their original positions.
92+
- Multiple operators created via “Connect to newly created operator” no longer overlap.
93+
- Fixed the display of workflow reports inside the modal dialog.
94+
- The workflow report now correctly marks failed workflows with a red danger icon and shows “Execution failed” on failed operators.
95+
- Transform execution reports no longer show successful executions for rules that were not run.
96+
- Fixed graph uploads for large files or chunk sizes.
97+
- Exporting a workspace without project resources now succeeds.
98+
- Downloading a project/workspace with empty files stored in AWS S3 works again.
99+
- Fixed the display of workflow reports in Plugin Custom Action Reports so the code area stays within bounds.
100+
- **Mapping and rule authoring:**
101+
- Restrictions no longer expand the wrong SPARQL pattern.
102+
- URI rules are no longer created automatically when the default rule is opened in the rule editor.
103+
- Mapping Creator undo/redo actions behave as expected.
104+
- Suggest mappings now adds all selected rows, not just the ones currently visible after a text filter.
105+
- Rule editor nodes are no longer copied when text is selected in Firefox.
106+
- **Python plugins and orchestration:**
107+
- Listing Python plugins no longer fails when the orchestration feature `enable-extension EXTENSION=plugins` is used.
108+
- **Transformations, datasets, and templates:**
109+
- JSON `#arrayText` now returns an empty array representation for empty arrays.
110+
- Jinja templates no longer error when the `loop` variable is used inside functions within `for` loops.
111+
- Previewing parameter descriptions no longer renders broken Markdown.
112+
- Use `FROM` instead of `GRAPH` in the vocabulary loader to respect graph imports.
113+
- **Graph and project handling:**
114+
- When importing into a knowledge graph backend, the graph is only deleted if it exists.
115+
- **User interaction fixes:**
116+
- Global and workflow-specific hotkeys no longer trigger when another modal is open.
117+
- Display the template toggle consistently in the artefact creation dialog.
58118

59119
## eccenca Explore v25.3.0
60120

@@ -135,7 +195,19 @@ We are excited to announce cmemc v25.5.0, which introduces new features, improve
135195

136196
### eccenca DataIntegration
137197

138-
- …
198+
- **Updated plugin identifiers:** To guarantee unique plugin identifiers, several configuration keys changed. The updates only affect configuration references—existing projects remain intact.
199+
- `workspace.provider.plugin`: `inMemory` → `inMemoryWorkspaceProvider`, `file` → `fileWorkspaceProvider`.
200+
- `workspace.repository.plugin`: `inMemory` → `inMemoryResourceRepository`, `file` → `sharedFileRepository`.
201+
- `workspace.reportManager.plugin`: `inMemory` → `inMemoryExecutionReportManager`, `file` → `fileExecutionReportManager`.
202+
- **Python autocomplete:** Dropped support for autocomplete functions that omit the `depend_on_parameter_values` parameter. This was deprecated in 23.1; earlier versions logged warnings referencing “legacy autocomplete function”.
203+
- **Python sub-system upgrade:** The Python runtime now uses Python 3.13 with uv-based package management.
204+
- If you previously set `PIP_INDEX_URL`/`PIP_EXTRA_INDEX_URL`, switch to `UV_DEFAULT_INDEX`/`UV_EXTRA_INDEX_URL`.
205+
- Custom CA bundles now reside at `/usr/local/venv/lib/python3.13/site-packages/certifi/cacert.pem`.
206+
- Clean up the complete `python-packages` directory before or directly after upgrading (e.g., via `cmemc admin workspace python uninstall --all`).
207+
- Re-install required Python packages afterwards and ensure they are Python 3.13 compatible.
208+
- Please note that your packages need to be python 3.13 ready.
209+
- All eccenca packages are published as a new python 3.13 ready minor version.
210+
- If you maintain custom plugins using the [cmem-plugin-template](https://github.com/eccenca/cmem-plugin-template), update the template via copier as [described in the repository documentation](https://github.com/eccenca/cmem-plugin-template?tab=readme-ov-file#template-updates).
139211

140212
### eccenca Explore
141213

0 commit comments

Comments
 (0)