Skip to content

Commit 4698dfb

Browse files
fashxpclaude
andauthored
[Docs] Update and refine docs (#657)
* Restructure and correct Data Importer documentation Reorder the docs to follow the Studio "Data Setup" wizard, add a Getting Started walkthrough, and resolve duplicate numeric prefixes (03_Configuration/03_Upgrade, 06_Extending/06_Troubleshooting_FAQ). Correct content that no longer matched the source: - Installation targeted Pimcore 10.5/11 and admin-ui-classic-bundle. composer.json requires pimcore/pimcore ^2026.1 and PHP 8.4/8.5, and registerDependentBundles() wires Datahub automatically. - Element Location Update was documented as applying to all imported objects. Resolver::loadOrCreateAndPrepareElement() applies the creation strategy to new elements and the update strategy to existing ones only. - CSV "Skip First Row" was documented as turning the header row into column names. It only skips the row; the separate "Save Header Name" option does that. - The Extending page described the removed ExtJS plugin API. The frontend is now a Studio UI plugin using dynamic type registries. Document features present in code and UI but not in the docs: the Upload data source, the SQL file format, the String Replace operator, the Many-to-Many Relation data target, the Find or Create Folder and Do Not Create location strategies, As Variant, Include unpublished objects, the ID data index gating Delta Check and Cleanup, the Write If Target Is Not Empty / Write If Source Is Empty flags, Autofill, the one-time job schedule, and PostPreparationEvent. Replace doc/07_Studio.md, a hand-maintained mirror of the Studio REST API, with doc/06_Extending/03_Studio_Integration.md. All 20 Studio controllers are @internal and Studio serves generated OpenAPI docs at /pimcore-studio/api/docs, so the copy could only drift. It already had: it named the PHP constant PLUGIN_DATA_IMPORTER_CONFIG as the permission, whose value is plugin_datahub_config. The replacement keeps the permission model, the API docs location, and the supported ways to drive imports from outside. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019LmFQ3VND6uUqMpWsmppz6 * Replace Admin Classic screenshots with Pimcore Studio captures Recapture the Data Importer screenshots from Pimcore Studio on the public demo. The previous images showed the removed ExtJS Admin Classic interface. Replaced: datasource_asset, datasource_sql, import_preview, resolver_settings, transformation_pipeline, data_target_direct, execution, logging, and a fresher mapping. Added: data_target_types (all four data target types) and processing_settings, neither of which had an image. Correct three details the live UI disproved: - The tab order is General, Data Setup, Execution, Import Logs, Permissions. Getting Started listed Permissions second. - Start Import sits in the Manual Execution panel, not in the Execution Settings panel. - The Data Source Type dropdown lists HTTP before SFTP. Verified live against the demo: the six data sources, the five Data Setup steps, the four data target types including Many-to-Many Relation, the Import Asset options (Parent folder, Use existing, Overwrite existing, Preg match), the Processing Settings groups, and the two schedule types. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019LmFQ3VND6uUqMpWsmppz6 * Restructure installation and complete the Studio event list - Installation becomes a folder with the upgrade notes as its subpage, matching the other bundles. - Delete the Studio Integration page. It described @internal endpoints and duplicated permissions. The supported-interfaces table ("driving imports from outside Pimcore": push data source, the CLI commands, Messenger) was the valuable part and moves to Import Execution Details. - Permissions are now documented once, in the installation chapter, covering the gate, adapter and per-configuration permissions. - The events page named three Studio events as examples. There are fourteen. Document all of them, with payloads read from each event class. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019LmFQ3VND6uUqMpWsmppz6 * Require installing the Datahub bundle, not only loading it The page said Composer pulls Datahub in and the bundle registers it as a dependent bundle, which reads as "Datahub is set up automatically". It is not. registerDependentBundles only loads a bundle into the kernel, and pimcore:bundle:install never cascades. Datahub's own installer is what creates the plugin_datahub_config permission and the Datahub permission category this adapter's controllers require, so without it every non-admin user is locked out. Add the explicit install step for Datahub. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019LmFQ3VND6uUqMpWsmppz6 * Recapture the execution screenshots and document the XLSX sheet default The execution and troubleshooting screenshots were still Admin Classic. Recapture the manual execution, execution status and cron settings in Pimcore Studio, each cropped to its own section. Remove datasource.png, which no page references. Add the XLSX Sheet Name default (Sheet1), which XlsxFileInterpreter applies. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019LmFQ3VND6uUqMpWsmppz6 * Recapture Studio screenshots at the agreed resolution Reshoot the Data Setup screenshots against real demo data at 1440x900 and crop them to the relevant section: - execution / faq_execution_status: taken during an actual import run, so the progress bar and item counter show real values. - datasource_asset, datasource_sql, processing_settings: proper section crops, no cut content. - mapping: wider crop with real source preview values. - transformation_pipeline, data_target_types, data_target_direct: modal-only crops, no background bleed. data_target_direct now shows the Direct target with its own config fields instead of repeating the type dropdown. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Correct the per-configuration permission model The adapter permission and the Permissions tab are alternatives, not layers: Configuration::isAllowed() falls back to plugin_datahub_adapter_dataImporterDataObject only while the permission grid is empty, and ignores it entirely once a single user or role entry exists. Also document plugin_datahub_admin, which bypasses both. * Fix result type and operator details in the mapping reference The Many-to-Many Relation target accepts advancedDataObjectArray, not advancedDataObject; the product's own error message is wrong, supportsType() is authoritative. The Date operator has a Format setting defaulting to Y-m-d rather than requiring one. Add the 10 MB limit both preview file options enforce. * Document which triggers respect the Active flag prepareImport() is called with $ignoreActiveFlag = true only from the manual Start Import action; execute-cron and prepare-import skip inactive configurations. * Drop the duplicate execution status screenshot faq_execution_status.png was byte-identical to execution.png after the recapture. Reference the single remaining file from both pages. * Use Pimcore Studio consistently and wrap the upgrade notes at 120 chars * Address Copilot review findings on the restructured docs Verified each claim against the source before applying: - Custom Strategies: cleanup strategies and element factories have no Studio registry. The cleanup options are hard-coded in processing-settings-step.tsx and the factory is not exposed at all, so mark both backend-only in the extension point table instead of implying a settings form can be registered. - Import Execution Details: preparation does not always queue one item per record. AbstractInterpreter::processImportRow() skips unchanged records when Delta Check is enabled. - Data Sources: drop the recommendation to encode credentials into the HTTP URL. HttpLoader puts the full URL into its InvalidConfigurationException, so the password reaches logs and error reporting. - Getting Started: the Studio operator is labeled "Numeric", not "As Numeric" (dynamic-type-transformer-numeric.tsx). - Processing Settings: DeltaChecker::hasChanged() writes the hash during preparation, before the record is processed, so a failed record is skipped on the next identical run. Document the actual checkpoint. - Troubleshooting: the "command is already running" message is also the normal result of a concurrent run. Require process verification first, note the 24h lock TTL from SequentialProcessQueueCommand, and stop assuming the database lock store. - File Formats: there is no JMESPath validation on save. ConfigurationService persists the array directly; JsonFileInterpreter::setSettings() parses the expression only when the interpreter is instantiated for preview or import. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011gNY3NwTbQPwco5wyWgpE8 --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 25721c4 commit 4698dfb

46 files changed

Lines changed: 1525 additions & 1413 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 37 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,46 @@
11
---
22
title: Data Importer
33
---
4-
4+
55
# Pimcore Data Importer
66

7-
This extension adds a comprehensive import functionality to Pimcore Datahub. It allows importing data from external
8-
sources and adjusting it to Pimcore Data Objects based on a configured mapping without writing any code.
9-
10-
## Features in a Nutshell
11-
- Multiple imports configuration directly in Datahub.
12-
- Data import from various data sources.
13-
- Supported File Formats: `csv`, `xlsx`, `json`, `xml`.
14-
- Strategies configuration for:
15-
- loading existing elements for updating data.
16-
- defining location for newly imported data.
17-
- publishing data.
18-
- cleanup of existing data.
19-
- Mappings definition for adjusting data to Pimcore Data Objects with:
20-
- simple transformations.
21-
- preview of imported data.
22-
- Imports execution directly in Pimcore Datahub or on a regular base via cron definitions.
23-
- Import status updates and extensive logging information.
7+
The Data Importer adds import functionality to Pimcore Datahub. It reads data from external sources and writes it into
8+
Pimcore data objects based on a configured mapping, without writing code.
249

2510
![Mapping](./doc/img/mapping.png)
2611
*Mapping and preview of data*
2712

28-
## Documentation Overview
29-
- [Installation](./doc/01_Installation.md)
30-
- [Configuration](./doc/03_Configuration/README.md)
31-
- [Import Execution Details](./doc/04_Import_Execution_Details.md)
32-
- [Import Progress and Logging](./doc/05_Import_Progress_and_Logging.md)
33-
- [Extending](./doc/06_Extending/README.md)
34-
- [Troubleshooting/FAQ](./doc/06_Troubleshooting_FAQ.md)
35-
36-
## Further Information
37-
On other Pimcore Datahub adapters and export solutions:
38-
- [Datahub (GraphQL API)](https://pimcore.com/docs/platform/Datahub/)
39-
- [Datahub Simple Rest API](https://pimcore.com/docs/platform/Datahub_Simple_Rest/)
40-
- [Datahub File Export](https://pimcore.com/docs/platform/Datahub_File_Export/)
41-
- [Datahub Productsup](https://pimcore.com/docs/platform/Datahub_Productsup/)
13+
## Features
14+
15+
- Import configurations managed directly in Datahub, edited in Pimcore Studio.
16+
- Six data sources: asset, upload, HTTP, SFTP, push endpoint, and SQL.
17+
- Five file formats: CSV, JSON, XML, XLSX, and SQL query results.
18+
- Resolver strategies that decide whether a record updates an existing data object or creates a new one, where it is
19+
stored, and whether it is published.
20+
- A mapping editor with a live preview of the source data and of every transformation step.
21+
- Delta check to skip unchanged records, and cleanup to remove data objects that left the source.
22+
- Manual, scheduled, command-line and push-triggered execution.
23+
- Progress tracking and detailed import logs.
24+
25+
## Documentation
26+
27+
Start here:
28+
29+
- [Installation](./doc/01_Installation/README.md): install the bundle and set up queue processing.
30+
- [Getting Started](./doc/02_Getting_Started.md): build a first import end to end.
31+
32+
Reference:
33+
34+
- [Configuration](./doc/03_Configuration/README.md): every option of an import configuration.
35+
- [Import Execution Details](./doc/04_Import_Execution_Details.md): what happens between the trigger and the data.
36+
- [Import Progress and Logging](./doc/05_Import_Progress_and_Logging.md): watch and audit an import.
37+
- [Extending](./doc/06_Extending/README.md): custom strategies and events.
38+
- [Troubleshooting / FAQ](./doc/08_Troubleshooting_FAQ.md): common problems.
39+
- [Upgrade Notes](./doc/01_Installation/01_Upgrade.md): breaking changes per release.
40+
41+
## Other Datahub Adapters
42+
43+
- [Datahub (GraphQL API)](https://github.com/pimcore/data-hub/blob/2026.x/doc/01_Installation_and_Upgrade/README.md)
44+
- [Datahub Simple REST API](https://github.com/pimcore/data-hub-simple-rest/blob/2026.x/doc/01_Installation/README.md)
45+
- [Datahub File Export](https://github.com/pimcore/data-hub-file-export/blob/2026.x/doc/01_Installation/README.md)
46+
- [Datahub Productsup](https://github.com/pimcore/data-hub-productsup/blob/2026.x/doc/01_Installation/README.md)

doc/01_Installation.md

Lines changed: 0 additions & 103 deletions
This file was deleted.
Lines changed: 31 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
1-
# Update Notes
1+
---
2+
title: Upgrade Notes
3+
description: Breaking changes and migration steps per release.
4+
---
5+
6+
# Upgrade Notes
27

38
## Upgrade to 2026.1.0
49

@@ -9,15 +14,19 @@
914

1015
### Removed Admin Classic / ExtJS UI
1116

12-
- Removed `pimcore/admin-ui-classic-bundle` dependency. The bundle no longer implements `PimcoreBundleAdminClassicInterface` or uses `BundleAdminClassicTrait`.
13-
- Removed all ExtJS-based JavaScript and CSS paths (`getCssPaths()`, `getJsPaths()` methods removed from `PimcoreDataImporterBundle`).
17+
- Removed the `pimcore/admin-ui-classic-bundle` dependency. The bundle no longer implements
18+
`PimcoreBundleAdminClassicInterface` and no longer uses `BundleAdminClassicTrait`.
19+
- Removed all ExtJS-based JavaScript and CSS paths. `getCssPaths()` and `getJsPaths()` were removed from
20+
`PimcoreDataImporterBundle`.
1421
- `PimcoreAdminBundle` is no longer registered as a dependent bundle.
15-
- The Admin UI configuration panel is now implemented using the Pimcore Studio UI.
16-
- `Pimcore\Bundle\AdminBundle\Helper\QueryParams` is no longer used; replaced by `DataTypeServiceInterface::extractSortingSettings()`.
22+
- The configuration panel is now implemented in Pimcore Studio.
23+
- `Pimcore\Bundle\AdminBundle\Helper\QueryParams` is no longer used. It was replaced by
24+
`DataTypeServiceInterface::extractSortingSettings()`.
1725

18-
### Studio UI
26+
### Pimcore Studio
1927

20-
- Added Studio UI implementation for the Data Importer configuration panel (tabs: Data Setup, Execution, Import Logs).
28+
- Added the Pimcore Studio implementation of the Data Importer configuration panel, with the tabs Data Setup,
29+
Execution and Import Logs.
2130
- Studio frontend assets are now shipped with the bundle.
2231

2332
### Namespace Changes
@@ -26,23 +35,28 @@
2635

2736
### Messenger Transport Configuration
2837

29-
- The messenger transport DSN is now configurable via the `%pimcore.messenger.transport_dsn_prefix%` container parameter instead of being hardcoded to `doctrine://default`. This allows the installer to wire the transport DSN from environment variables (e.g. `PIMCORE_MESSENGER_TRANSPORT_DSN_PREFIX`).
38+
- The messenger transport DSN is configurable via the `%pimcore.messenger.transport_dsn_prefix%` container parameter
39+
instead of being hardcoded to `doctrine://default`. The installer can now wire the transport DSN from environment
40+
variables such as `PIMCORE_MESSENGER_TRANSPORT_DSN_PREFIX`.
3041

3142
### Interface & Return Type Changes
3243

33-
- `DataTypeServiceInterface`: Added new method `extractSortingSettings(?string $sort): array` to the interface. Any custom implementations of `DataTypeServiceInterface` must implement this method.
44+
- `DataTypeServiceInterface`: added the method `extractSortingSettings(?string $sort): array`. Custom implementations
45+
of the interface must implement it.
3446
- `CronValidationResponse::isValid(): bool` renamed to `CronValidationResponse::getIsValid(): bool`.
3547
- `ImportProgressResponse::isRunning(): bool` renamed to `ImportProgressResponse::getIsRunning(): bool`.
36-
- `PimcoreDataImporterBundle::getInstaller()` return type changed from `?InstallerInterface` to `InstallerInterface` (non-nullable).
48+
- `PimcoreDataImporterBundle::getInstaller()` return type changed from `?InstallerInterface` to
49+
`InstallerInterface` (non-nullable).
3750

3851
### Class Visibility & Finalization
3952

4053
- Many classes have been marked `final` and/or `@internal`. Custom subclasses of these classes are no longer supported:
41-
- `CronScheduler`, `JobScheduler`, `SchedulerFactory` — marked `final` and `@internal`
42-
- `PreviewData` — marked `final` and `@internal`
43-
- `PimcoreDataImporterBundle` — marked `final`
44-
- Various exception classes (`InvalidScheduleException`, etc.) — marked `final`
45-
- Several `protected` methods on final classes have been changed to `private`. If you were overriding these methods in subclasses, you must refactor.
54+
- `CronScheduler`, `JobScheduler`, `SchedulerFactory`: marked `final` and `@internal`
55+
- `PreviewData`: marked `final` and `@internal`
56+
- `PimcoreDataImporterBundle`: marked `final`
57+
- Various exception classes (`InvalidScheduleException`, etc.): marked `final`
58+
- Several `protected` methods on final classes were changed to `private`. Subclasses overriding them have to be
59+
refactored.
4660
- `ConfigDataObjectController::saveAction()` return type changed from `?JsonResponse` to `JsonResponse` (non-nullable).
4761
- `Installer::getLastMigrationVersionClassName()` return type changed from `?string` to `string` (non-nullable).
4862

@@ -56,7 +70,8 @@
5670

5771
### API Schema Changes
5872

59-
- `UnitDataResponse`: Schema property renamed from `UnitList` to `unitList` (camelCase). If you consume this API response, update your client accordingly.
73+
- `UnitDataResponse`: the schema property was renamed from `UnitList` to `unitList`. Update clients consuming this
74+
response.
6075

6176
## Update to Version 1.11
6277

0 commit comments

Comments
 (0)