fix: return self::FAILURE from CleanupUnusedIndicesCommand on exception

doc/02_Configuration/03_Index_Management.md

@@ -1,83 +1,64 @@
----
-title: Index Management
-description: Manage search indices for the Generic Data Index, including creation, updates, queue processing, and deployment.
----
-
 # Index Management
 
-The Generic Data Index must index all assets, data objects, and documents
-to power search and listing features in Pimcore.
-
-## Console Commands Overview
+It is important to index all assets and data object in Pimcore in order to be able to use the search and listing features powered by the Generic Data Index bundle.
 
-| Command | Description |
-|---------|-------------|
-| `generic-data-index:update:index` | Update index mappings and queue all elements for reindex from the database |
-| `generic-data-index:update:index -r` | Delete and recreate indices, then queue all elements |
-| `generic-data-index:reindex` | Native search engine reindex (reorganizes data within existing indices, no database read) |
-| `generic-data-index:deployment:reindex` | Update indices only for class definitions changed since the last deployment |
+## Define Index Prefix
 
-## Index Prefix
+To avoid duplicate names or index interferences in your search engine, it is necessary to define an index name prefix, which is added to all indices created by Generic Data Index. 
+Default one is `pimcore_`.
 
-Define an index name prefix to avoid naming collisions in shared search engine clusters.
-The default prefix is `pimcore_`.
+This can be done by defining following configuration:
 
-```yaml
+```yaml 
 pimcore_generic_data_index:
     index_service:
         client_params:
-            index_prefix: 'my_prefix'
+            index_prefix: 'my_prefix' # Prefix for all index names created by Generic Data Index
 ```
 
 ## Created Indices
 
-The Generic Data Index creates the following indices:
+The Generic Data Index generates indices for the following entities:
 
-- **Assets** - one alias and one index
-- **Data objects** - one alias and one index per class definition
+* Assets search index (one alias and one index)
+* Data objects search index (one alias and one index per class definition)
 
-Each index uses an alias (e.g. `<prefix>_asset`) pointing to the current index
-(e.g. `<prefix>_asset-odd`). The alias name stays constant; the backing index alternates
-between `-odd` and `-even` suffixes during reindexing (see
-[Updating index structure](#updating-index-structure) below).
+For the asset and data object indices the Generic Data Index uses an alias (e.g. `<index_prefix>_asset`) that points to the
+most current index (e.g. `<index_prefix>_asset-odd`). The alias name always stays the same, the index names alternate
+between `-odd` and `-even` suffix. For more details also see 'Updating index structure for data indices' in the next section.
 
 ## Keeping Indices Up to Date
 
-Create and update indices with:
+The element search indices need to be created via the following console commands:
 
-```bash
+```
+# create/update all indices + their mappings and add all items to the index queue
 bin/console generic-data-index:update:index
 ```
 
-This command creates the indices and queues all assets and data objects for indexing.
-The Symfony Messenger `pimcore_generic_data_index_queue` transport processes the queue.
+The command will create the indices and add all assets and data objects to the index queue. The queue will be processed by Symfony messenger workers (`pimcore_generic_data_index_queue` queue).
 
-### Index Refresh
+### Refreshing of the index
 
-By default, the index refreshes after each bulk operation since items are processed
-asynchronously via Symfony Messenger.
+By default, the index queue is refreshed after each bulk operation as the items are processed asynchronously with the Symfony messenger.
+If you want to perform index refresh immediately you can use enable synchronous processing by injecting the `SynchronousProcessingServiceInterface` and calling `enable()` method.
 
-To force synchronous processing (immediate refresh), inject
-`SynchronousProcessingServiceInterface` and call `enable()`:
+Available methods:
+- `enable()`: enable synchronous processing
+- `disable()`: disable synchronous processing
+- `isEnabled()`: check if synchronous processing is enabled
 
-| Method | Description |
-|--------|-------------|
-| `enable()` | Enable synchronous processing |
-| `disable()` | Disable synchronous processing |
-| `isEnabled()` | Check current mode |
+### Index Queue Options
 
-### Queue Options
+The indexing queue considers the following options:
 
-Configure the indexing queue batch behavior:
+- **worker_count** (default 1): number of messenger workers to process the queue. Set this to the actual used parallel number of `messenger:consume` workers to improve the calculation of items per batch.
+- **min_batch_size** (default 5): minimum number of items to process in one batch (when using multiple workers) 
+- **max_batch_size** (default 400): maximum number of items to process in one batch
 
-| Option | Default | Description |
-|--------|---------|-------------|
-| `worker_count` | 1 | Number of parallel `messenger:consume` workers. Improves batch size calculation. |
-| `min_batch_size` | 5 | Minimum items per batch (relevant with multiple workers) |
-| `max_batch_size` | 400 | Maximum items per batch |
+Based on this configuration, the queue will be processed in batches of `min_batch_size` to `max_batch_size` items. The number of items per batch is calculated based on the number of workers and the number of items in the queue.
 
-The queue calculates batch sizes dynamically between `min_batch_size` and `max_batch_size`
-based on the number of workers and queue depth.
+Sample configuration:
 
 ```yaml
 pimcore_generic_data_index:
@@ -88,67 +69,76 @@ pimcore_generic_data_index:
             max_batch_size: 400
 ```
 
-### Related Elements
+#### Related elements
 
-Updating an element automatically enqueues its related elements for reindexing.
-By default, this runs asynchronously through Symfony Messenger.
+The indexing queue is automatically populated whenever an element undergoes an update operation. This process includes not only the modified element itself but also any related elements. By default, this indexing occurs asynchronously through Symfony Messenger.
 
-For immediate processing, use `SynchronousProcessingRelatedIdsServiceInterface`:
+For scenarios requiring immediate processing, you can temporarily switch to synchronous mode by utilizing the `SynchronousProcessingRelatedIdsServiceInterface`.
+
+Available methods are:
 
 | Method | Description |
 |--------|-------------|
-| `enable()` | Activate synchronous processing |
-| `disable()` | Revert to asynchronous processing |
-| `isEnabled()` | Return current processing mode |
+| `enable()` | Activates synchronous processing mode |
+| `disable()` | Reverts to asynchronous processing mode |
+| `isEnabled()` | Returns the current processing mode status |
 
 :::info
 
-`SynchronousProcessingRelatedIdsServiceInterface` does not affect delete operations.
-Deletes always process synchronously.
+Currently the `SynchronousProcessingRelatedIdsServiceInterface` interface does not influence the behavior of delete operations. They are always processed synchronously.
 
 :::
 
+
 ### Repairing Indices
 
-To delete and recreate an index from the Pimcore database (e.g. after an incompatible
-mapping change), pass the `-r` option:
+Sometimes it might be needed to delete and recreate the index from the Pimcore database
+(for example if the mapping changed and cannot be updated).
 
-```bash
+Do this with the index update command and pass `-r` option (which deletes and recreates the index).
+```
+# delete index and recreate it
 bin/console generic-data-index:update:index -r
 ```
+Without the `-r` option, the index mapping is just updated and all items are added into the queue
+for a reindex from the Pimcore database. 
 
-Without `-r`, the command only updates the index mapping and queues all items for reindex.
 
-### Updating Index Structure
+### Updating Index Structure for Data Indices
 
-Index mappings update automatically when system languages or class definition fields change.
-For manual updates, run the reindex command. This performs a native OpenSearch/Elasticsearch
-reindex within the search indices (no database read):
+Index mapping is updated automatically e.g. when adding system languages or new fields to the class definition. 
+Sometimes it might be necessary to update the index structure manually.
 
-```bash
+Do this with the reindex command. This command does native opensearch/elasticsearch re-indexing. So it does not 
+index data from the database but reindexes data within the search indices.
+
+```
+# updates index mapping with native reindexing
 bin/console generic-data-index:reindex
 ```
 
 ### Handling Failed Messages
 
-The messenger retries failed messages 3 times, then routes them to the
-`pimcore_generic_data_index_failed` transport. Retry failed messages with:
+By default, the messenger will retry failed messages 3 times and then send them into the failed queue `pimcore_generic_data_index_failed`.
+If you want to retry failed messages, you can use the following command:
 
-```bash
-bin/console messenger:failed:retry -vv
+```
+php bin/console messenger:failed:retry -vv
 ```
 
-See the [Symfony Messenger documentation](https://symfony.com/doc/current/messenger.html#saving-retrying-failed-messages)
-for additional commands.
+For the further commands please refer to the [Symfony Messenger documentation](https://symfony.com/doc/current/messenger.html#saving-retrying-failed-messages).
 
-## Index Options
+## Configuring index options (Maximum Item Limit, ...)
 
-Configure search engine-specific index options. Check your engine's documentation
-for available settings.
+You can configure different options to use with your indices. The available options can differ depending on which 
+engine you are using. Make sure to check the corresponding documentation, before using any options.
 
-### Maximum Result Window
+See the `Maxium Item Limit` and `Total fields limit` section for examples.
 
-The default limit of retrievable items is 10,000. Increase it with:
+### Maximum Item limit
+
+A maximum of 10000 items can be retrieved and viewed, because of the maximum default item limit.
+To increase this limit, configuration can be adjusted as follows:
 
 ```yaml
 pimcore_generic_data_index:
@@ -157,9 +147,10 @@ pimcore_generic_data_index:
             max_result_window: 20000
 ```
 
-### Total Fields Limit
+### Total fields limit
 
-The default field limit per index is 1,000. Increase it with:
+A maximum of 1000 fields can be used with your indces. 
+To increase this limit, configuration can be adjusted as follows:
 
 ```yaml
 pimcore_generic_data_index:
@@ -170,23 +161,36 @@ pimcore_generic_data_index:
 
 :::info
 
-If the index already exists, recreate it after changing this setting:
+If an index was already created before setting this parameter, the index needs to be recreated.
 
-```bash
+```
 bin/console generic-data-index:update:index -r
 ```
 
 :::
 
 ## Deployment and Index Management
 
-### Class Definition Changes
+### Pimcore Class Definitions
+
+After every class definition update you should run the following command to update the index structure:
+
+```
+php bin/console generic-data-index:deployment:reindex
+```
+
+This command will update the index structure for all data object classes which were created/updated since the last deployment and reindex all data objects for relevant classes.
+
+### Cleaning Up Unused Indices
 
-After updating class definitions during deployment, run:
+To clean up old indices that are not referenced by any alias, use the following command:
 
-```bash
-bin/console generic-data-index:deployment:reindex
+```
+php bin/console generic-data-index:cleanup:unused-indices
 ```
 
-This updates the index structure for all class definitions modified since the last
-deployment and reindexes data objects for affected classes.
+To preview what would be deleted without performing any changes, run:
+
+```
+php bin/console generic-data-index:cleanup:unused-indices --dry-run
+```

src/Command/CleanupUnusedIndicesCommand.php

@@ -0,0 +1,103 @@
+<?php
+declare(strict_types=1);
+
+/**
+ * This source file is available under the terms of the
+ * Pimcore Open Core License (POCL)
+ * Full copyright and license information is available in
+ * LICENSE.md which is distributed with this source code.
+ *
+ *  @copyright  Copyright (c) Pimcore GmbH (https://www.pimcore.com)
+ *  @license    Pimcore Open Core License (POCL)
+ */
+
+namespace Pimcore\Bundle\GenericDataIndexBundle\Command;
+
+use Exception;
+use Pimcore\Bundle\GenericDataIndexBundle\Exception\CommandAlreadyRunningException;
+use Pimcore\Bundle\GenericDataIndexBundle\Service\SearchIndex\UnusedIndexCleanupService;
+use Pimcore\Console\AbstractCommand;
+use Symfony\Component\Console\Command\LockableTrait;
+use Symfony\Component\Console\Input\InputInterface;
+use Symfony\Component\Console\Input\InputOption;
+use Symfony\Component\Console\Output\OutputInterface;
+
+/**
+ * @internal
+ */
+final class CleanupUnusedIndicesCommand extends AbstractCommand
+{
+    use LockableTrait;
+
+    private const OPTION_DRY_RUN = 'dry-run';
+
+    public function __construct(
+        private readonly UnusedIndexCleanupService $unusedIndexCleanupService,
+        ?string $name = null
+    ) {
+        parent::__construct($name);
+    }
+
+    protected function configure(): void
+    {
+        $this
+            ->setName('generic-data-index:cleanup:unused-indices')
+            ->addOption(
+                self::OPTION_DRY_RUN,
+                null,
+                InputOption::VALUE_NONE,
+                'List unused indices without deleting them.'
+            )
+            ->setDescription(
+                'Deletes managed Generic Data Index indices with the configured index prefix and a -odd/-even suffix that are not referenced by any alias.'
+            )
+            ->setHelp(
+                'This command only targets managed Generic Data Index indices that use the configured index prefix and end with -odd or -even. It does not consider other indices.'
+            );
+    }
+
+    /**
+     * @throws CommandAlreadyRunningException
+     */
+    protected function execute(InputInterface $input, OutputInterface $output): int
+    {
+        if (!$this->lock()) {
+            throw new CommandAlreadyRunningException(
+                'The command is already running in another process.'
+            );
+        }
+
+        try {
+            $dryRun = (bool) $input->getOption(self::OPTION_DRY_RUN);
+            $unusedIndices = $this->unusedIndexCleanupService->cleanupUnusedIndices($dryRun);
+
+            if (empty($unusedIndices)) {
+                $output->writeln('<info>No unused indices found.</info>');
+
+                return self::SUCCESS;
+            }
+
+            $output->writeln('<info>Unused indices:</info>');
+            foreach ($unusedIndices as $indexName) {
+                $output->writeln(sprintf(' - %s', $indexName));
+            }
+
+            if ($dryRun) {
+                $output->writeln(
+                    sprintf('<comment>Dry run: %d indices would be deleted.</comment>', count($unusedIndices))
+                );
+            } else {
+                $output->writeln(
+                    sprintf('<info>Deleted %d unused indices.</info>', count($unusedIndices))
+                );
+            }
+        } catch (Exception $e) {
+            $output->writeln('<error>' . $e->getMessage() . '</error>');
+            return self::FAILURE;
+        } finally {
+            $this->release();
+        }
+
+        return self::SUCCESS;
+    }
+}