Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions .translation-cache/References.md.json

Large diffs are not rendered by default.

18 changes: 15 additions & 3 deletions .translation-cache/Server_settings/Common.md.json

Large diffs are not rendered by default.

150 changes: 145 additions & 5 deletions .translation-cache/Server_settings/Searchd.md.json

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions manual/chinese/References.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,6 +466,7 @@ index_converter {--config /path/to/config|--path}
* [log](Server_settings/Searchd.md#log) - Manticore 服务器日志文件路径
* [max_batch_queries](Server_settings/Searchd.md#max_batch_queries) - 限制每批查询的数量
* [max_connections](Server_settings/Searchd.md#max_connections) - 最大活动连接数
* [merge_chunks_per_job](Server_settings/Searchd.md#merge_chunks_per_job) - 每个 OPTIMIZE 任务合并的 RT 磁盘块数量
* [max_filters](Server_settings/Searchd.md#max_filters) - 每个查询允许的最大过滤器数量
* [max_filter_values](Server_settings/Searchd.md#max_filter_values) - 每个过滤器允许的最大值数量
* [max_open_files](Server_settings/Searchd.md#max_open_files) - 服务器允许打开的最大文件数
Expand All @@ -478,6 +479,7 @@ index_converter {--config /path/to/config|--path}
* [network_timeout](Server_settings/Searchd.md#network_timeout) - 客户端请求的网络超时
* [node_address](Server_settings/Searchd.md#node_address) - 指定节点的网络地址
* [persistent_connections_limit](Creating_a_table/Creating_a_distributed_table/Remote_tables.md#agent) - 到远程持久代理的持久连接最大数量
* [parallel_chunk_merges](Server_settings/Searchd.md#parallel_chunk_merges) - 在 OPTIMIZE 期间可以并行运行的 RT 磁盘块合并数量
* [pid_file](Server_settings/Searchd.md#pid_file) - Manticore 服务器 pid 文件路径
* [preopen_tables](Server_settings/Searchd.md#preopen_tables) - 确定是否在启动时强制预打开所有表
* [pseudo_sharding](Server_settings/Searchd.md#pseudo_sharding) - 为对普通表和实时表的搜索查询启用伪分片
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,4 +111,3 @@ ALTER CLUSTER mycluster ADD myindex;
在优化过程中,可以在任何节点上正常使用搜索操作。

<!-- proofread -->

2 changes: 1 addition & 1 deletion manual/chinese/Server_settings/Common.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ lemmatizer_base = /usr/share/manticore/
progressive_merge
------------------

progressive_merge 是一个配置指令,启用后,会将实时表的磁盘块从小到大合并。这种方式加快了合并过程,减少了读写放大。默认情况下,此设置是启用的。如果禁用,块将按创建顺序合并
progressive_merge 是一个配置指令,启用后会将实时表的磁盘块从较小的合并到较大的块中。这种方法加快了合并过程并减少了读写放大。此设置始终启用,仅保留配置中以保持向后兼容性

json_autoconv_keynames
------------------------
Expand Down
50 changes: 49 additions & 1 deletion manual/chinese/Server_settings/Searchd.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,55 @@ auto_optimize = 2 # OPTIMIZE starts at 16 chunks (on 4 cpu cores server)

<!-- end -->

### parallel_chunk_merges

<!-- example conf parallel_chunk_merges -->
此设置控制服务器在 [OPTIMIZE](../Securing_and_compacting_a_table/Compacting_a_table.md#OPTIMIZE-TABLE) 过程中允许并行运行的磁盘块合并任务数量,适用于实时表。

这仅影响磁盘块合并(压缩),不影响查询并行性。

将其设置为 `1` 以禁用并行块合并(合并任务将逐个运行)。较高的值可能会在具有快速存储的系统上加快压缩速度,但会增加并发磁盘 I/O。

默认值为 `max(1, min(2, threads/2))`。

此值可以通过 `SET GLOBAL parallel_chunk_merges = N` 在运行时更改,并通过 `SHOW VARIABLES` 查看。

<!-- intro -->
##### 示例:

<!-- request Disable -->
```ini
parallel_chunk_merges = 1
```

<!-- request Increase -->
```ini
parallel_chunk_merges = 4
```

<!-- end -->

### merge_chunks_per_job

<!-- example conf merge_chunks_per_job -->
此设置控制在单个 OPTIMIZE 任务中合并多少个 RT 磁盘块(N 路合并)。如果可用块少于此数量,任务将合并它能合并的块(最少 2 个)。

较低的值允许更多任务并行调度;较高的值会减少任务数量但增加每次合并的规模。

默认值为 `2`。

此值可以通过 `SET GLOBAL merge_chunks_per_job = N` 在运行时更改,并通过 `SHOW VARIABLES` 查看。

<!-- intro -->
##### 示例:

<!-- request Increase -->
```ini
merge_chunks_per_job = 4
```

<!-- end -->

### auto_schema

<!-- example conf auto_schema -->
Expand Down Expand Up @@ -1730,4 +1779,3 @@ watchdog = 0 # disable watchdog
```
<!-- end -->
<!-- proofread -->

2 changes: 2 additions & 0 deletions manual/english/References.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,6 +466,7 @@ To be put in the `searchd {}` section of the configuration file:
* [log](Server_settings/Searchd.md#log) - Path to Manticore server log file
* [max_batch_queries](Server_settings/Searchd.md#max_batch_queries) - Limits the number of queries per batch
* [max_connections](Server_settings/Searchd.md#max_connections) - Maximum number of active connections
* [merge_chunks_per_job](Server_settings/Searchd.md#merge_chunks_per_job) - How many RT disk chunks are merged per OPTIMIZE job
* [max_filters](Server_settings/Searchd.md#max_filters) - Maximum allowed per-query filter count
* [max_filter_values](Server_settings/Searchd.md#max_filter_values) - Maximum allowed per-filter values count
* [max_open_files](Server_settings/Searchd.md#max_open_files) - Maximum number of files allowed to be opened by server
Expand All @@ -478,6 +479,7 @@ To be put in the `searchd {}` section of the configuration file:
* [network_timeout](Server_settings/Searchd.md#network_timeout) - Network timeout for client requests
* [node_address](Server_settings/Searchd.md#node_address) - Specifies network address of the node
* [persistent_connections_limit](Creating_a_table/Creating_a_distributed_table/Remote_tables.md#agent) - Maximum number of simultaneous persistent connections to remote persistent agents
* [parallel_chunk_merges](Server_settings/Searchd.md#parallel_chunk_merges) - How many RT disk chunk merges can run in parallel during OPTIMIZE
* [pid_file](Server_settings/Searchd.md#pid_file) - Path to Manticore server pid file
* [preopen_tables](Server_settings/Searchd.md#preopen_tables) - Determines whether to forcibly preopen all tables on startup
* [pseudo_sharding](Server_settings/Searchd.md#pseudo_sharding) - Enables pseudo-sharding for search queries to plain and real-time tables
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Compacting a Table

Over time, RT tables may become fragmented into numerous disk chunks and/or contaminated with deleted, yet unpurged data, affecting search performance. In these cases, optimization is necessary. Essentially, the optimization process combines pairs of disk chunks, removing documents that were previously deleted using DELETE statements.
Over time, RT tables may become fragmented into numerous disk chunks and/or contaminated with deleted, yet unpurged data, affecting search performance. In these cases, optimization is necessary. Essentially, the optimization process combines disk chunks (N-way merge), removing documents that were previously deleted using DELETE statements.

Beginning with Manticore 4, this process occurs [automatically by default](../Server_settings/Searchd.md#auto_optimize). However, you can also use the following commands to manually initiate table compaction.

Expand Down Expand Up @@ -65,7 +65,7 @@ OPTIMIZE TABLE rt OPTION sync=1;

### Throttling the IO impact

Optimization can be a lengthy and I/O-intensive process. To minimize the impact, all actual merge work is executed serially in a special background thread, and the `OPTIMIZE` statement simply adds a job to its queue. The optimization thread can be I/O-throttled, and you can control the maximum number of I/Os per second and the maximum I/O size with the [rt_merge_iops](../Server_settings/Searchd.md#rt_merge_iops) and [rt_merge_maxiosize](../Server_settings/Searchd.md#rt_merge_maxiosize) directives, respectively.
Optimization can be a lengthy and I/O-intensive process. The `OPTIMIZE` statement adds a job to a background worker pool. You can control how many jobs run in parallel with [parallel_chunk_merges](../Server_settings/Searchd.md#parallel_chunk_merges) and how many chunks each job merges with [merge_chunks_per_job](../Server_settings/Searchd.md#merge_chunks_per_job). The optimization workers can be I/O-throttled, and you can control the maximum number of I/Os per second and the maximum I/O size with the [rt_merge_iops](../Server_settings/Searchd.md#rt_merge_iops) and [rt_merge_maxiosize](../Server_settings/Searchd.md#rt_merge_maxiosize) directives, respectively.

During optimization, the RT table being optimized remains online and available for both searching and updates nearly all the time. It is locked for a very brief period when a pair of disk chunks is successfully merged, allowing for the renaming of old and new files and updating the table header.

Expand Down Expand Up @@ -111,4 +111,3 @@ Once the table is added back to the cluster, you must resume write operations on
Search operations are available as usual during the process on any of the nodes.

<!-- proofread -->

2 changes: 1 addition & 1 deletion manual/english/Server_settings/Common.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ lemmatizer_base = /usr/share/manticore/
progressive_merge
------------------

The progressive_merge is a configuration directive that, when enabled, merges real-time table disk chunks from smaller to larger ones. This approach speeds up the merging process and reduces read/write amplification. By default, this setting is enabled. If disabled, the chunks are merged in the order they were created.
The progressive_merge is a configuration directive that, when enabled, merges real-time table disk chunks from smaller to larger ones. This approach speeds up the merging process and reduces read/write amplification. This setting is always enabled, and left in configuration only for backward compatibility.

json_autoconv_keynames
------------------------
Expand Down
50 changes: 49 additions & 1 deletion manual/english/Server_settings/Searchd.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,55 @@ auto_optimize = 2 # OPTIMIZE starts at 16 chunks (on 4 cpu cores server)

<!-- end -->

### parallel_chunk_merges

<!-- example conf parallel_chunk_merges -->
This setting controls how many disk chunk merge jobs the server is allowed to run in parallel during [OPTIMIZE](../Securing_and_compacting_a_table/Compacting_a_table.md#OPTIMIZE-TABLE) for real-time tables.

This affects only disk chunk merging (compaction), not query parallelism.

Set it to `1` to disable parallel chunk merging (merge jobs will run one-by-one). Higher values may speed up compaction on systems with fast storage, but will increase concurrent disk I/O.

Default is `max(1, min(2, threads/2))`.

This value can be changed at runtime using `SET GLOBAL parallel_chunk_merges = N` and inspected via `SHOW VARIABLES`.

<!-- intro -->
##### Example:

<!-- request Disable -->
```ini
parallel_chunk_merges = 1
```

<!-- request Increase -->
```ini
parallel_chunk_merges = 4
```

<!-- end -->

### merge_chunks_per_job

<!-- example conf merge_chunks_per_job -->
This setting controls how many RT disk chunks are merged in a single OPTIMIZE job (N-way merge). If fewer than this number are available, the job will merge what it can (minimum 2).

Lower values allow more jobs to be scheduled in parallel; higher values reduce the number of jobs but increase the size of each merge.

Default is `2`.

This value can be changed at runtime using `SET GLOBAL merge_chunks_per_job = N` and inspected via `SHOW VARIABLES`.

<!-- intro -->
##### Example:

<!-- request Increase -->
```ini
merge_chunks_per_job = 4
```

<!-- end -->

### auto_schema

<!-- example conf auto_schema -->
Expand Down Expand Up @@ -1730,4 +1779,3 @@ watchdog = 0 # disable watchdog
```
<!-- end -->
<!-- proofread -->

2 changes: 2 additions & 0 deletions manual/russian/References.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,6 +466,7 @@ index_converter {--config /path/to/config|--path}
* [log](Server_settings/Searchd.md#log) - Путь к файлу журнала сервера Manticore
* [max_batch_queries](Server_settings/Searchd.md#max_batch_queries) - Ограничивает количество запросов в пакете
* [max_connections](Server_settings/Searchd.md#max_connections) - Максимальное количество активных соединений
* [merge_chunks_per_job](Server_settings/Searchd.md#merge_chunks_per_job) - Сколько RT-дисковых чанков объединяется за одну задачу OPTIMIZE
* [max_filters](Server_settings/Searchd.md#max_filters) - Максимально допустимое количество фильтров на запрос
* [max_filter_values](Server_settings/Searchd.md#max_filter_values) - Максимально допустимое количество значений на фильтр
* [max_open_files](Server_settings/Searchd.md#max_open_files) - Максимальное количество файлов, разрешенных для открытия сервером
Expand All @@ -478,6 +479,7 @@ index_converter {--config /path/to/config|--path}
* [network_timeout](Server_settings/Searchd.md#network_timeout) - Сетевой таймаут для запросов клиентов
* [node_address](Server_settings/Searchd.md#node_address) - Определяет сетевой адрес узла
* [persistent_connections_limit](Creating_a_table/Creating_a_distributed_table/Remote_tables.md#agent) - Максимальное количество одновременных постоянных соединений с удаленными постоянными агентами
* [parallel_chunk_merges](Server_settings/Searchd.md#parallel_chunk_merges) - Сколько слияний RT-дисковых чанков может выполняться параллельно во время OPTIMIZE
* [pid_file](Server_settings/Searchd.md#pid_file) - Путь к pid-файлу сервера Manticore
* [preopen_tables](Server_settings/Searchd.md#preopen_tables) - Определяет, следует ли принудительно предварительно открывать все таблицы при запуске
* [pseudo_sharding](Server_settings/Searchd.md#pseudo_sharding) - Включает псевдошардинг для поисковых запросов к обычным и реального времени таблицам
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,4 +111,3 @@ ALTER CLUSTER mycluster ADD myindex;
Операции поиска доступны как обычно в процессе на любом из узлов.

<!-- proofread -->

2 changes: 1 addition & 1 deletion manual/russian/Server_settings/Common.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ lemmatizer_base = /usr/share/manticore/
progressive_merge
------------------

`progressive_merge` — это директива конфигурации, которая при включении объединяет дисковые чанки реального времени таблицы от меньших к большим. Такой подход ускоряет процесс слияния и уменьшает усиление при чтении/записи. По умолчанию эта настройка включена. Если выключена, чанки объединяются в порядке их создания.
`progressive_merge` — это директива конфигурации, которая при включении объединяет дисковые чанки таблиц реального времени от меньших к большим. Этот подход ускоряет процесс объединения и снижает усиление чтения/записи. Эта настройка всегда включена и оставлена в конфигурации только для обратной совместимости.

json_autoconv_keynames
------------------------
Expand Down
50 changes: 49 additions & 1 deletion manual/russian/Server_settings/Searchd.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,55 @@ auto_optimize = 2 # OPTIMIZE starts at 16 chunks (on 4 cpu cores server)

<!-- end -->

### parallel_chunk_merges

<!-- example conf parallel_chunk_merges -->
Эта настройка управляет тем, сколько заданий слияния дисковых чанков серверу разрешено выполнять параллельно во время [OPTIMIZE](../Securing_and_compacting_a_table/Compacting_a_table.md#OPTIMIZE-TABLE) для таблиц реального времени.

Это влияет только на слияние дисковых чанков (уплотнение), а не на параллелизм запросов.

Установите значение `1`, чтобы отключить параллельное слияние чанков (задания слияния будут выполняться одно за другим). Более высокие значения могут ускорить уплотнение на системах с быстрым хранилищем, но увеличат одновременный ввод-вывод на диск.

По умолчанию `max(1, min(2, threads/2))`.

Это значение можно изменить во время выполнения с помощью `SET GLOBAL parallel_chunk_merges = N` и проверить через `SHOW VARIABLES`.

<!-- intro -->
##### Пример:

<!-- request Disable -->
```ini
parallel_chunk_merges = 1
```

<!-- request Increase -->
```ini
parallel_chunk_merges = 4
```

<!-- end -->

### merge_chunks_per_job

<!-- example conf merge_chunks_per_job -->
Эта настройка управляет тем, сколько RT дисковых чанков объединяется в одном задании OPTIMIZE (N-стороннее слияние). Если доступно меньше этого числа, задание объединит то, что может (минимум 2).

Более низкие значения позволяют планировать больше заданий параллельно; более высокие значения уменьшают количество заданий, но увеличивают размер каждого слияния.

По умолчанию `2`.

Это значение можно изменить во время выполнения с помощью `SET GLOBAL merge_chunks_per_job = N` и проверить через `SHOW VARIABLES`.

<!-- intro -->
##### Пример:

<!-- request Increase -->
```ini
merge_chunks_per_job = 4
```

<!-- end -->

### auto_schema

<!-- example conf auto_schema -->
Expand Down Expand Up @@ -1730,4 +1779,3 @@ watchdog = 0 # disable watchdog
```
<!-- end -->
<!-- proofread -->

Loading