You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: hadoop-hdds/docs/content/feature/ErasureCoding.md
-35Lines changed: 0 additions & 35 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -228,41 +228,6 @@ When using ofs/o3fs, we can pass the EC Replication Config by setting the config
228
228
229
229
In the case bucket already has default EC Replication Config, there is no need of passing EC Replication Config while creating key.
230
230
231
-
#### Calculating EC Pipeline Limits
232
-
233
-
The target number of open EC pipelines SCM aims to maintain is calculated dynamically for each EC replication configuration (e.g., RS-6-3, RS-3-2). The calculation is based on the following two properties, with the final target being the greater of the two resulting values.
234
-
235
-
*`ozone.scm.ec.pipeline.minimum`
236
-
***Description**: The guaranteed minimum number of open pipelines to maintain for each EC configuration, regardless of other factors.
237
-
***Default Value**: `5`
238
-
239
-
*`ozone.scm.ec.pipeline.per.volume.factor`
240
-
***Description**: A factor used to calculate a target number of pipelines based on the total number of healthy volumes across all datanodes in the cluster.
241
-
***Default Value**: `1.0`
242
-
243
-
**Calculation Logic:**
244
-
245
-
SCM first calculates a volume-based target using the formula:
1. The volume-based target is: `(1.0 * 200) / 9 = 22`
258
-
2. The final target is: `max(22, 5) = 22`
259
-
260
-
SCM will attempt to create and maintain approximately **22** open, RS-6-3 EC pipelines.
261
-
262
-
**Production Recommendation:**
263
-
264
-
The default values are a good starting point for most clusters. If you have a very high number of volumes and a write-heavy EC workload, you might consider slightly increasing the `pipeline.per.volume.factor`. Conversely, for read-heavy workloads, the default minimum of 5 pipelines is often sufficient.
265
-
266
231
### Enable Intel ISA-L
267
232
268
233
Intel Intelligent Storage Acceleration Library (ISA-L) is an open-source collection of optimized low-level functions used for
0 commit comments