Skip to content

Commit 2836ec0

Browse files
committed
Batch 6: 21 data manipulation filter docs reviewed
Tier 2 (10 filters): CreateAttributeMatrix, CreateDataGroup, DeleteData, RenameDataObject, MoveData, CopyFeatureArrayToElementArray, CreateFeatureArrayFromElementArray, ConditionalSetValue, ConvertColorToGrayScale, ReshapeDataArray. Tier 3 (11 filters): CreateDataArray, CreateDataArrayAdvanced, CopyDataObject, CombineAttributeArrays, ConcatenateDataArrays, SplitDataArrayByComponent, SplitDataArrayByTuple, ExtractComponentAsArray, ConvertData, ArrayCalculator, InitializeData. Notable structural changes: - CreateAttributeMatrix and CreateDataGroup: added 'What is X?' sections with explicit guidance on when to choose one over the other. - DeleteData: documented cascade-delete behavior across container types. - RenameDataObject: warning about downstream parameter references. - MoveData: explained tuple-count vs tuple-shape compatibility. - CreateFeatureArrayFromElementArray: promoted 'last element copied' footgun to a Warning section; cross-linked ComputeArrayStatistics. - ConditionalSetValue: split the two modes (mask vs value-match) into their own subsections. - ReshapeDataArray: critical 'does not move data in memory' warning promoted to top-level with safe-vs-dangerous examples. - InitializeData: restructured from rambly bullets into clean per-mode subsections. Build verified clean (no new Sphinx warnings).
1 parent 8ce7c33 commit 2836ec0

22 files changed

Lines changed: 368 additions & 138 deletions

docs/documentation_review_tracker.md

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
| 3 | Segmentation / Feature Identification | 10 | 10 | 10 | Complete |
1616
| 4 | Neighbor / Kernel Operations | 11 | 11 | 11 | Complete |
1717
| 5 | Geometry Creation / Manipulation | 15 | 15 | 15 | Complete |
18-
| 6 | Data Manipulation (Copy, Create, Delete, Rename) | 21 | 21 | 0 | Triaged |
18+
| 6 | Data Manipulation (Copy, Create, Delete, Rename) | 21 | 21 | 21 | Complete |
1919
| 7 | I/O Filters (Read/Write) | ~15 | 0 | 0 | Not Started |
2020
| 8 | Image Processing (ITK wrappers) | ~88 | 0 | 0 | Not Started |
2121
| 9 | Remaining SimplnxCore filters | ~varies | 0 | 0 | Not Started |
@@ -451,16 +451,16 @@ None. No filter in this batch is incomprehensible to a non-expert.
451451

452452
| Filter | Plugin | Status |
453453
|--------|--------|--------|
454-
| CreateAttributeMatrix | SimplnxCore | Pending |
455-
| CreateDataGroup | SimplnxCore | Pending |
456-
| DeleteData | SimplnxCore | Pending |
457-
| RenameDataObject | SimplnxCore | Pending |
458-
| MoveData | SimplnxCore | Pending |
459-
| CopyFeatureArrayToElementArray | SimplnxCore | Pending |
460-
| CreateFeatureArrayFromElementArray | SimplnxCore | Pending |
461-
| ConditionalSetValue | SimplnxCore | Pending |
462-
| ConvertColorToGrayScale | SimplnxCore | Pending |
463-
| ReshapeDataArray | SimplnxCore | Pending |
454+
| CreateAttributeMatrix | SimplnxCore | Done |
455+
| CreateDataGroup | SimplnxCore | Done |
456+
| DeleteData | SimplnxCore | Done |
457+
| RenameDataObject | SimplnxCore | Done |
458+
| MoveData | SimplnxCore | Done |
459+
| CopyFeatureArrayToElementArray | SimplnxCore | Done |
460+
| CreateFeatureArrayFromElementArray | SimplnxCore | Done |
461+
| ConditionalSetValue | SimplnxCore | Done |
462+
| ConvertColorToGrayScale | SimplnxCore | Done |
463+
| ReshapeDataArray | SimplnxCore | Done |
464464

465465
- [ ] **CreateAttributeMatrixFilter** (SimplnxCore)
466466
- **Clarity:** Extremely brief (only ~20 lines). Lone example does not explain what an Attribute Matrix conceptually represents.
@@ -523,17 +523,17 @@ None. No filter in this batch is incomprehensible to a non-expert.
523523

524524
| Filter | Plugin | Status |
525525
|--------|--------|--------|
526-
| CreateDataArray | SimplnxCore | Pending |
527-
| CreateDataArrayAdvanced | SimplnxCore | Pending |
528-
| CopyDataObject | SimplnxCore | Pending |
529-
| CombineAttributeArrays | SimplnxCore | Pending |
530-
| ConcatenateDataArrays | SimplnxCore | Pending |
531-
| SplitDataArrayByComponent | SimplnxCore | Pending |
532-
| SplitDataArrayByTuple | SimplnxCore | Pending |
533-
| ExtractComponentAsArray | SimplnxCore | Pending |
534-
| ConvertData | SimplnxCore | Pending |
535-
| ArrayCalculator | SimplnxCore | Pending |
536-
| InitializeData | SimplnxCore | Pending |
526+
| CreateDataArray | SimplnxCore | Done |
527+
| CreateDataArrayAdvanced | SimplnxCore | Done |
528+
| CopyDataObject | SimplnxCore | Done |
529+
| CombineAttributeArrays | SimplnxCore | Done |
530+
| ConcatenateDataArrays | SimplnxCore | Done |
531+
| SplitDataArrayByComponent | SimplnxCore | Done |
532+
| SplitDataArrayByTuple | SimplnxCore | Done |
533+
| ExtractComponentAsArray | SimplnxCore | Done |
534+
| ConvertData | SimplnxCore | Done |
535+
| ArrayCalculator | SimplnxCore | Done |
536+
| InitializeData | SimplnxCore | Done |
537537

538538
- [ ] **CreateDataArrayFilter** (SimplnxCore)
539539
- **Notes:** Already has good data-type ranges table and semicolon-notation example. Minor polish: state that Number of Components must be ≥ 1; explicitly note that this filter creates a *single component dimension* (use CreateDataArrayAdvanced for multi-dimension components). Add Required Input Sources (none).

src/Plugins/SimplnxCore/docs/ArrayCalculatorFilter.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,11 @@ This equation takes the cube root of *8* and stores the result in an array with
135135

136136
This equation raises *3* to the power of the value stored in every component of every tuple in *Array1*. The resulting array has the same tuple and component size of *Array1*.
137137

138+
### Required Input Sources
139+
140+
- **Source Attribute Matrix** -- the Attribute Matrix that contains the input arrays referenced in the *Infix Expression*.
141+
- **Output Attribute Matrix** -- an existing Attribute Matrix with the matching tuple count (or with exactly one tuple, when the expression contains no arrays).
142+
138143
% Auto generated parameter table will be inserted here
139144

140145
## Example Pipelines

src/Plugins/SimplnxCore/docs/CombineAttributeArraysFilter.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ The user may also select to normalize the resulting combined array. The normaliz
3232

3333
**NOTE:** If you are wanting to instead concatenate/append data arrays together into a longer array, please see the [Concatenate Data Arrays](ConcatenateDataArraysFilter.md) filter.
3434

35+
A typical use case is building a multi-component coordinate or quaternion array from three or four separate single-component scalar arrays (e.g., combining `X_Coord`, `Y_Coord`, `Z_Coord` into a single 3-component `Vertices` array before passing to [Create Geometry](CreateGeometryFilter.md)).
36+
37+
### Required Input Sources
38+
39+
- **Input Attribute Arrays** -- two or more arrays of the same primitive type and tuple count, in the order you want their components to appear in the combined array.
40+
3541
% Auto generated parameter table will be inserted here
3642

3743
## Example Pipelines

src/Plugins/SimplnxCore/docs/ConcatenateDataArraysFilter.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,12 @@ This filter is designed to handle arrays of matching array types and component d
1414

1515
**NOTE:** If you are wanting to instead combine data arrays into a multi-component array, please see the [Combine Attribute Arrays](CombineAttributeArraysFilter.md) filter.
1616

17+
For example, given two 3-component float arrays of 5 and 7 tuples respectively, the output is a 3-component float array of 12 tuples whose first 5 tuples are the first input and last 7 tuples are the second input.
18+
19+
### Required Input Sources
20+
21+
- **Input Data Arrays** -- two or more arrays sharing the same primitive type and the same component dimensions.
22+
1723
% Auto generated parameter table will be inserted here
1824

1925
## Example Pipelines

src/Plugins/SimplnxCore/docs/ConditionalSetValueFilter.md

Lines changed: 36 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -6,23 +6,42 @@ Core (Misc)
66

77
## Description
88

9-
This **Filter** replaces values in a user specified **Attribute Array** with a user specified value a second boolean **Attribute Array** specifies, but only when **Use Conditional Mask** is *true*. For example, if the user entered a *Replace Value* of *5.5*, then for every occurence of *true* in the conditional boolean array, the selected **Attribute Array** would be changed to 5.5. If **Use Conditional Mask** is *false*, then **Value to Replace** will be searched for in the provided **Attribute Array** and all instances will be replaced. Below are the ranges for the values that can be entered for the different primitive types of arrays (for user reference). The selected **Attribute Array** must be a scalar array.
10-
11-
### Primitive Data Types
12-
13-
| Type | Size | Range |
14-
|------------------|------|--------------------|
15-
| Signed Integer | 8 bit |-128 to 127|
16-
| Unsigned Integer | 8 bit |0 to 255|
17-
| Signed Integer | 16 bit |-32,768 to 32,767|
18-
| Unsigned Integer | 16 bit |0 to 65,535|
19-
| Signed Integer | 32 bit |-2,147,483,648 to 2,147,483,647|
20-
| Unsigned Integer | 32 bit |0 to 4,294,967,295|
21-
| Signed Integer | 64 bit | 9,223,372,036,854,775,808 to 9,223,372,036,854,775,807|
22-
| Unsigned Integer | 64 bit |0 to 18,446,744,073,709,551,615|
23-
| Float | 32 bit | -3.4e+38 to -1.1e-38, 0.0, 1.1e-38 to 3.4e+38 (7 digits)|
24-
| Double | 64 bit | -1.7e+308 to -2.2e-308, 0.0, 2.2e-308 to 1.7e+308 (15 digits)|
25-
| Boolean | 8 bit |0 = false and any other value will be forced to 1 = true|
9+
This **Filter** replaces selected values in a scalar **Attribute Array** with a user-specified value. The filter operates in one of two modes selected by the *Use Conditional Mask* parameter.
10+
11+
### Mode 1 -- Conditional Mask (Use Conditional Mask = ON)
12+
13+
A second boolean array (the *Conditional Mask*) of the same length picks which tuples to overwrite. Every tuple where the mask is *true* has its scalar value set to *Replace Value*; tuples where the mask is *false* are unchanged.
14+
15+
Typical use case: replace cell values flagged by an upstream threshold (e.g., set all "bad" cells to 0).
16+
17+
### Mode 2 -- Value Match (Use Conditional Mask = OFF)
18+
19+
Every occurrence of *Value to Replace* in the target array is replaced with *Replace Value*. The Conditional Mask parameter is ignored.
20+
21+
Typical use case: remap a specific sentinel value (e.g., turn every -1 into 0).
22+
23+
### Numeric Type Compatibility
24+
25+
The target array must be a **scalar** (single-component) array. *Replace Value* (and *Value to Replace* in Mode 2) are reinterpreted as the array's data type. The valid range for each primitive type:
26+
27+
| Type | Size | Range |
28+
|------|------|-------|
29+
| Signed Integer | 8 bit | -128 to 127 |
30+
| Unsigned Integer | 8 bit | 0 to 255 |
31+
| Signed Integer | 16 bit | -32,768 to 32,767 |
32+
| Unsigned Integer | 16 bit | 0 to 65,535 |
33+
| Signed Integer | 32 bit | -2,147,483,648 to 2,147,483,647 |
34+
| Unsigned Integer | 32 bit | 0 to 4,294,967,295 |
35+
| Signed Integer | 64 bit | -9.2e18 to 9.2e18 |
36+
| Unsigned Integer | 64 bit | 0 to 1.8e19 |
37+
| Float | 32 bit | ±1.1e-38 to ±3.4e+38 (7 digits) |
38+
| Double | 64 bit | ±2.2e-308 to ±1.7e+308 (15 digits) |
39+
| Boolean | 8 bit | 0 = false, non-zero = true |
40+
41+
### Required Input Sources
42+
43+
- **Target Array** -- the scalar cell-level (or other tuple-level) array whose values will be overwritten.
44+
- **Conditional Mask** (only when *Use Conditional Mask* is enabled) -- a boolean array with the same number of tuples; typically produced by [Multi-Threshold Objects](MultiThresholdObjectsFilter.md).
2645

2746
% Auto generated parameter table will be inserted here
2847

src/Plugins/SimplnxCore/docs/ConvertColorToGrayScaleFilter.md

Lines changed: 20 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -6,25 +6,34 @@ Processing (Image)
66

77
## Description
88

9-
This **Filter** allows the user to select a *flattening* method for turning an array of RGB or RGBa values into grayscale values.
9+
This **Filter** converts one or more color image arrays (3-component RGB or 4-component RGBA, uint8) into single-component grayscale image arrays using one of four standard flattening algorithms. Use this filter as a preprocessing step before any image-processing filter that expects single-channel intensity data.
1010

1111
### Conversion Algorithm
1212

13-
The *Conversion Algorithm* parameter provides the following choices:
13+
The *Conversion Algorithm* parameter selects how the three (or four) color channels are combined into a single value:
1414

15-
- **Luminosity [0]**: A weighted average of RGB channels that accounts for human perception (more sensitive to green). Uses the BT709 formula by default: Red: 0.2125, Green: 0.7154, Blue: 0.0721. The user can set custom weightings.
16-
- **Average [1]**: Computes a simple arithmetic average of R, G, and B channel values: (R + G + B) / 3.
17-
- **Lightness [2]**: Averages the maximum and minimum channel values: (max(R, G, B) + min(R, G, B)) / 2.
18-
- **SingleChannel [3]**: The user selects a specific R, G, or B channel to use directly as the grayscale values.
15+
- **Luminosity [0]**: weighted average accounting for human perception (more sensitive to green). Default weights are the BT.709 standard: R=0.2125, G=0.7154, B=0.0721. The user may supply custom weights.
16+
- **Average [1]**: simple arithmetic mean of the three channels: (R + G + B) / 3.
17+
- **Lightness [2]**: average of the maximum and minimum channel values: (max(R, G, B) + min(R, G, B)) / 2.
18+
- **SingleChannel [3]**: the user picks one channel (R, G, or B) and that channel's value is used directly.
1919

20-
The user can select 1 or more image data arrays which are assumed to be multi-component arrays of unsigned 8 bit values. The user can create a new AttributeMatrix if they want to store all the newly created arrays in a separate AttributeMatrix.
20+
### Additional Standard Weights
2121

22-
### Additional GrayScale Conversions
22+
Other widely-used weighting schemes (not built-in, but easy to supply as custom Luminosity weights):
2323

24-
The following are some additional accepted grayscale conversions
24+
- **RMY Greyscale**: R=0.500, G=0.419, B=0.081
25+
- **YIQ / NTSC**: R=0.299, G=0.587, B=0.114
2526

26-
+ RMY Greyscale: Red: 0.5 Green: 0.419 Blue: 0.081
27-
+ (YIQ/NTSC): Red: 0.299 Green: 0.587 Blue: 0.114
27+
### Input/Output Format
28+
29+
- **Input arrays** must be uint8 with 3 components (RGB) or 4 components (RGBA). With RGBA input, the alpha channel is ignored.
30+
- **Output arrays** are single-component uint8.
31+
32+
Multiple input arrays can be processed in one filter pass; each produces its own output array. If *Output to New Attribute Matrix* is enabled, the new arrays are placed in a separate Attribute Matrix to keep them organized; otherwise they live alongside the originals.
33+
34+
### Required Input Sources
35+
36+
- **Input Image Arrays** -- 3- or 4-component uint8 arrays, typically produced by [ITK Import Image Stack](../ITKImageProcessing/ITKImportImageStackFilter.md) or another color-image reader.
2837

2938
% Auto generated parameter table will be inserted here
3039

src/Plugins/SimplnxCore/docs/ConvertDataFilter.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,15 @@ Down casting can have undefined behavior depending on the primitive types involv
4040

4141
When converting data from signed values to unsigned values or vice-versa, there can also be undefined behavior. For example, if the user were to convert a signed 4 byte integer array to an unsigned 4 byte integer array and the input array has negative values, then the conversion rules are undefined and may differ from operating system to operating system.
4242

43+
### Related Filters
44+
45+
- [Reshape Data Array](ReshapeDataArrayFilter.md) -- changes the tuple-dimensions interpretation of an array without converting its underlying data type. Use Reshape (not Convert) when you want to interpret existing bytes under a different shape.
46+
- [Create Data Array](CreateDataArrayFilter.md) -- create a fresh array of the desired type and copy values into it manually.
47+
48+
### Required Input Sources
49+
50+
- **Input Array** -- the **Data Array** whose values will be cast to the new type.
51+
4352
% Auto generated parameter table will be inserted here
4453

4554
## Example Pipelines

src/Plugins/SimplnxCore/docs/CopyDataObjectFilter.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,12 @@ Commonly used _BaseGroup_ children:
1919

2020
See the DataStructure section of the reference manual for a complete hierarchy.
2121

22-
When the _Copy to New Parent_ is toggled true a new parameter will appear. This parameter, _Copied Parent Group_, allows for the selected arrays to all be copied into whatever data container you place here.
22+
When *Copy to New Parent* is toggled on, a new parameter appears. This parameter, *Copied Parent Group*, lets the selected objects be copied into a different parent **DataGroup**, **Attribute Matrix**, or **Geometry** rather than alongside their originals.
23+
24+
### Required Input Sources
25+
26+
- **Objects to Copy** -- one or more existing **DataObjects** in the Data Structure.
27+
- **Copied Parent Group** (only when *Copy to New Parent* is enabled) -- an existing parent container where the copies will be placed.
2328

2429
% Auto generated parameter table will be inserted here
2530

src/Plugins/SimplnxCore/docs/CopyFeatureArrayToElementArrayFilter.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,19 @@ Core (Memory/Management)
66

77
## Description
88

9-
This **Filter** copies the values associated with a **Feature** to all the **Elements** that belong to that **Feature**. Xmdf visualization files write only the **Element** attributes, so if the user wants to display a spatial map of a **Feature** level attribute, this **Filter** will transfer that information down to the **Element** level.
9+
This **Filter** broadcasts a per-**Feature** value to every **Cell** (element) that belongs to that Feature, producing a new cell-level **Data Array** that has the same length as the Feature Ids array. Each cell receives the value of the Feature it belongs to.
10+
11+
### When to Use This Filter
12+
13+
- **Visualization.** Most exporters (Xdmf, VTK, ParaView state files) treat Cell-level arrays as the natural per-voxel scalar to render. Broadcasting a per-grain statistic (size, average misorientation, Schmid factor, biased flag, etc.) down to the Cell level produces a colorable map of that quantity.
14+
- **Downstream filtering on cells.** When you want to threshold cells based on a Feature-level criterion (e.g., "keep cells whose parent grain is unbiased"), it is simpler to broadcast the Feature flag to all cells and then apply a cell-level threshold.
15+
16+
The output is always the same data type and component shape as the source Feature array.
17+
18+
### Required Input Sources
19+
20+
- **Feature Array to Copy** -- any feature-level array; common sources include [Compute Feature Sizes](ComputeFeatureSizesFilter.md), [Compute Average Orientations](../OrientationAnalysis/ComputeAvgOrientationsFilter.md), [Compute Schmid Factors](../OrientationAnalysis/ComputeSchmidsFilter.md), or a custom flag from [Compute Biased Features](ComputeBiasedFeaturesFilter.md) / [Compute Surface Features](ComputeSurfaceFeaturesFilter.md).
21+
- **Cell Feature Ids** -- produced by a segmentation filter such as [Segment Features (Misorientation)](../OrientationAnalysis/EBSDSegmentFeaturesFilter.md) or [Segment Features (Scalar)](ScalarSegmentFeaturesFilter.md).
1022

1123
% Auto generated parameter table will be inserted here
1224

src/Plugins/SimplnxCore/docs/CreateAttributeMatrixFilter.md

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,29 @@ Core (Generation)
66

77
## Description
88

9-
This **Filter** creates a new **Attribute Matrix**.
9+
This **Filter** creates a new empty **Attribute Matrix** at a user-specified location in the **Data Structure**. The matrix is created with a user-specified set of *tuple dimensions* but contains no arrays initially; subsequent filters can add arrays to it.
1010

11-
### Example Usage
11+
### What is an Attribute Matrix?
1212

13-
If you wanted to create an **Attribute Matrix** to represent a 3D volume where the dimensions of the 3 orthogonal axesare X=3, y=4 and Z=5, then the *Tuple Dimensions* would have a value of (3, 4, 5).
13+
An **Attribute Matrix** is a container for **Data Arrays** that share the same *number of tuples* and the same *tuple shape*. All arrays inside one Attribute Matrix represent values "at the same set of points" -- for example, all per-cell arrays of an Image Geometry live in a single Cell Attribute Matrix, all per-feature arrays for a segmented dataset live in a single Feature Attribute Matrix, and all per-ensemble (phase) arrays live in an Ensemble Attribute Matrix.
14+
15+
This shared-tuple-shape contract is enforced: any array added to an existing Attribute Matrix must match the matrix's tuple dimensions. Use an Attribute Matrix when the values you are grouping are all parallel arrays over the same domain.
16+
17+
### When to Use This Filter
18+
19+
- Pre-building the destination for a future filter that creates new feature-level or ensemble-level arrays.
20+
- Creating a custom Attribute Matrix for organizational purposes (e.g., grouping all "computed" arrays separately from the "raw" arrays in the same Cell-level domain).
21+
- Restructuring imported data so that arrays sharing a common dimension end up in the same matrix.
22+
23+
For containers that hold heterogeneous data (arrays of different sizes, geometries, sub-groups), use [Create Data Group](CreateDataGroupFilter.md) instead.
24+
25+
### Tuple Dimensions
26+
27+
For a 3D volume with axes X=3, Y=4, Z=5, the *Tuple Dimensions* are (3, 4, 5). For a 1D feature-list with 100 features, the tuple dimensions are (100). The product of all dimensions equals the total number of tuples that any array added to this matrix must have.
28+
29+
### Required Input Sources
30+
31+
- **Parent Data Object Path** -- an existing **DataGroup** or **Geometry** under which the new Attribute Matrix will be created. Typically the output of [Create Image Geometry](CreateImageGeometryFilter.md), [Create Geometry](CreateGeometryFilter.md), or [Create Data Group](CreateDataGroupFilter.md).
1432

1533
% Auto generated parameter table will be inserted here
1634

0 commit comments

Comments
 (0)