Commit ee69e51
authored
[BWARE] Track more compressed-friendly ops in FederatedWorkloadAnalyzer (#2481)
* Track more compressed-friendly ops in FederatedWorkloadAnalyzer
Extends the federated workload counter so that compression decisions
account for additional instruction shapes beyond AggregateBinary.
- Pass the right-hand column count to incOverlappingDecompressions
so the cost model reflects the actual decompression size rather
than counting a single column
- Count MMChainCPInstruction as one LMM and one RMM contribution per
invocation
- Count AggregateUnaryCPInstruction: when reducing columns with a
sum/mean operator, treat it as a dict-op (compression-friendly);
otherwise count it as a decompression
- Minor formatting cleanup in compressRun
* Add unit tests for FederatedWorkloadAnalyzer workload tracking
Cover the instruction-shape branches in incrementWorkload that drive
federated compression decisions, which previously had no direct tests:
- AggregateBinary: RMM/LMM counting, overlapping-decompress sizing by the
right-hand column count, and the validSize row/column guards
- MMChain: one LMM and one RMM contribution per invocation
- AggregateUnary: dict-op vs decompression classification across
ReduceAll/ReduceRow/ReduceCol with sum, mean, product, and max operators
- Instance-level dispatch and compressRun threshold behavior, asserting
async compression materializes when the cost model would compress1 parent 3b564bb commit ee69e51
2 files changed
Lines changed: 388 additions & 5 deletions
File tree
- src
- main/java/org/apache/sysds/runtime/controlprogram/federated
- test/java/org/apache/sysds/test/component/federated
Lines changed: 45 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
30 | 35 | | |
31 | 36 | | |
| 37 | + | |
32 | 38 | | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
33 | 42 | | |
34 | 43 | | |
35 | 44 | | |
| |||
55 | 64 | | |
56 | 65 | | |
57 | 66 | | |
58 | | - | |
| 67 | + | |
59 | 68 | | |
60 | 69 | | |
61 | 70 | | |
| |||
68 | 77 | | |
69 | 78 | | |
70 | 79 | | |
| 80 | + | |
71 | 81 | | |
72 | 82 | | |
73 | 83 | | |
| |||
81 | 91 | | |
82 | 92 | | |
83 | 93 | | |
84 | | - | |
| 94 | + | |
85 | 95 | | |
86 | 96 | | |
87 | 97 | | |
88 | 98 | | |
89 | 99 | | |
90 | 100 | | |
91 | | - | |
92 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
93 | 133 | | |
94 | 134 | | |
95 | 135 | | |
| |||
117 | 157 | | |
118 | 158 | | |
119 | 159 | | |
120 | | - | |
121 | | - | |
| 160 | + | |
| 161 | + | |
122 | 162 | | |
123 | 163 | | |
124 | 164 | | |
| |||
0 commit comments