[SYSTEMDS-3333] Add a DP optimization for matrix chains with transposes

This adds a new HOP rewrite rule,
RewriteMatrixMultChainWithTransOptimization.java, to find the optimal
execution plan for matrix multiplication chains containing transposes.
Previously, these chains were optimized using a simple heuristic that
just pushes transposes down from t(A %*% B) -> t(B) %*% t(A), which
fails to be the optimal plan in some instances especially with large
matrices.

An example would be R = t(A %*% B) %*% C with dimensions A = [16, 23], B
= [23, 22], C = [16, 34]
which would be according to the old rewrite class solved with (t(B) %*%
t(A)) %*% C -> costs: t(B) -> 23*22 + t(A) -> 16 * 23 + t(B) %*% t(A) ->
22*23*16 + [...] %*% C -> 22*16*34 = 20938 FLOPs
Optimal would be simply: t(A %*% B) %*% C - costs: A %*% B -> 16*23*22 +
t(A %*% B) -> 16*22 + [...] %*% C -> 22*16*34 = 20416 FLOPs - difference
gets larger with higher matrix dimensions.

To solve this, we applied a DP Algorithm with a Memo Table containing
Plans without transposing and Plans containing Transposing subchains
calculating wether an algebraic transpose pushdown or direct transpose
operation is cheaper.

This also includes 24 automated DML test cases asserting intermediate
HOP dimensions to validate optimal parenthesization and transpose
placement. = 20938 FLOPs
Optimal would be simply: t(A %*% B) %*% C - costs: A %*% B -> 16*23*22 +
t(A %*% B) -> 16*22 + [...] %*% C -> 22*16*34 = 20416 FLOPs - difference
gets larger with higher matrix dimensions.

To solve this, we applied a DP Algorithm with a Memo Table containing
Plans without transposing and Plans containing Transposing subchains
calculating wether an algebraic transpose pushdown or direct transpose
operation is cheaper.

This also includes 24 automated DML test cases asserting intermediate
HOP dimensions to validate optimal parenthesization and transpose
placement.

Closes #2465.

Author: Elmanjhg

pom.xml

@@ -1577,6 +1577,5 @@
 			<artifactId>fastdoubleparser</artifactId>
 			<version>0.9.0</version>
 		</dependency>
-
 	</dependencies>
 </project>

src/main/java/org/apache/sysds/hops/OptimizerUtils.java

@@ -204,6 +204,11 @@ public enum MemoryManager {
 	 * ALLOW_SUM_PRODUCT_REWRITES.
 	 */
 	public static boolean ALLOW_ADVANCED_MMCHAIN_REWRITES = false;
+
+	/**
+	 * Enables a DPSize inspired algorithm rewrite for MMChain with transposes
+	 */
+	public static boolean ALLOW_NEW_MMCHAIN_REWRITE = false;
 
 	/**
 	 * Enables a specific hop dag rewrite that splits hop dags after csv persistent reads with 

src/main/java/org/apache/sysds/hops/rewrite/ProgramRewriter.java

@@ -136,6 +136,9 @@ public ProgramRewriter(boolean staticRewrites, boolean dynamicRewrites)
 				if( OptimizerUtils.ALLOW_SUM_PRODUCT_REWRITES2 )
 					_dagRuleSet.add( new RewriteElementwiseMultChainOptimization()); //dependency: cse
 			}
+			if( OptimizerUtils.ALLOW_NEW_MMCHAIN_REWRITE ) {
+				_dagRuleSet.add( new RewriteMatrixMultChainWithTransOptimization()		);
+			}
 			if(OptimizerUtils.ALLOW_ADVANCED_MMCHAIN_REWRITES){
 				_dagRuleSet.add( new RewriteMatrixMultChainOptimizationTranspose()      ); //dependency: cse
 				_dagRuleSet.add( new RewriteMatrixMultChainOptimizationSparse()         ); //dependency: cse