[ConstantFolding] Fold vector.partial.reduce.add constants - #212112
[ConstantFolding] Fold vector.partial.reduce.add constants#212112JihyeonJeong129 wants to merge 1 commit into
Conversation
|
Hello @JihyeonJeong129 👋 Thank you for submitting a Pull Request (PR) to the LLVM Project. Since this is your first PR, here are a few useful links covering our main contribution policies and review practices.
Please reply to this message to confirm that you have read these policies, especially the LLVM AI Tool Use Policy, and that any AI tool usage has been noted in the PR description. Frequently asked questionsHow do I add reviewers? This PR will be automatically labeled, and the relevant teams will be notified. For some parts of the project, reviewers may also be added automatically. You can also add reviewers manually using the Reviewers section on this page. If you cannot use that section, it is probably because you do not have write permissions for the repository. In that case, you can request a review by tagging reviewers in a comment using What if there are no comments? If you have not received any comments on your PR after a week, you can request a review by pinging the PR with a comment such as “Ping”. The common courtesy ping rate is once a week. Please remember that you are asking for volunteer time from other developers. Are any special GitHub settings required to contribute to LLVM? We only require contributors to have a public email address associated with their GitHub commits, see this section of LLVM Developer Policy for details. If you have questions, feel free to leave a comment on this PR, or ask on LLVM Discord or LLVM Discourse. Thank you, |
llvmorg-github-actions
Bot
added
llvm:instcombine
|
@llvm/pr-subscribers-llvm-transforms @llvm/pr-subscribers-llvm-analysis Author: Jeong Jihyeon (JihyeonJeong129) ChangesThis patch adds constant folding support for The intrinsic leaves the grouping of input elements into result lanes Tests cover:
Testing:
Fixes #211558 Full diff: https://github.com/llvm/llvm-project/pull/212112.diff 2 Files Affected:
diff --git a/llvm/lib/Analysis/ConstantFolding.cpp b/llvm/lib/Analysis/ConstantFolding.cpp
index 8901a5384e37e..96b99b1b7822f 100644
--- a/llvm/lib/Analysis/ConstantFolding.cpp
+++ b/llvm/lib/Analysis/ConstantFolding.cpp
@@ -1792,6 +1792,7 @@ bool llvm::canConstantFoldCallTo(const CallBase *Call, const Function *F) {
case Intrinsic::vector_reduce_smax:
case Intrinsic::vector_reduce_umin:
case Intrinsic::vector_reduce_umax:
+ case Intrinsic::vector_partial_reduce_add:
case Intrinsic::vector_extract:
case Intrinsic::vector_insert:
case Intrinsic::vector_interleave2:
@@ -2393,6 +2394,50 @@ Constant *constantFoldVectorReduce(Intrinsic::ID IID, Constant *Op) {
return ConstantInt::get(Op->getContext(), Acc);
}
+/// Fold a vector partial reduction add using the deterministic grouping
+/// chosen by TargetLowering::expandPartialReduceMLA. Although the
+/// LangRef leaves the grouping unspecified, input element I is accumulated
+/// into result lane I % NumAccElts, with each accumulator element seeding
+/// its corresponding result lane. Returns nullptr if any element cannot be
+/// folded.
+static Constant *constantFoldVectorPartialReduceAdd(Constant *Acc,
+ Constant *Input,
+ const DataLayout &DL) {
+ auto *AccTy = cast<FixedVectorType>(Acc->getType());
+ auto *InputTy = cast<FixedVectorType>(Input->getType());
+
+ unsigned NumAccElts = AccTy->getNumElements();
+ unsigned NumInputElts = InputTy->getNumElements();
+
+ SmallVector<Constant *> ResultElts;
+ ResultElts.reserve(NumAccElts);
+
+ for (unsigned I = 0; I < NumAccElts; ++I) {
+ Constant *AccElt = Acc->getAggregateElement(I);
+
+ if (!AccElt)
+ return nullptr;
+
+ ResultElts.push_back(AccElt);
+ }
+
+ for (unsigned I = 0; I < NumInputElts; ++I) {
+ Constant *InputElt = Input->getAggregateElement(I);
+ if (!InputElt)
+ return nullptr;
+
+ unsigned ResultIdx = I % NumAccElts;
+ Constant *Folded = ConstantFoldBinaryOpOperands(
+ Instruction::Add, ResultElts[ResultIdx], InputElt, DL);
+ if (!Folded)
+ return nullptr;
+
+ ResultElts[ResultIdx] = Folded;
+ }
+
+ return ConstantVector::get(ResultElts);
+}
+
/// Attempt to fold an SSE floating point to integer conversion of a constant
/// floating point. If roundTowardZero is false, the default IEEE rounding is
/// used (toward nearest, ties to even). This matches the behavior of the
@@ -4442,6 +4487,8 @@ static Constant *ConstantFoldFixedVectorCall(
}
return ConstantVector::get(Result);
}
+ case Intrinsic::vector_partial_reduce_add:
+ return constantFoldVectorPartialReduceAdd(Operands[0], Operands[1], DL);
case Intrinsic::wasm_dot: {
unsigned NumElements =
cast<FixedVectorType>(Operands[0]->getType())->getNumElements();
diff --git a/llvm/test/Transforms/InstSimplify/ConstProp/vecreduce.ll b/llvm/test/Transforms/InstSimplify/ConstProp/vecreduce.ll
index 479b3f8ea4128..e62cd8961065d 100644
--- a/llvm/test/Transforms/InstSimplify/ConstProp/vecreduce.ll
+++ b/llvm/test/Transforms/InstSimplify/ConstProp/vecreduce.ll
@@ -928,3 +928,107 @@ define i32 @umax_poison_elt() {
%x = call i32 @llvm.vector.reduce.umax.v8i32(<8 x i32> <i32 1, i32 1, i32 poison, i32 1, i32 1, i32 poison, i32 1, i32 1>)
ret i32 %x
}
+
+define <4 x i32> @partial_reduce_add_constants() {
+; CHECK-LABEL: @partial_reduce_add_constants(
+; CHECK-NEXT: ret <4 x i32> <i32 128, i32 232, i32 336, i32 440>
+;
+ %x = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(
+ <4 x i32> <i32 100, i32 200, i32 300, i32 400>,
+ <16 x i32> <i32 1, i32 2, i32 3, i32 4,
+ i32 5, i32 6, i32 7, i32 8,
+ i32 9, i32 10, i32 11, i32 12,
+ i32 13, i32 14, i32 15, i32 16>)
+ ret <4 x i32> %x
+}
+
+define <4 x i32> @partial_reduce_add_nonconstant_acc(<4 x i32> %acc) {
+; CHECK-LABEL: @partial_reduce_add_nonconstant_acc(
+; CHECK-NEXT: [[X:%.*]] = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(<4 x i32> [[ACC:%.*]], <16 x i32> <i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15, i32 16>)
+; CHECK-NEXT: ret <4 x i32> [[X]]
+;
+ %x = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(
+ <4 x i32> %acc,
+ <16 x i32> <i32 1, i32 2, i32 3, i32 4,
+ i32 5, i32 6, i32 7, i32 8,
+ i32 9, i32 10, i32 11, i32 12,
+ i32 13, i32 14, i32 15, i32 16>)
+ ret <4 x i32> %x
+}
+
+define <4 x i32> @partial_reduce_add_nonconstant_input(<16 x i32> %input) {
+; CHECK-LABEL: @partial_reduce_add_nonconstant_input(
+; CHECK-NEXT: [[X:%.*]] = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(<4 x i32> <i32 100, i32 200, i32 300, i32 400>, <16 x i32> [[INPUT:%.*]])
+; CHECK-NEXT: ret <4 x i32> [[X]]
+;
+ %x = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(
+ <4 x i32> <i32 100, i32 200, i32 300, i32 400>,
+ <16 x i32> %input)
+ ret <4 x i32> %x
+}
+
+define <4 x i32> @partial_reduce_add_poison_element() {
+; CHECK-LABEL: @partial_reduce_add_poison_element(
+; CHECK-NEXT: ret <4 x i32> <i32 128, i32 poison, i32 336, i32 440>
+;
+ %x = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(
+ <4 x i32> <i32 100, i32 200, i32 300, i32 400>,
+ <16 x i32> <i32 1, i32 2, i32 3, i32 4,
+ i32 5, i32 poison, i32 7, i32 8,
+ i32 9, i32 10, i32 11, i32 12,
+ i32 13, i32 14, i32 15, i32 16>)
+ ret <4 x i32> %x
+}
+
+define <4 x i32> @partial_reduce_add_undef_element() {
+; CHECK-LABEL: @partial_reduce_add_undef_element(
+; CHECK-NEXT: ret <4 x i32> <i32 128, i32 undef, i32 336, i32 440>
+;
+ %x = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v16i32(
+ <4 x i32> <i32 100, i32 200, i32 300, i32 400>,
+ <16 x i32> <i32 1, i32 2, i32 3, i32 4,
+ i32 5, i32 undef, i32 7, i32 8,
+ i32 9, i32 10, i32 11, i32 12,
+ i32 13, i32 14, i32 15, i32 16>)
+ ret <4 x i32> %x
+}
+
+define <4 x i32> @partial_reduce_add_ratio_one() {
+; CHECK-LABEL: @partial_reduce_add_ratio_one(
+; CHECK-NEXT: ret <4 x i32> <i32 101, i32 202, i32 303, i32 404>
+;
+ %x = call <4 x i32> @llvm.vector.partial.reduce.add.v4i32.v4i32(
+ <4 x i32> <i32 100, i32 200, i32 300, i32 400>,
+ <4 x i32> <i32 1, i32 2, i32 3, i32 4>)
+ ret <4 x i32> %x
+}
+
+define <2 x i32> @partial_reduce_add_ratio_two() {
+; CHECK-LABEL: @partial_reduce_add_ratio_two(
+; CHECK-NEXT: ret <2 x i32> <i32 104, i32 206>
+;
+ %x = call <2 x i32> @llvm.vector.partial.reduce.add.v2i32.v4i32(
+ <2 x i32> <i32 100, i32 200>,
+ <4 x i32> <i32 1, i32 2, i32 3, i32 4>)
+ ret <2 x i32> %x
+}
+
+define <2 x i32> @partial_reduce_add_negative() {
+; CHECK-LABEL: @partial_reduce_add_negative(
+; CHECK-NEXT: ret <2 x i32> <i32 -96, i32 -194>
+;
+ %x = call <2 x i32> @llvm.vector.partial.reduce.add.v2i32.v4i32(
+ <2 x i32> <i32 -100, i32 -200>,
+ <4 x i32> <i32 1, i32 2, i32 3, i32 4>)
+ ret <2 x i32> %x
+}
+
+define <2 x i8> @partial_reduce_add_wrap() {
+; CHECK-LABEL: @partial_reduce_add_wrap(
+; CHECK-NEXT: ret <2 x i8> <i8 -126, i8 -125>
+;
+ %x = call <2 x i8> @llvm.vector.partial.reduce.add.v2i8.v4i8(
+ <2 x i8> <i8 127, i8 126>,
+ <4 x i8> <i8 1, i8 1, i8 2, i8 4>)
+ ret <2 x i8> %x
+}
|
|
@RKSimon could you please review this PR when you have time? This is my first contribution to LLVM, so I would especially appreciate any feedback on the implementation, test coverage, or coding style. |
You can test this locally with the following command:git diff -U0 --pickaxe-regex -S '([^a-zA-Z0-9#_-]undef([^a-zA-Z0-9_-]|$)|UndefValue::get)' 'HEAD~1' HEAD llvm/lib/Analysis/ConstantFolding.cpp llvm/test/Transforms/InstSimplify/ConstProp/vecreduce.llThe following files introduce new uses of undef:
Undef is now deprecated and should only be used in the rare cases where no replacement is possible. For example, a load of uninitialized memory yields In tests, avoid using For example, this is considered a bad practice: define void @fn() {
...
br i1 undef, ...
}Please use the following instead: define void @fn(i1 %cond) {
...
br i1 %cond, ...
}Please refer to the Undefined Behavior Manual for more information. |
|
@RKSimon The However, the undef deprecator flags this newly added use. Would you prefer that I remove this test, or is there an accepted way to keep an intentional |
|
I'm on PTO at the moment, not sure when I can help |
2 participants
This patch adds constant folding support for
llvm.vector.partial.reduce.add.The intrinsic leaves the grouping of input elements into result lanes
unspecified. This implementation uses the deterministic grouping selected by
the generic lowering in
TargetLowering::expandPartialReduceMLA: input elementIis accumulated into result laneI % NumAccElts.Tests cover:
Testing:
llvm/test/Transforms/InstSimplify/ConstProp/vecreduce.llninja -C build check-llvmFixes #211558