From: Kyrylo Tkachov <[email protected]>
The documentation of widen_ssum and widen_usum says only that operand 1 is
added to operand 2, which reads as though each element of operand 1 has to
be accumulated into the element of operand 0 at the same position. Nothing
says the pattern is only ever used for a reassociable reduction, so a port
cannot tell from the documentation whether it is allowed to regroup the
input elements, for example by using a pairwise widening add.
vect_recog_widen_sum_pattern only forms WIDEN_SUM_EXPR through
vect_reassociating_reduction_p, which requires the
statement to be a reduction, refuses a statement nested in the inner loop
of an outer-loop vectorization because the order of the computation matters
there, and refuses a type that needs a fold-left reduction. The comment on
the pattern already describes the idiom as producing N/2 results by summing
up pairs of intermediate results. The accumulator is only ever consumed by
a horizontal sum in the epilogue, so the distribution of input elements
over accumulator lanes is not observable.
Spell that out, and note the constraint that comes with it: an
implementation that adds elements before they reach the width of operand 0
has to widen them first so that no intermediate sum can overflow.
Is it okay to document this relaxation so that patch 2 can make use of
it in the aarch64 backend?
Thanks,
Kyrill
gcc/ChangeLog:
* doc/md.texi (widen_ssum@var{n}@var{m}3, widen_usum@var{n}@var{m}3):
Document that the assignment of input elements to accumulator
elements is unconstrained, and fix a typo.
Signed-off-by: Kyrylo Tkachov <[email protected]>
---
gcc/doc/md.texi | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
index c0026c17318..3235d456874 100644
--- a/gcc/doc/md.texi
+++ b/gcc/doc/md.texi
@@ -5097,10 +5097,21 @@ in operand 0, which is of the same mode as operand 3.
@itemx @samp{widen_usum@var{n}@var{m}3}
Operands 0 and 2 are of the same mode, which is wider than the mode of
operand 1. Add operand 1 to operand 2 and place the widened result in
-operand 0. (This is used express accumulation of elements into an accumulator
-of a wider mode.)
+operand 0. (This is used to express accumulation of elements into an
+accumulator of a wider mode.)
@var{m} is the mode of operand 1 and @var{n} is the mode of operand 0.
+These patterns are only used for a reduction whose summation the vectorizer
+has already established may be reassociated, and the accumulator is reduced
+to a scalar by a horizontal sum once the loop is done. An implementation is
+therefore free to choose which elements of operand 1 it accumulates into
+which element of operand 0, so long as every element of operand 1 is
+accumulated exactly once. In particular it may add adjacent elements of
+operand 1 to each other before accumulating them, which is what a pairwise
+widening add instruction does. Doing so must not lose any value, so an
+implementation that adds elements together before they reach the width of
+operand 0 has to widen them enough for the intermediate sums to be exact.
+
@mdindex smulhs@var{m}3
@mdindex umulhs@var{m}3
@item @samp{smulhs@var{m}3}
--
2.50.1 (Apple Git-155)
