On Mon, 10 Aug 2026, [email protected] wrote:
> From: Kyrylo Tkachov <[email protected]>
>
> The documentation of the reduc_widen_ssum and reduc_widen_usum patterns says
> only that operand 1 is added to operand 2, which reads as though each element
> of operand 1 has to be accumulated into the element of operand 0 at the same
> position. Nothing says the patterns are only ever used for a reassociable
> reduction, so a port cannot tell whether it is allowed to regroup the input
> elements, for example by using a pairwise widening add.
>
> vect_recog_widen_sum_pattern only forms WIDEN_SUM_EXPR
> through vect_reassociating_reduction_p, which requires the statement to be a
> reduction, refuses a statement nested in the inner loop of an outer-loop
> vectorization because the order of the computation matters there, and refuses
> a type that needs a fold-left reduction. The comment on the pattern already
> describes the idiom as producing N/2 results by summing up pairs of
> intermediate results. The accumulator is only ever consumed by a horizontal
> sum in the epilogue, so the distribution of input elements over accumulator
> elements is not observable.
>
> Spell that out in md.texi, along with the constraint that comes with it: an
> implementation that adds elements together before they reach the element width
> of operand 0 has to widen them first so that no intermediate sum can overflow.
> Note that the widen_[us]sum names are left free for a lane preserving pattern.
>
> Give WIDEN_SUM_EXPR the same statement in tree.def. DOT_PROD_EXPR and
> SAD_EXPR
> are formed through vect_reassociating_reduction_p as well and read the same
> way,
> so add a sentence to each pointing at the WIDEN_SUM_EXPR rule.
>
> Ok for trunk?
OK.
Thanks,
Richard.
> Thanks,
> Kyrill
>
> gcc/ChangeLog:
>
> * doc/md.texi (reduc_widen_ssum@var{n}@var{m}3)
> (reduc_widen_usum@var{n}@var{m}3): Document that the assignment of
> input elements to accumulator elements is unconstrained, that
> intermediate sums must not overflow, and that the widen_[us]sum
> names are reserved. Fix a typo.
> * tree.def (WIDEN_SUM_EXPR): Document that the assignment of elements
> of the first argument to elements of the second is unconstrained.
> (DOT_PROD_EXPR, SAD_EXPR): Note the same freedom.
>
> Signed-off-by: Kyrylo Tkachov <[email protected]>
> ---
> gcc/doc/md.texi | 23 +++++++++++++++++++++--
> gcc/tree.def | 19 +++++++++++++++++--
> 2 files changed, 38 insertions(+), 4 deletions(-)
>
> diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
> index 4b3dc950aeb..f86fa8038e8 100644
> --- a/gcc/doc/md.texi
> +++ b/gcc/doc/md.texi
> @@ -5097,10 +5097,29 @@ in operand 0, which is of the same mode as operand 3.
> @itemx @samp{reduc_widen_usum@var{n}@var{m}3}
> Operands 0 and 2 are of the same mode, which is wider than the mode of
> operand 1. Add operand 1 to operand 2 and place the widened result in
> -operand 0. (This is used express accumulation of elements into an accumulator
> -of a wider mode.)
> +operand 0. (This is used to express accumulation of elements into an
> +accumulator of a wider mode.)
> @var{m} is the mode of operand 1 and @var{n} is the mode of operand 0.
>
> +These patterns are only used for a reduction whose summation the vectorizer
> +has already established may be reassociated. Operand 0 is the accumulator
> +of that reduction and is itself reduced to a scalar by a horizontal sum once
> +the loop is done, so which element of the accumulator a given input element
> +ends up in is not observable. An implementation is therefore free to choose
> +which elements of operand 1 it accumulates into which element of operand 0,
> +provided that every element of operand 1 is accumulated exactly once. In
> +particular it may add adjacent elements of operand 1 to each other before
> +accumulating them, which is what a pairwise widening add instruction does.
> +
> +The intermediate sums must not overflow. An implementation that combines
> +elements of operand 1 with each other before they reach the element width of
> +operand 0 therefore has to widen them first.
> +
> +The names @samp{widen_ssum@var{n}@var{m}3} and
> +@samp{widen_usum@var{n}@var{m}3} are deliberately left free for a future
> +lane preserving pattern, which would have to add element @var{i} of
> +operand 1 to element @var{i} of operand 2 for every @var{i}.
> +
> @mdindex smulhs@var{m}3
> @mdindex umulhs@var{m}3
> @item @samp{smulhs@var{m}3}
> diff --git a/gcc/tree.def b/gcc/tree.def
> index ad8e676e942..2e767dbe045 100644
> --- a/gcc/tree.def
> +++ b/gcc/tree.def
> @@ -1465,7 +1465,10 @@ DEFTREECODE (TRANSACTION_EXPR, "transaction_expr",
> tcc_expression, 1)
> arg3 = PLUS_EXPR (tmp, arg3);
> or:
> tmp = WIDEN_MULT_EXPR(arg1, arg2);
> - arg3 = WIDEN_SUM_EXPR (tmp, arg3); */
> + arg3 = WIDEN_SUM_EXPR (tmp, arg3);
> + Like WIDEN_SUM_EXPR, this code is only created for a reduction whose
> + summation may be reassociated, so which products are added into which
> + element of arg3 is not observable. */
> DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3)
>
> /* Widening summation.
> @@ -1474,7 +1477,16 @@ DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr",
> tcc_expression, 3)
> the size of t1. The type of the entire expression is also t2.
> WIDEN_SUM_EXPR is equivalent to first widening (promoting)
> the first argument from type t1 to type t2, and then summing it
> - with the second argument. */
> + with the second argument.
> +
> + This code is only created for a reduction whose summation the vectorizer
> + has established may be reassociated, and the result is reduced to a scalar
> + by a horizontal sum once the loop is done. Which element of the second
> + argument a given element of the first argument is added to is therefore
> + not observable, and an expander may choose that assignment freely, so long
> + as every element of the first argument is accumulated exactly once and no
> + intermediate sum overflows. See the reduc_widen_ssum and reduc_widen_usum
> + patterns in md.texi. */
> DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2)
>
> /* Widening sad (sum of absolute differences).
> @@ -1490,6 +1502,9 @@ DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr",
> tcc_binary, 2)
> tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2)
> tmp2 = ABS_EXPR (tmp)
> arg3 = WIDEN_SUM_EXPR (tmp2, arg3)
> + Like WIDEN_SUM_EXPR, this code is only created for a reduction whose
> + summation may be reassociated, so which absolute differences are added
> + into which element of arg3 is not observable.
> */
> DEFTREECODE (SAD_EXPR, "sad_expr", tcc_expression, 3)
>
>
--
Richard Biener <[email protected]>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Jochen Jaser, Andrew McDonald, Abhinav Puri; (HRB 36809, AG Nuernberg)
