On Wed, May 26, 2021 at 10:28:16AM +0200, Richard Biener wrote: > > > > > > > > -- Target Hook: rtx TARGET_GEN_MEMSET_VALUE (rtx DATA, scalar_int_mode > > > > MODE) > > > > This function returns the RTL of a register containing > > > > 'GET_MODE_SIZE (MODE)' consecutive copies of the unsigned char > > > > value given in the RTL register DATA. For example, if MODE is 4 > > > > bytes wide, return the RTL for 0x01010101*DATA. > > > > > > For this one I wonder if it should be an optab instead. Couldn't you > > > use the existing vec_duplicate for this by using (paradoxical) subregs > > > like (subreg:TI (vec_duplicate:VnQI (subreg:VnQI (reg:QI ...)))? > > > > I tried. It doesn't even work on x86. See: > > > > https://gcc.gnu.org/pipermail/gcc-patches/2021-May/570661.html > > Not sure what I should read from there... > > > There are special cases to subreg HI, SI and DI modes of TI mode in > > ix86_gen_memset_value_from_prev. simplify_gen_subreg doesn't > > work here. Each backend may need its own special handling. > > OK, I guess I'm not (RTL) qualified enough to further review these parts, > sorry. Since we're doing code generation the canonical way to communicate > with backends should be optabs, not some set of disconnected target hooks. > But as said, I probably don't know enough of RTL to see why it's the only way. > > Richard.
Here is the patch to add optabs instead. Does it look OK? Thanks. H.J. --- Add 2 optabs: 1. integer_extract: Extract lower bit value from the integer value in TImode, OImode or XImode. 2. vec_const_duplicate: Broadcast a QImode constant to a vector. It is similar to vec_duplicate. Since the resulting vector is computable at compile-time, vec_duplicate may not be faster and backend can opt out broadcasting from a constant while opting in broadcasting from a variable. and rewrite builtin_memset_read_str/builtin_memset_gen_str to support target instructions to duplicate QImode value to TImode, OImode or XImode value for memmset. Add TARGET_GEN_MEMSET_SCRATCH_RTX to allow the backend to use a hard scratch register to avoid stack realignment when expanding memset. PR middle-end/90773 * builtins.c (gen_memset_value_from_prev): New function. (gen_memset_broadcast): Likewise. (builtin_memset_read_str): Use gen_memset_value_from_prev and gen_memset_broadcast. (builtin_memset_gen_str): Likewise. * optabs.def: Add integer_extract and vec_const_duplicate. * target.def (gen_memset_scratch_rtx): New hook. * doc/md.texi: Document vec_const_duplicate and integer_extract. * doc/tm.texi.in: Add TARGET_GEN_MEMSET_SCRATCH_RTX. * doc/tm.texi: Regenerated. --- gcc/builtins.c | 117 +++++++++++++++++++++++++++++++++++++-------- gcc/doc/md.texi | 16 +++++++ gcc/doc/tm.texi | 5 ++ gcc/doc/tm.texi.in | 2 + gcc/optabs.def | 4 ++ gcc/target.def | 7 +++ 6 files changed, 131 insertions(+), 20 deletions(-) diff --git a/gcc/builtins.c b/gcc/builtins.c index af1fe49bb48..7683169eb96 100644 --- a/gcc/builtins.c +++ b/gcc/builtins.c @@ -6598,26 +6598,106 @@ expand_builtin_strncpy (tree exp, rtx target) return NULL_RTX; } +/* Return the RTL of a register in MODE generated from PREV in the + previous iteration. */ + +static rtx +gen_memset_value_from_prev (void *prevp, scalar_int_mode mode) +{ + rtx target = nullptr; + by_pieces_prev *prev = (by_pieces_prev *) prevp; + if (prev != nullptr && prev->data != nullptr) + { + /* Use the previous data in the same mode. */ + if (prev->mode == mode) + return prev->data; + + /* Extract the RTL in MODE from PREV. */ + enum insn_code icode + = convert_optab_handler (integer_extract_optab, mode, + prev->mode); + if (icode != CODE_FOR_nothing) + { + target = gen_reg_rtx (mode); + class expand_operand ops[2]; + create_output_operand (&ops[0], target, mode); + create_input_operand (&ops[1], prev->data, prev->mode); + expand_insn (icode, 2, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); + } + else + target = simplify_gen_subreg (mode, prev->data, prev->mode, 0); + } + return target; +} + +/* Return the RTL of a register in MODE broadcasted from DATA. */ + +static rtx +gen_memset_broadcast (enum optab_tag broadcast_optab, rtx data, + scalar_int_mode mode) +{ + /* Skip if regno_reg_rtx isn't initialized. */ + if (!regno_reg_rtx) + return nullptr; + + rtx target = nullptr; + + unsigned int nunits = GET_MODE_SIZE (mode) / GET_MODE_SIZE (QImode); + machine_mode vector_mode; + if (!mode_for_vector (QImode, nunits).exists (&vector_mode)) + gcc_unreachable (); + + enum insn_code icode = optab_handler (broadcast_optab, vector_mode); + if (icode != CODE_FOR_nothing) + { + target = targetm.gen_memset_scratch_rtx (vector_mode); + class expand_operand ops[2]; + create_output_operand (&ops[0], target, vector_mode); + create_input_operand (&ops[1], (rtx) data, QImode); + expand_insn (icode, 2, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); + if (REGNO (target) < FIRST_PSEUDO_REGISTER) + target = gen_rtx_REG (mode, REGNO (target)); + else + target = convert_to_mode (mode, target, 1); + } + + return target; +} + /* Callback routine for store_by_pieces. Read GET_MODE_BITSIZE (MODE) bytes from constant string DATA + OFFSET and return it as target constant. If PREV isn't nullptr, it has the RTL info from the previous iteration. */ rtx -builtin_memset_read_str (void *data, void *prevp, +builtin_memset_read_str (void *data, void *prev, HOST_WIDE_INT offset ATTRIBUTE_UNUSED, scalar_int_mode mode) { - by_pieces_prev *prev = (by_pieces_prev *) prevp; - if (prev != nullptr && prev->data != nullptr) + rtx target; + + /* Don't use the previous value if size is 1. */ + if (GET_MODE_SIZE (mode) != 1) { - /* Use the previous data in the same mode. */ - if (prev->mode == mode) - return prev->data; + target = gen_memset_value_from_prev (prev, mode); + if (target != nullptr) + return target; } const char *c = (const char *) data; - char *p = XALLOCAVEC (char, GET_MODE_SIZE (mode)); + char *p = XALLOCAVEC (char, GET_MODE_SIZE (QImode)); + memset (p, *c, GET_MODE_SIZE (QImode)); + rtx src = c_readstr (p, QImode); + target = gen_memset_broadcast (vec_const_duplicate_optab, src, + mode); + if (target != nullptr) + return target; + + p = XALLOCAVEC (char, GET_MODE_SIZE (mode)); memset (p, *c, GET_MODE_SIZE (mode)); @@ -6631,7 +6711,7 @@ builtin_memset_read_str (void *data, void *prevp, nullptr, it has the RTL info from the previous iteration. */ static rtx -builtin_memset_gen_str (void *data, void *prevp, +builtin_memset_gen_str (void *data, void *prev, HOST_WIDE_INT offset ATTRIBUTE_UNUSED, scalar_int_mode mode) { @@ -6639,22 +6719,19 @@ builtin_memset_gen_str (void *data, void *prevp, size_t size; char *p; - by_pieces_prev *prev = (by_pieces_prev *) prevp; - if (prev != nullptr && prev->data != nullptr) - { - /* Use the previous data in the same mode. */ - if (prev->mode == mode) - return prev->data; - - target = simplify_gen_subreg (mode, prev->data, prev->mode, 0); - if (target != nullptr) - return target; - } - size = GET_MODE_SIZE (mode); if (size == 1) return (rtx) data; + target = gen_memset_value_from_prev (prev, mode); + if (target != nullptr) + return target; + + target = gen_memset_broadcast (vec_duplicate_optab, (rtx) data, + mode); + if (target != nullptr) + return target; + p = XALLOCAVEC (char, size); memset (p, 1, size); coeff = c_readstr (p, mode); diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi index 00caf3844cc..a798fb1a97e 100644 --- a/gcc/doc/md.texi +++ b/gcc/doc/md.texi @@ -5079,6 +5079,16 @@ vectors go through the @code{mov@var{m}} pattern instead. This pattern is not allowed to @code{FAIL}. +@cindex @code{vec_const_duplicate@var{m}} instruction pattern +@item @samp{vec_const_duplicate@var{m}} +Initialize vector output operand 0 in mode @var{m} so that each element +has the value given by constant input operand 1. + +This pattern only handles duplicates of constant inputs. Non-constant +vectors go through the @code{vec_duplicate@var{m}} pattern instead. + +This pattern is not allowed to @code{FAIL}. + @cindex @code{vec_series@var{m}} instruction pattern @item @samp{vec_series@var{m}} Initialize vector output operand 0 so that element @var{i} is equal to @@ -7904,6 +7914,12 @@ inclusive and operand 1 exclusive. If this pattern is not defined, a call to the library function @code{__clear_cache} is used. +@cindex @code{integer_extract@var{m}@var{n}} instruction pattern +@item @samp{integer_extract@var{m}@var{n}} +Extract lower bit value from the integer value in @code{TImode}, +@code{OImode} or @code{XImode}. Operand 1 is the integer in mode +@var{n} and operand 0 stores value to be extracted in mode @var{m}. + @end table @end ifset diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi index e3a080e4a7c..8ccc262b1fc 100644 --- a/gcc/doc/tm.texi +++ b/gcc/doc/tm.texi @@ -11894,6 +11894,11 @@ This function prepares to emit a conditional comparison within a sequence @var{bit_code} is @code{AND} or @code{IOR}, which is the op on the compares. @end deftypefn +@deftypefn {Target Hook} rtx TARGET_GEN_MEMSET_SCRATCH_RTX (machine_mode @var{mode}) +This hook should return an rtx for scratch register in @var{mode} to +be used by memset broadcast. The default is @code{gen_reg_rtx}. +@end deftypefn + @deftypefn {Target Hook} unsigned TARGET_LOOP_UNROLL_ADJUST (unsigned @var{nunroll}, class loop *@var{loop}) This target hook returns a new value for the number of times @var{loop} should be unrolled. The parameter @var{nunroll} is the number of times diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in index d9fbbe20e6f..99bf01fe25d 100644 --- a/gcc/doc/tm.texi.in +++ b/gcc/doc/tm.texi.in @@ -7960,6 +7960,8 @@ lists. @hook TARGET_GEN_CCMP_NEXT +@hook TARGET_GEN_MEMSET_SCRATCH_RTX + @hook TARGET_LOOP_UNROLL_ADJUST @defmac POWI_MAX_MULTS diff --git a/gcc/optabs.def b/gcc/optabs.def index b192a9d070b..fd8ab8b4a26 100644 --- a/gcc/optabs.def +++ b/gcc/optabs.def @@ -100,6 +100,8 @@ OPTAB_CD(vec_init_optab, "vec_init$a$b") OPTAB_CD (while_ult_optab, "while_ult$a$b") +OPTAB_CD (integer_extract_optab, "integer_extract$a$b") + OPTAB_NL(add_optab, "add$P$a3", PLUS, "add", '3', gen_int_fp_fixed_libfunc) OPTAB_NX(add_optab, "add$F$a3") OPTAB_NX(add_optab, "add$Q$a3") @@ -453,3 +455,5 @@ OPTAB_DC (vec_series_optab, "vec_series$a", VEC_SERIES) OPTAB_D (vec_shl_insert_optab, "vec_shl_insert_$a") OPTAB_D (len_load_optab, "len_load_$a") OPTAB_D (len_store_optab, "len_store_$a") + +OPTAB_DC (vec_const_duplicate_optab, "vec_const_duplicate$a", VEC_DUPLICATE) diff --git a/gcc/target.def b/gcc/target.def index 1dffedc81e4..b89e7c24471 100644 --- a/gcc/target.def +++ b/gcc/target.def @@ -2724,6 +2724,13 @@ DEFHOOK rtx, (rtx_insn **prep_seq, rtx_insn **gen_seq, rtx prev, int cmp_code, tree op0, tree op1, int bit_code), NULL) +DEFHOOK +(gen_memset_scratch_rtx, + "This hook should return an rtx for scratch register in @var{mode} to\n\ +be used by memset broadcast. The default is @code{gen_reg_rtx}.", + rtx, (machine_mode mode), + gen_reg_rtx) + /* Return a new value for loop unroll size. */ DEFHOOK (loop_unroll_adjust, -- 2.31.1