On 10/26/18, Uros Bizjak <ubiz...@gmail.com> wrote: > On Fri, Oct 26, 2018 at 9:37 AM Uros Bizjak <ubiz...@gmail.com> wrote: >> >> On Fri, Oct 26, 2018 at 9:35 AM Uros Bizjak <ubiz...@gmail.com> wrote: >> > >> > On Fri, Oct 26, 2018 at 9:19 AM H.J. Lu <hjl.to...@gmail.com> wrote: >> > > >> > > On 10/25/18, Uros Bizjak <ubiz...@gmail.com> wrote: >> > > > On Fri, Oct 26, 2018 at 8:07 AM H.J. Lu <hjl.to...@gmail.com> >> > > > wrote: >> > > >> >> > > >> Many x86 pmovzx/pmovsx instructions with memory operands are >> > > >> modeled in >> > > >> a wrong way. For example: >> > > >> >> > > >> (define_insn "sse4_1_<code>v8qiv8hi2<mask_name>" >> > > >> [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") >> > > >> (any_extend:V8HI >> > > >> (vec_select:V8QI >> > > >> (match_operand:V16QI 1 "nonimmediate_operand" >> > > >> "Yrm,*xm,vm") >> > > >> (parallel [(const_int 0) (const_int 1) >> > > >> (const_int 2) (const_int 3) >> > > >> (const_int 4) (const_int 5) >> > > >> (const_int 6) (const_int 7)]))))] >> > > >> >> > > >> should be defind for memory operands as: >> > > >> >> > > >> (define_insn "sse4_1_<code>v8qiv8hi2<mask_name>" >> > > >> [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") >> > > >> (any_extend:V8HI >> > > >> (match_operand:V8QI "memory_operand" "m,m,m")))] >> > > >> >> > > >> This set of patches updates them to >> > > >> >> > > >> (define_insn "sse4_1_<code>v8qiv8hi2<mask_name>" >> > > >> [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") >> > > >> (any_extend:V8HI >> > > >> (vec_select:V8QI >> > > >> (match_operand:V16QI 1 "nonimmediate_operand" "Yr,*x,v") >> > > >> (parallel [(const_int 0) (const_int 1) >> > > >> (const_int 2) (const_int 3) >> > > >> (const_int 4) (const_int 5) >> > > >> (const_int 6) (const_int 7)]))))] >> > > >> >> > > >> (define_insn "*sse4_1_<code>v8qiv8hi2<mask_name>_1" >> > > >> [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") >> > > >> (any_extend:V8HI >> > > >> (match_operand:V8QI "subreg_memory_operand" "m,m,m")))] >> > > >> >> > > >> with a splitter: >> > > >> >> > > >> (define_insn_and_split "*sse4_1_<code>v8qiv8hi2<mask_name>_2" >> > > >> [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") >> > > > >> > > > No constraints needed for pre-reload splitter. >> > > > >> > > >> (any_extend:V8HI >> > > >> (vec_select:V8QI >> > > >> (subreg:V16QI >> > > >> (vec_concat:V2DI >> > > >> (match_operand:DI 1 "memory_operand" "m,*m,m") >> > > >> (const_int 0)) 0) >> > > >> (parallel [(const_int 0) (const_int 1) >> > > >> (const_int 2) (const_int 3) >> > > >> (const_int 4) (const_int 5) >> > > >> (const_int 6) (const_int 7)]))))] >> > > >> "TARGET_SSE4_1 && <mask_avx512bw_condition> && >> > > >> <mask_avx512vl_condition>" >> > > >> "#" >> > > >> "&& can_create_pseudo_p ()" >> > > >> [(set (match_dup 0) (match_dup 1))] >> > > > >> > > > [(set (match_dup 0) >> > > > (any_extend:V8HI (match_dup 1)))] >> > > > >> > > >> { >> > > >> operands[1] = gen_rtx_<CODE> (V8HImode, >> > > >> gen_rtx_SUBREG (V8QImode, >> > > >> operands[1], 0)); >> > > >> }) >> > > > >> > > > Don't create subregs of memory. Use adjust_address_nv. >> > > >> > > Here is the updated patch. >> > >> > > with a splitter: >> > > >> > > (define_insn_and_split "*sse4_1_<code>v8qiv8hi2<mask_name>_2" >> > > [(set (match_operand:V8HI 0 "register_operand") >> > > (any_extend:V8HI >> > > (vec_select:V8QI >> > > (subreg:V16QI >> > > (vec_concat:V2DI >> > > (match_operand:DI 1 "memory_operand") >> > > (const_int 0)) 0) >> > > (parallel [(const_int 0) (const_int 1) >> > > (const_int 2) (const_int 3) >> > > (const_int 4) (const_int 5) >> > > (const_int 6) (const_int 7)]))))] >> > > "TARGET_SSE4_1 && <mask_avx512bw_condition> && >> > > <mask_avx512vl_condition>" >> > > "#" >> > > "&& can_create_pseudo_p ()" >> > >> > "can_create_pseudo_p ()" should go to the insn constraint and "&& 1" >> > should be used for split constraint. Both, insn and splitter are valid >> > only before reload. >> > >> > > [(set (match_dup 0) >> > > (any_extend:V8HI (match_dup 1)))] >> > > { >> > > operands[1] = adjust_address_nv (operands[1], V8QImode, 0); >> > > }) >> > >> > Please use double quotes for one-line preparation statement. >> > >> > > (any_extend:V4SI >> > > (match_operand:V4HI 1 "memory_operand" "m,*m,m")))] >> > >> > Please remove star in front of memory constraint. >> > >> > OK with the above changes. >> >> Oh, and you should remove "q" and "k" operand modifiers in all old >> patterns. > > Well, the new ones, obviously.
This is the patch I am going to check in after the apply_subst_iterator fix is approved. Thanks. -- H.J.
From 585a4e65822b07c85b83096905d8c6130bf0381a Mon Sep 17 00:00:00 2001 From: "H.J. Lu" <hjl.to...@gmail.com> Date: Sat, 15 Sep 2018 20:54:42 -0700 Subject: [PATCH] x86: Add pmovzx/pmovsx patterns with memory operands Many x86 pmovzx/pmovsx instructions with memory operands are modeled in a wrong way. For example: (define_insn "sse4_1_<code>v8qiv8hi2<mask_name>" [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") (any_extend:V8HI (vec_select:V8QI (match_operand:V16QI 1 "nonimmediate_operand" "Yrm,*xm,vm") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3) (const_int 4) (const_int 5) (const_int 6) (const_int 7)]))))] should be defind for memory operands as: (define_insn "sse4_1_<code>v8qiv8hi2<mask_name>" [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") (any_extend:V8HI (match_operand:V8QI "memory_operand" "m,m,m")))] This patch updates them to (define_insn "sse4_1_<code>v8qiv8hi2<mask_name>" [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") (any_extend:V8HI (vec_select:V8QI (match_operand:V16QI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3) (const_int 4) (const_int 5) (const_int 6) (const_int 7)]))))] (define_insn "*sse4_1_<code>v8qiv8hi2<mask_name>_1" [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") (any_extend:V8HI (match_operand:V8QI "subreg_memory_operand" "m,m,m")))] with a splitter: (define_insn_and_split "*sse4_1_<code>v8qiv8hi2<mask_name>_2" [(set (match_operand:V8HI 0 "register_operand") (any_extend:V8HI (vec_select:V8QI (subreg:V16QI (vec_concat:V2DI (match_operand:DI 1 "memory_operand") (const_int 0)) 0) (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3) (const_int 4) (const_int 5) (const_int 6) (const_int 7)]))))] "TARGET_SSE4_1 && <mask_avx512bw_condition> && <mask_avx512vl_condition> "&& can_create_pseudo_p ()" "#" "&& 1" [(set (match_dup 0) (any_extend:V8HI (match_dup 1)))] { operands[1] = adjust_address_nv (operands[1], V8QImode, 0); }) This patch requires updating apply_subst_iterator to handle define_insn_and_split. gcc/ PR target/87317 * config/i386/sse.md (sse4_1_<code>v8qiv8hi2<mask_name>): Replace nonimmediate_operand with register_operand. (avx2_<code>v8qiv8si2<mask_name>): Likewise. (sse4_1_<code>v4qiv4si2<mask_name>): Likewise. (sse4_1_<code>v4hiv4si2<mask_name>): Likewise. (sse4_1_<code>v2qiv2di2<mask_name>): Likewise. (avx512f_<code>v8qiv8di2<mask_name>): Likewise. (avx2_<code>v4qiv4di2<mask_name>): Likewise. (avx2_<code>v4hiv4di2<mask_name>): Likewise. (sse4_1_<code>v2hiv2di2<mask_name>): Likewise. (sse4_1_<code>v2siv2di2<mask_name>): Likewise. (*sse4_1_<code>v8qiv8hi2<mask_name>_1): New pattern. (*sse4_1_<code>v8qiv8hi2<mask_name>_2): Likewise. (*avx2_<code>v8qiv8si2<mask_name>_1): Likewise. (*avx2_<code>v8qiv8si2<mask_name>_2): Likewise. (*sse4_1_<code>v4qiv4si2<mask_name>_1): Likewise. (*sse4_1_<code>v4qiv4si2<mask_name>_2): Likewise. (*sse4_1_<code>v4hiv4si2<mask_name>_1): Likewise. (*sse4_1_<code>v4hiv4si2<mask_name>_2): Likewise. (*avx512f_<code>v8qiv8di2<mask_name>_1): Likewise. (*avx512f_<code>v8qiv8di2<mask_name>_2): Likewise. (*avx2_<code>v4qiv4di2<mask_name>_1): Likewise. (*avx2_<code>v4qiv4di2<mask_name>_2): Likewise. (*avx2_<code>v4hiv4di2<mask_name>_1): Likewise. (*avx2_<code>v4hiv4di2<mask_name>_2): Likewise. (*sse4_1_<code>v2hiv2di2<mask_name>_1): Likewise. (*sse4_1_<code>v2hiv2di2<mask_name>_2): Likewise. (*sse4_1_<code>v2siv2di2<mask_name>_1): Likewise. (*sse4_1_<code>v2siv2di2<mask_name>_2): Likewise. gcc/testsuite/ PR target/87317 * gcc.target/i386/pr87317-1.c: New file. * gcc.target/i386/pr87317-2.c: Likewise. * gcc.target/i386/pr87317-3.c: Likewise. * gcc.target/i386/pr87317-4.c: Likewise. * gcc.target/i386/pr87317-5.c: Likewise. * gcc.target/i386/pr87317-6.c: Likewise. * gcc.target/i386/pr87317-7.c: Likewise. * gcc.target/i386/pr87317-8.c: Likewise. * gcc.target/i386/pr87317-9.c: Likewise. * gcc.target/i386/pr87317-10.c: Likewise. * gcc.target/i386/pr87317-11.c: Likewise. * gcc.target/i386/pr87317-12.c: Likewise. * gcc.target/i386/pr87317-13.c: Likewise. --- gcc/config/i386/sse.md | 349 +++++++++++++++++++-- gcc/testsuite/gcc.target/i386/pr87317-1.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-10.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-11.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-12.c | 22 ++ gcc/testsuite/gcc.target/i386/pr87317-13.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-2.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-3.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-4.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-5.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-6.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-7.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-8.c | 14 + gcc/testsuite/gcc.target/i386/pr87317-9.c | 14 + 14 files changed, 519 insertions(+), 20 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-1.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-10.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-11.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-12.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-13.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-2.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-3.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-4.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-5.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-6.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-7.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-8.c create mode 100644 gcc/testsuite/gcc.target/i386/pr87317-9.c diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index ee73e1fdf80..4ba2c96a35b 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -15878,19 +15878,55 @@ [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") (any_extend:V8HI (vec_select:V8QI - (match_operand:V16QI 1 "nonimmediate_operand" "Yrm,*xm,vm") + (match_operand:V16QI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3) (const_int 4) (const_int 5) (const_int 6) (const_int 7)]))))] "TARGET_SSE4_1 && <mask_avx512bw_condition> && <mask_avx512vl_condition>" - "%vpmov<extsuffix>bw\t{%1, %0<mask_operand2>|%0<mask_operand2>, %q1}" + "%vpmov<extsuffix>bw\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "isa" "noavx,noavx,avx") (set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "orig,orig,maybe_evex") (set_attr "mode" "TI")]) +(define_insn "*sse4_1_<code>v8qiv8hi2<mask_name>_1" + [(set (match_operand:V8HI 0 "register_operand" "=Yr,*x,v") + (any_extend:V8HI + (match_operand:V8QI 1 "memory_operand" "m,m,m")))] + "TARGET_SSE4_1 && <mask_avx512bw_condition> && <mask_avx512vl_condition>" + "%vpmov<extsuffix>bw\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "isa" "noavx,noavx,avx") + (set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "orig,orig,maybe_evex") + (set_attr "mode" "TI")]) + +(define_insn_and_split "*sse4_1_<code>v8qiv8hi2<mask_name>_2" + [(set (match_operand:V8HI 0 "register_operand") + (any_extend:V8HI + (vec_select:V8QI + (subreg:V16QI + (vec_concat:V2DI + (match_operand:DI 1 "memory_operand") + (const_int 0)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7)]))))] + "TARGET_SSE4_1 + && <mask_avx512bw_condition> + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V8HI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V8QImode, 0); +}) + (define_insn "<mask_codefor>avx512f_<code>v16qiv16si2<mask_name>" [(set (match_operand:V16SI 0 "register_operand" "=v") (any_extend:V16SI @@ -15905,33 +15941,105 @@ [(set (match_operand:V8SI 0 "register_operand" "=v") (any_extend:V8SI (vec_select:V8QI - (match_operand:V16QI 1 "nonimmediate_operand" "vm") + (match_operand:V16QI 1 "register_operand" "v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3) (const_int 4) (const_int 5) (const_int 6) (const_int 7)]))))] "TARGET_AVX2 && <mask_avx512vl_condition>" - "vpmov<extsuffix>bd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %q1}" + "vpmov<extsuffix>bd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "maybe_evex") (set_attr "mode" "OI")]) +(define_insn "*avx2_<code>v8qiv8si2<mask_name>_1" + [(set (match_operand:V8SI 0 "register_operand" "=v") + (any_extend:V8SI + (match_operand:V8QI 1 "memory_operand" "m")))] + "TARGET_AVX2 && <mask_avx512vl_condition>" + "%vpmov<extsuffix>bd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "maybe_evex") + (set_attr "mode" "OI")]) + +(define_insn_and_split "*avx2_<code>v8qiv8si2<mask_name>_2" + [(set (match_operand:V8SI 0 "register_operand") + (any_extend:V8SI + (vec_select:V8QI + (subreg:V16QI + (vec_concat:V2DI + (match_operand:DI 1 "memory_operand") + (const_int 0)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7)]))))] + "TARGET_AVX2 + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V8SI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V8QImode, 0); +}) + (define_insn "sse4_1_<code>v4qiv4si2<mask_name>" [(set (match_operand:V4SI 0 "register_operand" "=Yr,*x,v") (any_extend:V4SI (vec_select:V4QI - (match_operand:V16QI 1 "nonimmediate_operand" "Yrm,*xm,vm") + (match_operand:V16QI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3)]))))] "TARGET_SSE4_1 && <mask_avx512vl_condition>" - "%vpmov<extsuffix>bd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %k1}" + "%vpmov<extsuffix>bd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "isa" "noavx,noavx,avx") (set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "orig,orig,maybe_evex") (set_attr "mode" "TI")]) +(define_insn "*sse4_1_<code>v4qiv4si2<mask_name>_1" + [(set (match_operand:V4SI 0 "register_operand" "=Yr,*x,v") + (any_extend:V4SI + (match_operand:V4QI 1 "memory_operand" "m,m,m")))] + "TARGET_SSE4_1 && <mask_avx512vl_condition>" + "%vpmov<extsuffix>bd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "isa" "noavx,noavx,avx") + (set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "orig,orig,maybe_evex") + (set_attr "mode" "TI")]) + +(define_insn_and_split "*sse4_1_<code>v4qiv4si2<mask_name>_2" + [(set (match_operand:V4SI 0 "register_operand") + (any_extend:V4SI + (vec_select:V4QI + (subreg:V16QI + (vec_merge:V4SI + (vec_duplicate:V4SI + (match_operand:SI 1 "memory_operand")) + (const_vector:V4SI + [(const_int 0) (const_int 0) + (const_int 0) (const_int 0)]) + (const_int 1)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3)]))))] + "TARGET_SSE4_1 + && <mask_avx512bw_condition> + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V4SI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V4QImode, 0); +}) + (define_insn "avx512f_<code>v16hiv16si2<mask_name>" [(set (match_operand:V16SI 0 "register_operand" "=v") (any_extend:V16SI @@ -15957,54 +16065,155 @@ [(set (match_operand:V4SI 0 "register_operand" "=Yr,*x,v") (any_extend:V4SI (vec_select:V4HI - (match_operand:V8HI 1 "nonimmediate_operand" "Yrm,*xm,vm") + (match_operand:V8HI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3)]))))] "TARGET_SSE4_1 && <mask_avx512vl_condition>" - "%vpmov<extsuffix>wd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %q1}" + "%vpmov<extsuffix>wd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "isa" "noavx,noavx,avx") (set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "orig,orig,maybe_evex") (set_attr "mode" "TI")]) +(define_insn "*sse4_1_<code>v4hiv4si2<mask_name>_1" + [(set (match_operand:V4SI 0 "register_operand" "=Yr,*x,v") + (any_extend:V4SI + (match_operand:V4HI 1 "memory_operand" "m,m,m")))] + "TARGET_SSE4_1 && <mask_avx512vl_condition>" + "%vpmov<extsuffix>wd\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "isa" "noavx,noavx,avx") + (set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "orig,orig,maybe_evex") + (set_attr "mode" "TI")]) + +(define_insn_and_split "*sse4_1_<code>v4hiv4si2<mask_name>_2" + [(set (match_operand:V4SI 0 "register_operand") + (any_extend:V4SI + (vec_select:V4HI + (subreg:V8HI + (vec_concat:V2DI + (match_operand:DI 1 "memory_operand") + (const_int 0)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3)]))))] + "TARGET_SSE4_1 + && <mask_avx512bw_condition> + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V4SI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V4HImode, 0); +}) + (define_insn "avx512f_<code>v8qiv8di2<mask_name>" [(set (match_operand:V8DI 0 "register_operand" "=v") (any_extend:V8DI (vec_select:V8QI - (match_operand:V16QI 1 "nonimmediate_operand" "vm") + (match_operand:V16QI 1 "register_operand" "v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3) (const_int 4) (const_int 5) (const_int 6) (const_int 7)]))))] "TARGET_AVX512F" - "vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %k1}" + "vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "type" "ssemov") + (set_attr "prefix" "evex") + (set_attr "mode" "XI")]) + +(define_insn "*avx512f_<code>v8qiv8di2<mask_name>_1" + [(set (match_operand:V8DI 0 "register_operand" "=v") + (any_extend:V8DI + (match_operand:V8QI 1 "memory_operand" "m")))] + "TARGET_AVX512F" + "vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "type" "ssemov") (set_attr "prefix" "evex") (set_attr "mode" "XI")]) +(define_insn_and_split "*avx512f_<code>v8qiv8di2<mask_name>_2" + [(set (match_operand:V8DI 0 "register_operand") + (any_extend:V8DI + (vec_select:V8QI + (subreg:V16QI + (vec_concat:V2DI + (match_operand:DI 1 "memory_operand") + (const_int 0)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7)]))))] + "TARGET_AVX512F && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V8DI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V8QImode, 0); +}) + (define_insn "avx2_<code>v4qiv4di2<mask_name>" [(set (match_operand:V4DI 0 "register_operand" "=v") (any_extend:V4DI (vec_select:V4QI - (match_operand:V16QI 1 "nonimmediate_operand" "vm") + (match_operand:V16QI 1 "register_operand" "v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3)]))))] "TARGET_AVX2 && <mask_avx512vl_condition>" - "vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %k1}" + "vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "maybe_evex") (set_attr "mode" "OI")]) +(define_insn "*avx2_<code>v4qiv4di2<mask_name>_1" + [(set (match_operand:V4DI 0 "register_operand" "=v") + (any_extend:V4DI + (match_operand:V4QI 1 "memory_operand" "m")))] + "TARGET_AVX2 && <mask_avx512vl_condition>" + "vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "maybe_evex") + (set_attr "mode" "OI")]) + +(define_insn_and_split "*avx2_<code>v4qiv4di2<mask_name>_2" + [(set (match_operand:V4DI 0 "register_operand") + (any_extend:V4DI + (vec_select:V4QI + (subreg:V16QI + (vec_merge:V4SI + (vec_duplicate:V4SI + (match_operand:SI 1 "memory_operand")) + (const_vector:V4SI + [(const_int 0) (const_int 0) + (const_int 0) (const_int 0)]) + (const_int 1)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3)]))))] + "TARGET_AVX2 + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V4DI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V4QImode, 0); +}) + (define_insn "sse4_1_<code>v2qiv2di2<mask_name>" [(set (match_operand:V2DI 0 "register_operand" "=Yr,*x,v") (any_extend:V2DI (vec_select:V2QI - (match_operand:V16QI 1 "nonimmediate_operand" "Yrm,*xm,vm") + (match_operand:V16QI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1)]))))] "TARGET_SSE4_1 && <mask_avx512vl_condition>" - "%vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %w1}" + "%vpmov<extsuffix>bq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "isa" "noavx,noavx,avx") (set_attr "type" "ssemov") (set_attr "prefix_extra" "1") @@ -16025,30 +16234,98 @@ [(set (match_operand:V4DI 0 "register_operand" "=v") (any_extend:V4DI (vec_select:V4HI - (match_operand:V8HI 1 "nonimmediate_operand" "vm") + (match_operand:V8HI 1 "register_operand" "v") (parallel [(const_int 0) (const_int 1) (const_int 2) (const_int 3)]))))] "TARGET_AVX2 && <mask_avx512vl_condition>" - "vpmov<extsuffix>wq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %q1}" + "vpmov<extsuffix>wq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "maybe_evex") (set_attr "mode" "OI")]) +(define_insn "*avx2_<code>v4hiv4di2<mask_name>_1" + [(set (match_operand:V4DI 0 "register_operand" "=v") + (any_extend:V4DI + (match_operand:V4HI 1 "memory_operand" "m")))] + "TARGET_AVX2 && <mask_avx512vl_condition>" + "vpmov<extsuffix>wq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "maybe_evex") + (set_attr "mode" "OI")]) + +(define_insn_and_split "*avx2_<code>v4hiv4di2<mask_name>_2" + [(set (match_operand:V4DI 0 "register_operand") + (any_extend:V4DI + (vec_select:V4HI + (subreg:V8HI + (vec_concat:V2DI + (match_operand:DI 1 "memory_operand") + (const_int 0)) 0) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3)]))))] + "TARGET_AVX2 + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V4DI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V4HImode, 0); +}) + (define_insn "sse4_1_<code>v2hiv2di2<mask_name>" [(set (match_operand:V2DI 0 "register_operand" "=Yr,*x,v") (any_extend:V2DI (vec_select:V2HI - (match_operand:V8HI 1 "nonimmediate_operand" "Yrm,*xm,vm") + (match_operand:V8HI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1)]))))] "TARGET_SSE4_1 && <mask_avx512vl_condition>" - "%vpmov<extsuffix>wq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %k1}" + "%vpmov<extsuffix>wq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "isa" "noavx,noavx,avx") + (set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "orig,orig,maybe_evex") + (set_attr "mode" "TI")]) + +(define_insn "*sse4_1_<code>v2hiv2di2<mask_name>_1" + [(set (match_operand:V2DI 0 "register_operand" "=Yr,*x,v") + (any_extend:V2DI + (match_operand:V2HI 1 "memory_operand" "m,m,m")))] + "TARGET_SSE4_1 && <mask_avx512vl_condition>" + "%vpmov<extsuffix>wq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "isa" "noavx,noavx,avx") (set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "orig,orig,maybe_evex") (set_attr "mode" "TI")]) +(define_insn_and_split "*sse4_1_<code>v2hiv2di2<mask_name>_2" + [(set (match_operand:V2DI 0 "register_operand") + (any_extend:V2DI + (vec_select:V2HI + (subreg:V8HI + (vec_merge:V4SI + (vec_duplicate:V4SI + (match_operand:SI 1 "memory_operand")) + (const_vector:V4SI + [(const_int 0) (const_int 0) + (const_int 0) (const_int 0)]) + (const_int 1)) 0) + (parallel [(const_int 0) (const_int 1)]))))] + "TARGET_SSE4_1 + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V2DI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V2HImode, 0); +}) + (define_insn "avx512f_<code>v8siv8di2<mask_name>" [(set (match_operand:V8DI 0 "register_operand" "=v") (any_extend:V8DI @@ -16074,16 +16351,48 @@ [(set (match_operand:V2DI 0 "register_operand" "=Yr,*x,v") (any_extend:V2DI (vec_select:V2SI - (match_operand:V4SI 1 "nonimmediate_operand" "Yrm,*xm,vm") + (match_operand:V4SI 1 "register_operand" "Yr,*x,v") (parallel [(const_int 0) (const_int 1)]))))] "TARGET_SSE4_1 && <mask_avx512vl_condition>" - "%vpmov<extsuffix>dq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %q1}" + "%vpmov<extsuffix>dq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" [(set_attr "isa" "noavx,noavx,avx") (set_attr "type" "ssemov") (set_attr "prefix_extra" "1") (set_attr "prefix" "orig,orig,maybe_evex") (set_attr "mode" "TI")]) +(define_insn "*sse4_1_<code>v2siv2di2<mask_name>_1" + [(set (match_operand:V2DI 0 "register_operand" "=Yr,*x,v") + (any_extend:V2DI + (match_operand:V2SI 1 "memory_operand" "m,m,m")))] + "TARGET_SSE4_1 && <mask_avx512vl_condition>" + "%vpmov<extsuffix>dq\t{%1, %0<mask_operand2>|%0<mask_operand2>, %1}" + [(set_attr "isa" "noavx,noavx,avx") + (set_attr "type" "ssemov") + (set_attr "prefix_extra" "1") + (set_attr "prefix" "orig,orig,maybe_evex") + (set_attr "mode" "TI")]) + +(define_insn_and_split "*sse4_1_<code>v2siv2di2<mask_name>_2" + [(set (match_operand:V2DI 0 "register_operand") + (any_extend:V2DI + (vec_select:V2SI + (subreg:V4SI + (vec_concat:V2DI + (match_operand:DI 1 "memory_operand") + (const_int 0)) 0) + (parallel [(const_int 0) (const_int 1)]))))] + "TARGET_SSE4_1 + && <mask_avx512vl_condition> + && can_create_pseudo_p ()" + "#" + "&& 1" + [(set (match_dup 0) + (any_extend:V2DI (match_dup 1)))] +{ + operands[1] = adjust_address_nv (operands[1], V2SImode, 0); +}) + ;; ptestps/ptestpd are very similar to comiss and ucomiss when ;; setting FLAGS_REG. But it is not a really compare instruction. (define_insn "avx_vtest<ssemodesuffix><avxsizesuffix>" diff --git a/gcc/testsuite/gcc.target/i386/pr87317-1.c b/gcc/testsuite/gcc.target/i386/pr87317-1.c new file mode 100644 index 00000000000..ec6b11d371d --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-1.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxbw" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi64_si128(*(long long int*)ptr); + data = _mm_cvtepu8_epi16(data); + _mm_storeu_si128((__m128i*)dst, data); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-10.c b/gcc/testsuite/gcc.target/i386/pr87317-10.c new file mode 100644 index 00000000000..ea9a7a2f101 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-10.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxbd" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i y = _mm_cvtsi64_si128(*(long long int*)ptr); + __m256i z = _mm256_cvtepu8_epi32 (y); + _mm256_storeu_si256((__m256i*)dst, z); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-11.c b/gcc/testsuite/gcc.target/i386/pr87317-11.c new file mode 100644 index 00000000000..13f0c23458a --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-11.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxwq" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i y = _mm_cvtsi64_si128(*(long long int*)ptr); + __m256i z = _mm256_cvtepu16_epi64 (y); + _mm256_storeu_si256((__m256i*)dst, z); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-12.c b/gcc/testsuite/gcc.target/i386/pr87317-12.c new file mode 100644 index 00000000000..1090966c4d1 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-12.c @@ -0,0 +1,22 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O3 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovsxwq" 1 } } */ + +#include <immintrin.h> + +#define MAX 4 + +long long int dst[MAX]; +short src[MAX]; + +void +foo (void) +{ + int i; + for (i = 0; i < MAX; i += 4) + { + __m128i data = _mm_cvtsi64_si128(*(long long int*)(src + i)); + __m256i x = _mm256_cvtepi16_epi64(data); + _mm256_storeu_si256((__m256i*)(dst + i), x); + } +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-13.c b/gcc/testsuite/gcc.target/i386/pr87317-13.c new file mode 100644 index 00000000000..d3c3def8680 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-13.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -mavx512f" } */ +/* { dg-final { scan-assembler-times "vpmovzxbq" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i y = _mm_cvtsi64_si128(*(long long int*)ptr); + __m512i z = _mm512_cvtepu8_epi64 (y); + _mm512_storeu_si512((__m512i*)dst, z); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-2.c b/gcc/testsuite/gcc.target/i386/pr87317-2.c new file mode 100644 index 00000000000..e7eaaf66eef --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-2.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovsxwd" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi64_si128(*(long long int*)ptr); + data = _mm_cvtepi16_epi32(data); + _mm_storeu_si128((__m128i*)dst, data); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-3.c b/gcc/testsuite/gcc.target/i386/pr87317-3.c new file mode 100644 index 00000000000..f2e041ab4af --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-3.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovsxdq" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi64_si128(*(long long int*)ptr); + data = _mm_cvtepi32_epi64(data); + _mm_storeu_si128((__m128i*)dst, data); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-4.c b/gcc/testsuite/gcc.target/i386/pr87317-4.c new file mode 100644 index 00000000000..2d4f24a89e9 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-4.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxbd" 1 } } */ +/* { dg-final { scan-assembler-not "vmovd" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi32_si128(*(int*)ptr); + data = _mm_cvtepu8_epi32(data); + _mm_storeu_si128((__m128i*)dst, data); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-5.c b/gcc/testsuite/gcc.target/i386/pr87317-5.c new file mode 100644 index 00000000000..96f82847e5d --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-5.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxwq" 1 } } */ +/* { dg-final { scan-assembler-not "vmovd" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi32_si128(*(int*)ptr); + data = _mm_cvtepu16_epi64(data); + _mm_storeu_si128((__m128i*)dst, data); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-6.c b/gcc/testsuite/gcc.target/i386/pr87317-6.c new file mode 100644 index 00000000000..4fe9b11c1be --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-6.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxbq" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i y = _mm_cvtsi32_si128(*(int*)ptr); + __m256i z = _mm256_cvtepu8_epi64 (y); + _mm256_storeu_si256((__m256i*)dst, z); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-7.c b/gcc/testsuite/gcc.target/i386/pr87317-7.c new file mode 100644 index 00000000000..2c043d9eb26 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-7.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxbd" 1 } } */ +/* { dg-final { scan-assembler-not "vmovd" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi32_si128(*(int*)ptr); + data = _mm_cvtepu8_epi32(data); + _mm_storeu_si128((__m128i*)dst, data); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-8.c b/gcc/testsuite/gcc.target/i386/pr87317-8.c new file mode 100644 index 00000000000..178455f42c1 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-8.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxwq" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi64_si128(*(long long int*)ptr); + __m256i x = _mm256_cvtepu16_epi64(data); + _mm256_storeu_si256((__m256i*)dst, x); +} diff --git a/gcc/testsuite/gcc.target/i386/pr87317-9.c b/gcc/testsuite/gcc.target/i386/pr87317-9.c new file mode 100644 index 00000000000..c5144fb667b --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr87317-9.c @@ -0,0 +1,14 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-options "-O2 -march=haswell" } */ +/* { dg-final { scan-assembler-times "vpmovzxbd" 1 } } */ +/* { dg-final { scan-assembler-not "vmovq" } } */ + +#include <immintrin.h> + +void +f (void *dst, void *ptr) +{ + __m128i data = _mm_cvtsi64_si128(*(long long int*)ptr); + __m256i x = _mm256_cvtepu8_epi32(data); + _mm256_storeu_si256((__m256i*)dst, x); +} -- 2.17.2