ACLE has added intrinsics to bridge between SVE and Neon.
The NEON_SVE Bridge adds intrinsics that allow conversions between NEON and
SVE vectors.
This patch adds support to GCC for the following 3 intrinsics:
svset_neonq, svget_neonq and svdup_neonq
gcc/ChangeLog:
* config.gcc: Adds new header to config.
* config/aarch64/aarch64-builtins.cc (enum aarch64_type_qualifiers):
Moved to header file.
(ENTRY): Likewise.
(enum aarch64_simd_type): Likewise.
(struct aarch64_simd_type_info): Make extern.
(GTY): Likewise.
* config/aarch64/aarch64-c.cc (aarch64_pragma_aarch64):
Defines pragma for arm_neon_sve_bridge.h.
* config/aarch64/aarch64-protos.h: New function.
* config/aarch64/aarch64-sve-builtins-base.h: New intrinsics.
* config/aarch64/aarch64-sve-builtins-base.cc
(class svget_neonq_impl): New intrinsic implementation.
(class svset_neonq_impl): Likewise.
(class svdup_neonq_impl): Likewise.
(NEON_SVE_BRIDGE_FUNCTION): New intrinsics.
* config/aarch64/aarch64-sve-builtins-functions.h
(NEON_SVE_BRIDGE_FUNCTION): Defines macro for NEON_SVE_BRIDGE
functions.
* config/aarch64/aarch64-sve-builtins-shapes.h: New shapes.
* config/aarch64/aarch64-sve-builtins-shapes.cc
(parse_element_type): Add NEON element types.
(parse_type): Likewise.
(struct get_neonq_def): Defines function shape for get_neonq.
(struct set_neonq_def): Defines function shape for set_neonq.
(struct dup_neonq_def): Defines function shape for dup_neonq.
* config/aarch64/aarch64-sve-builtins.cc (DEF_SVE_TYPE_SUFFIX):
(DEF_SVE_NEON_TYPE_SUFFIX): Defines
macro for NEON_SVE_BRIDGE type suffixes.
(DEF_NEON_SVE_FUNCTION): Defines
macro for NEON_SVE_BRIDGE functions.
(function_resolver::infer_neon128_vector_type): Infers type suffix
for overloaded functions.
(init_neon_sve_builtins): Initialise neon_sve_bridge_builtins for LTO.
(handle_arm_neon_sve_bridge_h): Handles #pragma arm_neon_sve_bridge.h.
* config/aarch64/aarch64-sve-builtins.def
(DEF_SVE_NEON_TYPE_SUFFIX): Macro for handling neon_sve type suffixes.
(bf16): Replace entry with neon-sve entry.
(f16): Likewise.
(f32): Likewise.
(f64): Likewise.
(s8): Likewise.
(s16): Likewise.
(s32): Likewise.
(s64): Likewise.
(u8): Likewise.
(u16): Likewise.
(u32): Likewise.
(u64): Likewise.
* config/aarch64/aarch64-sve-builtins.h
(GCC_AARCH64_SVE_BUILTINS_H): Include aarch64-builtins.h.
(ENTRY): Add aarch64_simd_type definiton.
(enum aarch64_simd_type): Add neon information to type_suffix_info.
(struct type_suffix_info): New function.
* config/aarch64/aarch64-sve.md
(@aarch64_sve_get_neonq_<mode>): New intrinsic insn for big endian.
(@aarch64_sve_set_neonq_<mode>): Likewise.
(@aarch64_sve_dup_neonq_<mode>): Likewise.
* config/aarch64/aarch64.cc (aarch64_init_builtins):
Add call to init_neon_sve_builtins.
* config/aarch64/iterators.md: Add UNSPEC_SET_NEONQ.
* config/aarch64/aarch64-builtins.h: New file.
* config/aarch64/aarch64-neon-sve-bridge-builtins.def: New file.
* config/aarch64/arm_neon_sve_bridge.h: New file.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/sve/acle/asm/test_sve_acle.h: Add include
arm_neon_sve_bridge header file
* gcc.dg/torture/neon-sve-bridge.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_bf16.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_f16.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_f32.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_f64.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_s16.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_s32.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_s64.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_s8.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_u16.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_u32.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_u64.c: New test.
* gcc.target/aarch64/sve/acle/asm/dup_neonq_u8.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_bf16.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_f16.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_f32.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_f64.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_s16.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_s32.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_s64.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_s8.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_u16.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_u32.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_u64.c: New test.
* gcc.target/aarch64/sve/acle/asm/get_neonq_u8.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_bf16.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_f16.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_f32.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_f64.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_s16.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_s32.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_s64.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_s8.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_u16.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_u32.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_u64.c: New test.
* gcc.target/aarch64/sve/acle/asm/set_neonq_u8.c: New test.
* gcc.target/aarch64/sve/acle/general-c/dup_neonq_1.c: New test.
* gcc.target/aarch64/sve/acle/general-c/get_neonq_1.c: New test.
* gcc.target/aarch64/sve/acle/general-c/set_neonq_1.c: New test.diff --git a/gcc/config.gcc b/gcc/config.gcc
index
415e0e1ebc5739f1c1a13196cc8c651ac9d07eea..41b542c7a463752e00c665c6eae43e340eebd222
100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -334,7 +334,7 @@ m32c*-*-*)
;;
aarch64*-*-*)
cpu_type=aarch64
- extra_headers="arm_fp16.h arm_neon.h arm_bf16.h arm_acle.h arm_sve.h"
+ extra_headers="arm_fp16.h arm_neon.h arm_bf16.h arm_acle.h arm_sve.h
arm_neon_sve_bridge.h"
c_target_objs="aarch64-c.o"
cxx_target_objs="aarch64-c.o"
d_target_objs="aarch64-d.o"
diff --git a/gcc/config/aarch64/aarch64-builtins.h
b/gcc/config/aarch64/aarch64-builtins.h
new file mode 100644
index
0000000000000000000000000000000000000000..ec4580981587ab3acbb39e0b0721ed247e309a74
--- /dev/null
+++ b/gcc/config/aarch64/aarch64-builtins.h
@@ -0,0 +1,86 @@
+/* Builtins' description for AArch64 SIMD architecture.
+ Copyright (C) 2023 Free Software Foundation, Inc.
+ This file is part of GCC.
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+#ifndef GCC_AARCH64_BUILTINS_H
+#define GCC_AARCH64_BUILTINS_H
+
+enum aarch64_type_qualifiers
+{
+ /* T foo. */
+ qualifier_none = 0x0,
+ /* unsigned T foo. */
+ qualifier_unsigned = 0x1, /* 1 << 0 */
+ /* const T foo. */
+ qualifier_const = 0x2, /* 1 << 1 */
+ /* T *foo. */
+ qualifier_pointer = 0x4, /* 1 << 2 */
+ /* Used when expanding arguments if an operand could
+ be an immediate. */
+ qualifier_immediate = 0x8, /* 1 << 3 */
+ qualifier_maybe_immediate = 0x10, /* 1 << 4 */
+ /* void foo (...). */
+ qualifier_void = 0x20, /* 1 << 5 */
+ /* 1 << 6 is now unused */
+ /* Some builtins should use the T_*mode* encoded in a simd_builtin_datum
+ rather than using the type of the operand. */
+ qualifier_map_mode = 0x80, /* 1 << 7 */
+ /* qualifier_pointer | qualifier_map_mode */
+ qualifier_pointer_map_mode = 0x84,
+ /* qualifier_const | qualifier_pointer | qualifier_map_mode */
+ qualifier_const_pointer_map_mode = 0x86,
+ /* Polynomial types. */
+ qualifier_poly = 0x100,
+ /* Lane indices - must be in range, and flipped for bigendian. */
+ qualifier_lane_index = 0x200,
+ /* Lane indices for single lane structure loads and stores. */
+ qualifier_struct_load_store_lane_index = 0x400,
+ /* Lane indices selected in pairs. - must be in range, and flipped for
+ bigendian. */
+ qualifier_lane_pair_index = 0x800,
+ /* Lane indices selected in quadtuplets. - must be in range, and flipped for
+ bigendian. */
+ qualifier_lane_quadtup_index = 0x1000,
+};
+#define ENTRY(E, M, Q, G) E,
+enum aarch64_simd_type
+{
+#include "aarch64-simd-builtin-types.def"
+ ARM_NEON_H_TYPES_LAST
+};
+#undef ENTRY
+struct GTY(()) aarch64_simd_type_info
+{
+ enum aarch64_simd_type type;
+ /* Internal type name. */
+ const char *name;
+ /* Internal type name(mangled). The mangled names conform to the
+ AAPCS64 (see "Procedure Call Standard for the ARM 64-bit Architecture",
+ Appendix A). To qualify for emission with the mangled names defined in
+ that document, a vector type must not only be of the correct mode but also
+ be of the correct internal AdvSIMD vector type (e.g. __Int8x8_t); these
+ types are registered by aarch64_init_simd_builtin_types (). In other
+ words, vector types defined in other ways e.g. via vector_size attribute
+ will get default mangled names. */
+ const char *mangle;
+ /* Internal type. */
+ tree itype;
+ /* Element type. */
+ tree eltype;
+ /* Machine mode the internal type maps to. */
+ enum machine_mode mode;
+ /* Qualifiers. */
+ enum aarch64_type_qualifiers q;
+};
+extern aarch64_simd_type_info aarch64_simd_types[];
+#endif
\ No newline at end of file
diff --git a/gcc/config/aarch64/aarch64-builtins.cc
b/gcc/config/aarch64/aarch64-builtins.cc
index
04f59fd9a54306d6422b03e32dce79bc00aed4f8..0b039c075a5cb312339729d388c9be0072f80b91
100644
--- a/gcc/config/aarch64/aarch64-builtins.cc
+++ b/gcc/config/aarch64/aarch64-builtins.cc
@@ -47,6 +47,7 @@
#include "stringpool.h"
#include "attribs.h"
#include "gimple-fold.h"
+#include "aarch64-builtins.h"
#define v8qi_UP E_V8QImode
#define v8di_UP E_V8DImode
@@ -183,47 +184,8 @@
#define SIMD_INTR_QUAL(suffix) QUAL_##suffix
#define SIMD_INTR_LENGTH_CHAR(length) LENGTH_##length
-
#define SIMD_MAX_BUILTIN_ARGS 5
-enum aarch64_type_qualifiers
-{
- /* T foo. */
- qualifier_none = 0x0,
- /* unsigned T foo. */
- qualifier_unsigned = 0x1, /* 1 << 0 */
- /* const T foo. */
- qualifier_const = 0x2, /* 1 << 1 */
- /* T *foo. */
- qualifier_pointer = 0x4, /* 1 << 2 */
- /* Used when expanding arguments if an operand could
- be an immediate. */
- qualifier_immediate = 0x8, /* 1 << 3 */
- qualifier_maybe_immediate = 0x10, /* 1 << 4 */
- /* void foo (...). */
- qualifier_void = 0x20, /* 1 << 5 */
- /* 1 << 6 is now unused */
- /* Some builtins should use the T_*mode* encoded in a simd_builtin_datum
- rather than using the type of the operand. */
- qualifier_map_mode = 0x80, /* 1 << 7 */
- /* qualifier_pointer | qualifier_map_mode */
- qualifier_pointer_map_mode = 0x84,
- /* qualifier_const | qualifier_pointer | qualifier_map_mode */
- qualifier_const_pointer_map_mode = 0x86,
- /* Polynomial types. */
- qualifier_poly = 0x100,
- /* Lane indices - must be in range, and flipped for bigendian. */
- qualifier_lane_index = 0x200,
- /* Lane indices for single lane structure loads and stores. */
- qualifier_struct_load_store_lane_index = 0x400,
- /* Lane indices selected in pairs. - must be in range, and flipped for
- bigendian. */
- qualifier_lane_pair_index = 0x800,
- /* Lane indices selected in quadtuplets. - must be in range, and flipped for
- bigendian. */
- qualifier_lane_quadtup_index = 0x1000,
-};
-
/* Flags that describe what a function might do. */
const unsigned int FLAG_NONE = 0U;
const unsigned int FLAG_READ_FPCR = 1U << 0;
@@ -883,47 +845,9 @@ const char *aarch64_scalar_builtin_types[] = {
NULL
};
-#define ENTRY(E, M, Q, G) E,
-enum aarch64_simd_type
-{
-#include "aarch64-simd-builtin-types.def"
- ARM_NEON_H_TYPES_LAST
-};
-#undef ENTRY
-
-struct GTY(()) aarch64_simd_type_info
-{
- enum aarch64_simd_type type;
-
- /* Internal type name. */
- const char *name;
-
- /* Internal type name(mangled). The mangled names conform to the
- AAPCS64 (see "Procedure Call Standard for the ARM 64-bit Architecture",
- Appendix A). To qualify for emission with the mangled names defined in
- that document, a vector type must not only be of the correct mode but also
- be of the correct internal AdvSIMD vector type (e.g. __Int8x8_t); these
- types are registered by aarch64_init_simd_builtin_types (). In other
- words, vector types defined in other ways e.g. via vector_size attribute
- will get default mangled names. */
- const char *mangle;
-
- /* Internal type. */
- tree itype;
-
- /* Element type. */
- tree eltype;
-
- /* Machine mode the internal type maps to. */
- enum machine_mode mode;
-
- /* Qualifiers. */
- enum aarch64_type_qualifiers q;
-};
-
#define ENTRY(E, M, Q, G) \
{E, "__" #E, #G "__" #E, NULL_TREE, NULL_TREE, E_##M##mode, qualifier_##Q},
-static GTY(()) struct aarch64_simd_type_info aarch64_simd_types [] = {
+extern GTY(()) struct aarch64_simd_type_info aarch64_simd_types [] = {
#include "aarch64-simd-builtin-types.def"
};
#undef ENTRY
diff --git a/gcc/config/aarch64/aarch64-c.cc b/gcc/config/aarch64/aarch64-c.cc
index
578ec6f45b06347d90f951b37064006786baf10f..ada8b81a7bef6c2e58b07324a7bfc38eecb651da
100644
--- a/gcc/config/aarch64/aarch64-c.cc
+++ b/gcc/config/aarch64/aarch64-c.cc
@@ -294,6 +294,8 @@ aarch64_pragma_aarch64 (cpp_reader *)
handle_arm_neon_h ();
else if (strcmp (name, "arm_acle.h") == 0)
handle_arm_acle_h ();
+ else if (strcmp (name, "arm_neon_sve_bridge.h") == 0)
+ aarch64_sve::handle_arm_neon_sve_bridge_h ();
else
error ("unknown %<#pragma GCC aarch64%> option %qs", name);
}
diff --git a/gcc/config/aarch64/aarch64-neon-sve-bridge-builtins.def
b/gcc/config/aarch64/aarch64-neon-sve-bridge-builtins.def
new file mode 100644
index
0000000000000000000000000000000000000000..0c3cf233c9382b2f7420379054a53fa846d46c8c
--- /dev/null
+++ b/gcc/config/aarch64/aarch64-neon-sve-bridge-builtins.def
@@ -0,0 +1,28 @@
+/* Builtin lists for AArch64 NEON-SVE-Bridge
+ Copyright (C) 2023 Free Software Foundation, Inc.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef DEF_NEON_SVE_FUNCTION
+#define DEF_NEON_SVE_FUNCTION(A, B, C, D)
+#endif
+
+DEF_NEON_SVE_FUNCTION (svset_neonq, set_neonq, all_data, none)
+DEF_NEON_SVE_FUNCTION (svget_neonq, get_neonq, all_data, none)
+DEF_NEON_SVE_FUNCTION (svdup_neonq, dup_neonq, all_data, none)
+
+#undef DEF_NEON_SVE_FUNCTION
\ No newline at end of file
diff --git a/gcc/config/aarch64/aarch64-protos.h
b/gcc/config/aarch64/aarch64-protos.h
index
70303d6fd953e0c397b9138ede8858c2db2e53db..7d0ffb381a5c04161ec4394e4f498855bb847d5b
100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -986,7 +986,9 @@ void handle_arm_neon_h (void);
namespace aarch64_sve {
void init_builtins ();
+ void init_neon_sve_builtins ();
void handle_arm_sve_h ();
+ void handle_arm_neon_sve_bridge_h ();
tree builtin_decl (unsigned, bool);
bool builtin_type_p (const_tree);
bool builtin_type_p (const_tree, unsigned int *, unsigned int *);
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-base.h
b/gcc/config/aarch64/aarch64-sve-builtins-base.h
index
d300e3a85d00b58ad790851a81d43af709b66bce..df75e4c1ecf81f3ddfa256edbcf8637d092fcfde
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-base.h
+++ b/gcc/config/aarch64/aarch64-sve-builtins-base.h
@@ -299,6 +299,12 @@ namespace aarch64_sve
extern const function_base *const svzip2;
extern const function_base *const svzip2q;
}
+ namespace neon_sve_bridge_functions
+ {
+ extern const function_base *const svset_neonq;
+ extern const function_base *const svget_neonq;
+ extern const function_base *const svdup_neonq;
+ }
}
#endif
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-base.cc
b/gcc/config/aarch64/aarch64-sve-builtins-base.cc
index
9010ecca6da23c107f5ded9ab3cfa678e308daf9..410ad4edd4c39f92d6f3e9d8ce94408434925391
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-base.cc
+++ b/gcc/config/aarch64/aarch64-sve-builtins-base.cc
@@ -44,6 +44,7 @@
#include "aarch64-sve-builtins-shapes.h"
#include "aarch64-sve-builtins-base.h"
#include "aarch64-sve-builtins-functions.h"
+#include "aarch64-builtins.h"
#include "ssa.h"
#include "gimple-fold.h"
@@ -1064,6 +1065,136 @@ public:
}
};
+class svget_neonq_impl : public function_base
+{
+public:
+ gimple *
+ fold (gimple_folder &f) const override
+ {
+ if (BYTES_BIG_ENDIAN)
+ return NULL;
+ tree rhs_tuple = gimple_call_arg (f.call, 0);
+ tree rhs_vector = build3 (BIT_FIELD_REF, TREE_TYPE (f.lhs),
+ rhs_tuple, bitsize_int(128), bitsize_int(0));
+ return gimple_build_assign (f.lhs, rhs_vector);
+ }
+ rtx
+ expand (function_expander &e) const override
+ {
+ if (BYTES_BIG_ENDIAN)
+ {
+ machine_mode mode = e.vector_mode (0);
+ insn_code icode = code_for_aarch64_sve_get_neonq (mode);
+ unsigned int nunits = 128 / GET_MODE_UNIT_BITSIZE (mode);
+ rtx indices = aarch64_gen_stepped_int_parallel
+ (nunits, (nunits - 1) , -1);
+
+ e.add_output_operand (icode);
+ e.add_input_operand (icode, e.args[0]);
+ e.add_fixed_operand (indices);
+ return e.generate_insn (icode);
+ }
+ return simplify_gen_subreg (e.vector_mode (0), e.args[0],
+ GET_MODE (e.args[0]),
+ INTVAL (e.args[1]) * BYTES_PER_SVE_VECTOR);
+ }
+};
+
+class svset_neonq_impl : public function_base
+{
+public:
+ gimple *
+ fold (gimple_folder &f) const override
+ {
+ if (BYTES_BIG_ENDIAN)
+ return NULL;
+ tree rhs_tuple = gimple_call_arg (f.call, 0);
+ tree rhs_vector = gimple_call_arg (f.call, 1);
+ gassign *copy = gimple_build_assign (unshare_expr (f.lhs), rhs_tuple);
+ tree lhs_vector = build3 (BIT_INSERT_EXPR, TREE_TYPE (rhs_vector),
+ f.lhs, rhs_vector, bitsize_int(0));
+ gassign *update = gimple_build_assign (f.lhs, lhs_vector);
+ gsi_insert_after (f.gsi, update, GSI_SAME_STMT);
+ return copy;
+ }
+ rtx
+ expand (function_expander &e) const override
+ {
+ if (BYTES_BIG_ENDIAN)
+ {
+ return e.use_exact_insn (code_for_aarch64_sve_set_neonq (e.vector_mode
(0)));
+ }
+ unsigned int index = INTVAL (e.args[1]);
+ rtx rhs_tuple = e.args[0];
+ rtx rhs_vector = e.args[2];
+ rtx lhs_tuple = e.get_nonoverlapping_reg_target ();
+ emit_move_insn (lhs_tuple, rhs_tuple);
+ rtx lhs_vector = simplify_gen_subreg (GET_MODE (rhs_vector),
+ lhs_tuple, GET_MODE (lhs_tuple),
+ index * BYTES_PER_SVE_VECTOR);
+ emit_move_insn (lhs_vector, rhs_vector);
+ return lhs_vector;
+ }
+};
+
+class svdup_neonq_impl : public function_base
+{
+public:
+ gimple *
+ fold (gimple_folder &f) const override
+ {
+ if (BYTES_BIG_ENDIAN)
+ {
+ return NULL;
+ }
+ tree rhs_vector = gimple_call_arg (f.call, 0);
+ unsigned int nargs = gimple_call_num_args (f.call);
+ unsigned HOST_WIDE_INT NEONnelts;
+ TYPE_VECTOR_SUBPARTS (TREE_TYPE (rhs_vector)).is_constant (&NEONnelts);
+ poly_uint64 SVEnelts;
+ SVEnelts = TYPE_VECTOR_SUBPARTS (TREE_TYPE (f.lhs));
+ vec_perm_builder builder (SVEnelts, NEONnelts, 1);
+ for (unsigned int i = 0; i < NEONnelts; i++)
+ {
+ builder.quick_push (i);
+ }
+ vec_perm_indices indices (builder, 1, NEONnelts);
+ tree perm_type = build_vector_type (ssizetype, SVEnelts);
+ return gimple_build_assign (f.lhs, VEC_PERM_EXPR,
+ rhs_vector,
+ rhs_vector,
+ vec_perm_indices_to_tree (perm_type, indices));
+ }
+ rtx
+ expand (function_expander &e) const override
+ {
+ insn_code icode;
+ machine_mode mode = e.vector_mode (0);
+ if (BYTES_BIG_ENDIAN)
+ {
+ icode = code_for_aarch64_sve_dup_neonq (mode);
+ unsigned int nunits = 128 / GET_MODE_UNIT_BITSIZE (mode);
+ rtx indices = aarch64_gen_stepped_int_parallel
+ (nunits, (nunits - 1) , -1);
+
+ e.add_output_operand (icode);
+ e.add_input_operand (icode, e.args[0]);
+ e.add_fixed_operand (indices);
+ return e.generate_insn (icode);
+ }
+ if (valid_for_const_vector_p (GET_MODE_INNER (mode), e.args.last ()))
+ /* Duplicate the constant to fill a vector. The pattern optimizes
+ various cases involving constant operands, falling back to SEL
+ if necessary. */
+ icode = code_for_vcond_mask (mode, mode);
+ else
+ /* Use the pattern for selecting between a duplicated scalar
+ variable and a vector fallback. */
+ icode = code_for_aarch64_sel_dup (mode);
+ return e.use_vcond_mask_insn (icode);
+ }
+};
+
class svindex_impl : public function_base
{
public:
@@ -3028,5 +3159,8 @@ FUNCTION (svzip1q, unspec_based_function, (UNSPEC_ZIP1Q,
UNSPEC_ZIP1Q,
FUNCTION (svzip2, svzip_impl, (1))
FUNCTION (svzip2q, unspec_based_function, (UNSPEC_ZIP2Q, UNSPEC_ZIP2Q,
UNSPEC_ZIP2Q))
+NEON_SVE_BRIDGE_FUNCTION (svget_neonq, svget_neonq_impl,)
+NEON_SVE_BRIDGE_FUNCTION (svset_neonq, svset_neonq_impl,)
+NEON_SVE_BRIDGE_FUNCTION (svdup_neonq, svdup_neonq_impl,)
} /* end namespace aarch64_sve */
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-functions.h
b/gcc/config/aarch64/aarch64-sve-builtins-functions.h
index
2729877d914414eff33182e03ab1dfc94a3515fa..bfb7fea674a905a2eb99f2bac7cbcb72af681b52
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-functions.h
+++ b/gcc/config/aarch64/aarch64-sve-builtins-functions.h
@@ -622,4 +622,8 @@ public:
namespace { static CONSTEXPR const CLASS NAME##_obj ARGS; } \
namespace functions { const function_base *const NAME = &NAME##_obj; }
+#define NEON_SVE_BRIDGE_FUNCTION(NAME, CLASS, ARGS) \
+ namespace { static CONSTEXPR const CLASS NAME##_obj ARGS; } \
+ namespace neon_sve_bridge_functions { const function_base *const NAME =
&NAME##_obj; }
+
#endif
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
b/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
index
7483c1d04b8e463e607e8e65aa94233460f77648..5aff20d1d21afddb934be4d5a103049b0b6c40ea
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
+++ b/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
@@ -105,9 +105,11 @@ namespace aarch64_sve
extern const function_shape *const count_vector;
extern const function_shape *const create;
extern const function_shape *const dupq;
+ extern const function_shape *const dup_neonq;
extern const function_shape *const ext;
extern const function_shape *const fold_left;
extern const function_shape *const get;
+ extern const function_shape *const get_neonq;
extern const function_shape *const inc_dec;
extern const function_shape *const inc_dec_pat;
extern const function_shape *const inc_dec_pred;
@@ -135,6 +137,7 @@ namespace aarch64_sve
extern const function_shape *const reduction_wide;
extern const function_shape *const set;
extern const function_shape *const setffr;
+ extern const function_shape *const set_neonq;
extern const function_shape *const shift_left_imm_long;
extern const function_shape *const shift_left_imm_to_uint;
extern const function_shape *const shift_right_imm;
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
b/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
index
af816c4c9e705d9cc4bce5cc50481cb27e6a03a7..4b0a84fe0cb5b5f4bc6b7dd012de0bc75ee4326b
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
+++ b/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
@@ -29,6 +29,7 @@
#include "optabs.h"
#include "aarch64-sve-builtins.h"
#include "aarch64-sve-builtins-shapes.h"
+#include "aarch64-builtins.h"
/* In the comments below, _t0 represents the first type suffix and _t1
represents the second. Square brackets enclose characters that are
@@ -158,6 +159,8 @@ parse_element_type (const function_instance &instance,
const char *&format)
s<elt> - a scalar type with the given element suffix
t<elt> - a vector or tuple type with given element suffix [*1]
v<elt> - a vector with the given element suffix
+ D<elt> - a 64 bit neon vector
+ Q<elt> - a 128 bit neon vector
where <elt> has the format described above parse_element_type
@@ -224,6 +227,20 @@ parse_type (const function_instance &instance, const char
*&format)
return acle_vector_types[0][type_suffixes[suffix].vector_type];
}
+ if (ch == 'D')
+ {
+ type_suffix_index suffix = parse_element_type (instance, format);
+ int neon_index = type_suffixes[suffix].neon64_type;
+ return aarch64_simd_types[neon_index].itype;
+ }
+
+ if (ch == 'Q')
+ {
+ type_suffix_index suffix = parse_element_type (instance, format);
+ int neon_index = type_suffixes[suffix].neon128_type;
+ return aarch64_simd_types[neon_index].itype;
+ }
+
gcc_unreachable ();
}
@@ -1917,6 +1934,67 @@ struct get_def : public overloaded_base<0>
};
SHAPE (get)
+/* <t0>xN_t svfoo[_t0](sv<t0>_t). */
+struct get_neonq_def : public overloaded_base<0>
+{
+ void
+ build (function_builder &b, const function_group_info &group) const override
+ {
+ b.add_overloaded_functions (group, MODE_none);
+ build_all (b, "Q0,v0", group, MODE_none);
+ }
+ tree
+ resolve (function_resolver &r) const override
+ {
+ return r.resolve_unary ();
+ }
+};
+SHAPE (get_neonq)
+
+/* sv<t0>_t svfoo[_t0](sv<t0>_t, <t0>xN_t). */
+struct set_neonq_def : public overloaded_base<0>
+{
+ void
+ build (function_builder &b, const function_group_info &group) const override
+ {
+ b.add_overloaded_functions (group, MODE_none);
+ build_all (b, "v0,v0,Q0", group, MODE_none);
+ }
+ tree
+ resolve (function_resolver &r) const override
+ {
+ unsigned int i, nargs;
+ type_suffix_index type;
+ if (!r.check_gp_argument (2, i, nargs)
+ || (type = r.infer_neon128_vector_type (i + 1)) == NUM_TYPE_SUFFIXES)
+ return error_mark_node;
+ return r.resolve_to (r.mode_suffix_id, type);
+ }
+};
+SHAPE (set_neonq)
+
+/* sv<t0>_t svfoo[_t0](<t0>xN_t). */
+struct dup_neonq_def : public overloaded_base<0>
+{
+ void
+ build (function_builder &b, const function_group_info &group) const override
+ {
+ b.add_overloaded_functions (group, MODE_none);
+ build_all (b, "v0,Q0", group, MODE_none);
+ }
+ tree
+ resolve (function_resolver &r) const override
+ {
+ unsigned int i, nargs;
+ type_suffix_index type;
+ if (!r.check_gp_argument (1, i, nargs)
+ || (type = r.infer_neon128_vector_type (i)) == NUM_TYPE_SUFFIXES)
+ return error_mark_node;
+ return r.resolve_to (r.mode_suffix_id, type);
+ }
+};
+SHAPE (dup_neonq)
+
/* sv<t0>_t svfoo[_t0](sv<t0>_t, uint64_t)
<t0>_t svfoo[_n_t0](<t0>_t, uint64_t)
diff --git a/gcc/config/aarch64/aarch64-sve-builtins.cc
b/gcc/config/aarch64/aarch64-sve-builtins.cc
index
161a14edde7c9fb1b13b146cf50463e2d78db264..6ff5c65e2610de8309a57b004e16d4602ea76999
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins.cc
+++ b/gcc/config/aarch64/aarch64-sve-builtins.cc
@@ -52,6 +52,7 @@
#include "aarch64-sve-builtins-base.h"
#include "aarch64-sve-builtins-sve2.h"
#include "aarch64-sve-builtins-shapes.h"
+#include "aarch64-builtins.h"
namespace aarch64_sve {
@@ -127,7 +128,8 @@ CONSTEXPR const mode_suffix_info mode_suffixes[] = {
/* Static information about each type_suffix_index. */
CONSTEXPR const type_suffix_info type_suffixes[NUM_TYPE_SUFFIXES + 1] = {
-#define DEF_SVE_TYPE_SUFFIX(NAME, ACLE_TYPE, CLASS, BITS, MODE) \
+#define DEF_SVE_NEON_TYPE_SUFFIX(NAME, ACLE_TYPE, CLASS, BITS, MODE, \
+ NEON64, NEON128) \
{ "_" #NAME, \
VECTOR_TYPE_##ACLE_TYPE, \
TYPE_##CLASS, \
@@ -138,10 +140,15 @@ CONSTEXPR const type_suffix_info
type_suffixes[NUM_TYPE_SUFFIXES + 1] = {
TYPE_##CLASS == TYPE_float, \
TYPE_##CLASS == TYPE_bool, \
0, \
- MODE },
+ MODE, \
+ NEON64, \
+ NEON128 },
+#define DEF_SVE_TYPE_SUFFIX(NAME, ACLE_TYPE, CLASS, BITS, MODE) \
+ DEF_SVE_NEON_TYPE_SUFFIX (NAME, ACLE_TYPE, CLASS, BITS, MODE, \
+ ARM_NEON_H_TYPES_LAST, ARM_NEON_H_TYPES_LAST)
#include "aarch64-sve-builtins.def"
{ "", NUM_VECTOR_TYPES, TYPE_bool, 0, 0, false, false, false, false,
- 0, VOIDmode }
+ 0, VOIDmode, ARM_NEON_H_TYPES_LAST, ARM_NEON_H_TYPES_LAST }
};
/* Define a TYPES_<combination> macro for each combination of type
@@ -529,6 +536,13 @@ static CONSTEXPR const function_group_info
function_groups[] = {
#include "aarch64-sve-builtins.def"
};
+/* A list of all NEON-SVE-Bridge ACLE functions. */
+static CONSTEXPR const function_group_info neon_sve_function_groups[] = {
+#define DEF_NEON_SVE_FUNCTION(NAME, SHAPE, TYPES, PREDS) \
+ { #NAME, &neon_sve_bridge_functions::NAME, &shapes::SHAPE, types_##TYPES,
preds_##PREDS },
+#include "aarch64-neon-sve-bridge-builtins.def"
+};
+
/* The scalar type associated with each vector type. */
extern GTY(()) tree scalar_types[NUM_VECTOR_TYPES];
tree scalar_types[NUM_VECTOR_TYPES];
@@ -1403,6 +1417,32 @@ function_resolver::infer_integer_vector_type (unsigned
int argno)
return type;
}
+type_suffix_index
+function_resolver::infer_neon128_vector_type (unsigned int argno)
+{
+ tree actual = get_argument_type (argno);
+ if (actual == error_mark_node)
+ return NUM_TYPE_SUFFIXES;
+
+ for (unsigned int suffix_i = 0; suffix_i < NUM_TYPE_SUFFIXES; ++suffix_i)
+ {
+ int neon_index = type_suffixes[suffix_i].neon128_type;
+ if (neon_index != ARM_NEON_H_TYPES_LAST)
+ {
+ tree type = aarch64_simd_types[neon_index].itype;
+ if (type && matches_type_p (type, actual))
+ {
+ return type_suffix_index (suffix_i);
+ }
+ }
+ }
+
+ error_at (location, "passing %qT to argument %d of %qE, which"
+ " expects a 128 bit NEON vector type", actual, argno + 1, fndecl);
+ return NUM_TYPE_SUFFIXES;
+}
+
+
/* Like infer_vector_type, but also require the type to be an unsigned
integer. */
type_suffix_index
@@ -3410,6 +3450,13 @@ init_builtins ()
handle_arm_sve_h ();
}
+void
+init_neon_sve_builtins ()
+{
+ if (in_lto_p)
+ handle_arm_neon_sve_bridge_h ();
+}
+
/* Register vector type TYPE under its arm_sve.h name. */
static void
register_vector_type (vector_type_index type)
@@ -3560,6 +3607,16 @@ handle_arm_sve_h ()
builder.register_function_group (function_groups[i]);
}
+/* Implement #pragma GCC aarch64 "arm_neon_sve_bridge.h". */
+void
+handle_arm_neon_sve_bridge_h ()
+{
+ /* Define the functions. */
+ function_builder builder;
+ for (unsigned int i = 0; i < ARRAY_SIZE (neon_sve_function_groups); ++i)
+ builder.register_function_group (neon_sve_function_groups[i]);
+}
+
/* Return the function decl with SVE function subcode CODE, or error_mark_node
if no such function exists. */
tree
diff --git a/gcc/config/aarch64/aarch64-sve-builtins.def
b/gcc/config/aarch64/aarch64-sve-builtins.def
index
534f6e69d72342fdcfcc00bd330585db1eae32e1..e8b4a919e1bb7a2d5d3239e6d303c9ee4e73d54f
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins.def
+++ b/gcc/config/aarch64/aarch64-sve-builtins.def
@@ -29,6 +29,11 @@
#define DEF_SVE_TYPE_SUFFIX(A, B, C, D, E)
#endif
+#ifndef DEF_SVE_NEON_TYPE_SUFFIX
+#define DEF_SVE_NEON_TYPE_SUFFIX(A, B, C, D, E, F, G) \
+ DEF_SVE_TYPE_SUFFIX(A, B, C, D, E)
+#endif
+
#ifndef DEF_SVE_FUNCTION
#define DEF_SVE_FUNCTION(A, B, C, D)
#endif
@@ -82,23 +87,36 @@ DEF_SVE_TYPE_SUFFIX (b8, svbool_t, bool, 8, VNx16BImode)
DEF_SVE_TYPE_SUFFIX (b16, svbool_t, bool, 16, VNx8BImode)
DEF_SVE_TYPE_SUFFIX (b32, svbool_t, bool, 32, VNx4BImode)
DEF_SVE_TYPE_SUFFIX (b64, svbool_t, bool, 64, VNx2BImode)
-DEF_SVE_TYPE_SUFFIX (bf16, svbfloat16_t, bfloat, 16, VNx8BFmode)
-DEF_SVE_TYPE_SUFFIX (f16, svfloat16_t, float, 16, VNx8HFmode)
-DEF_SVE_TYPE_SUFFIX (f32, svfloat32_t, float, 32, VNx4SFmode)
-DEF_SVE_TYPE_SUFFIX (f64, svfloat64_t, float, 64, VNx2DFmode)
-DEF_SVE_TYPE_SUFFIX (s8, svint8_t, signed, 8, VNx16QImode)
-DEF_SVE_TYPE_SUFFIX (s16, svint16_t, signed, 16, VNx8HImode)
-DEF_SVE_TYPE_SUFFIX (s32, svint32_t, signed, 32, VNx4SImode)
-DEF_SVE_TYPE_SUFFIX (s64, svint64_t, signed, 64, VNx2DImode)
-DEF_SVE_TYPE_SUFFIX (u8, svuint8_t, unsigned, 8, VNx16QImode)
-DEF_SVE_TYPE_SUFFIX (u16, svuint16_t, unsigned, 16, VNx8HImode)
-DEF_SVE_TYPE_SUFFIX (u32, svuint32_t, unsigned, 32, VNx4SImode)
-DEF_SVE_TYPE_SUFFIX (u64, svuint64_t, unsigned, 64, VNx2DImode)
+DEF_SVE_NEON_TYPE_SUFFIX (bf16, svbfloat16_t, bfloat, 16, VNx8BFmode,
+ Bfloat16x4_t, Bfloat16x8_t)
+DEF_SVE_NEON_TYPE_SUFFIX (f16, svfloat16_t, float, 16, VNx8HFmode,
+ Float16x4_t, Float16x8_t)
+DEF_SVE_NEON_TYPE_SUFFIX (f32, svfloat32_t, float, 32, VNx4SFmode,
+ Float32x2_t, Float32x4_t)
+DEF_SVE_NEON_TYPE_SUFFIX (f64, svfloat64_t, float, 64, VNx2DFmode,
+ Float64x1_t, Float64x2_t)
+DEF_SVE_NEON_TYPE_SUFFIX (s8, svint8_t, signed, 8, VNx16QImode,
+ Int8x8_t, Int8x16_t)
+DEF_SVE_NEON_TYPE_SUFFIX (s16, svint16_t, signed, 16, VNx8HImode,
+ Int16x4_t, Int16x8_t)
+DEF_SVE_NEON_TYPE_SUFFIX (s32, svint32_t, signed, 32, VNx4SImode,
+ Int32x2_t, Int32x4_t)
+DEF_SVE_NEON_TYPE_SUFFIX (s64, svint64_t, signed, 64, VNx2DImode,
+ Int64x1_t, Int64x2_t)
+DEF_SVE_NEON_TYPE_SUFFIX (u8, svuint8_t, unsigned, 8, VNx16QImode,
+ Uint8x8_t, Uint8x16_t)
+DEF_SVE_NEON_TYPE_SUFFIX (u16, svuint16_t, unsigned, 16, VNx8HImode,
+ Uint16x4_t, Uint16x8_t)
+DEF_SVE_NEON_TYPE_SUFFIX (u32, svuint32_t, unsigned, 32, VNx4SImode,
+ Uint32x2_t, Uint32x4_t)
+DEF_SVE_NEON_TYPE_SUFFIX (u64, svuint64_t, unsigned, 64, VNx2DImode,
+ Uint64x1_t, Uint64x2_t)
#include "aarch64-sve-builtins-base.def"
#include "aarch64-sve-builtins-sve2.def"
#undef DEF_SVE_FUNCTION
+#undef DEF_SVE_NEON_TYPE_SUFFIX
#undef DEF_SVE_TYPE_SUFFIX
#undef DEF_SVE_TYPE
#undef DEF_SVE_MODE
diff --git a/gcc/config/aarch64/aarch64-sve-builtins.h
b/gcc/config/aarch64/aarch64-sve-builtins.h
index
a301570b82ed3477306d203080ccb76608322c09..d32bf5b57ae7b48a130a7794f3f8277ad59ed03e
100644
--- a/gcc/config/aarch64/aarch64-sve-builtins.h
+++ b/gcc/config/aarch64/aarch64-sve-builtins.h
@@ -20,6 +20,8 @@
#ifndef GCC_AARCH64_SVE_BUILTINS_H
#define GCC_AARCH64_SVE_BUILTINS_H
+#include "aarch64-builtins.h"
+
/* The full name of an SVE ACLE function is the concatenation of:
- the base name ("svadd", etc.)
@@ -206,6 +208,14 @@ struct mode_suffix_info
units_index displacement_units;
};
+#define ENTRY(E, M, Q, G) E,
+enum aarch64_simd_type
+{
+#include "aarch64-simd-builtin-types.def"
+ ARM_NEON_H_TYPES_LAST
+};
+#undef ENTRY
+
/* Static information about a type suffix. */
struct type_suffix_info
{
@@ -235,6 +245,11 @@ struct type_suffix_info
/* The associated vector or predicate mode. */
machine_mode vector_mode : 16;
+
+ /* The corresponding 64-bit and 128-bit arm_neon.h types, or
+ ARM_NEON_H_TYPES_LAST if none. */
+ aarch64_simd_type neon64_type;
+ aarch64_simd_type neon128_type;
};
/* Static information about a set of functions. */
@@ -400,6 +415,7 @@ public:
type_suffix_index infer_vector_or_tuple_type (unsigned int, unsigned int);
type_suffix_index infer_vector_type (unsigned int);
type_suffix_index infer_integer_vector_type (unsigned int);
+ type_suffix_index infer_neon128_vector_type (unsigned int);
type_suffix_index infer_unsigned_vector_type (unsigned int);
type_suffix_index infer_sd_vector_type (unsigned int);
type_suffix_index infer_tuple_type (unsigned int);
diff --git a/gcc/config/aarch64/aarch64-sve.md
b/gcc/config/aarch64/aarch64-sve.md
index
da5534c3e32b3a8819c57a26582cfa5e22e63753..cf375c2a07bf6be9fc322cda015bda8ae9277cef
100644
--- a/gcc/config/aarch64/aarch64-sve.md
+++ b/gcc/config/aarch64/aarch64-sve.md
@@ -10766,3 +10766,48 @@
}
[(set_attr "movprfx" "*,yes")]
)
+
+(define_insn_and_split "@aarch64_sve_get_neonq_<mode>"
+ [(set (match_operand:<V128> 0 "register_operand" "=w")
+ (vec_select:<V128>
+ (match_operand:SVE_FULL 1 "register_operand" "w")
+ (match_operand 2 "descending_int_parallel")))]
+ "TARGET_SVE
+ && BYTES_BIG_ENDIAN
+ && known_eq (INTVAL (XVECEXP (operands[2], 0, 0)),
+ GET_MODE_NUNITS (<V128>mode) - 1)"
+ "#"
+ "&& reload_completed"
+ [(set (match_dup 0) (match_dup 1))]
+ {
+ operands[1] = gen_rtx_REG (<V128>mode, REGNO (operands[1]));
+ }
+)
+
+(define_insn "@aarch64_sve_set_neonq_<mode>"
+ [(set (match_operand:SVE_FULL 0 "register_operand" "=w")
+ (unspec:SVE_FULL
+ [(match_operand:SVE_FULL 1 "register_operand" "=w")
+ (match_operand:<V128> 2 "register_operand" "w")]
+ UNSPEC_SET_NEONQ))]
+ "TARGET_SVE
+ && BYTES_BIG_ENDIAN"
+)
+
+(define_insn_and_split "@aarch64_sve_dup_neonq_<mode>"
+ [(set (match_operand:SVE_FULL 0 "register_operand" "=w")
+ (vec_duplicate:SVE_FULL
+ (vec_select:<V128>
+ (match_operand:<V128> 1 "register_operand" "w")
+ (match_operand 2 "descending_int_parallel"))))]
+ "TARGET_SVE
+ && BYTES_BIG_ENDIAN
+ && known_eq (INTVAL (XVECEXP (operands[2], 0, 0)),
+ GET_MODE_NUNITS (<MODE>mode) - 1)"
+ "#"
+ "&& reload_completed"
+ [(set (match_dup 0) (match_dup 1))]
+ {
+ operands[1] = gen_rtx_REG (<MODE>mode, REGNO (operands[1]));
+ }
+)
\ No newline at end of file
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index
eba5d4a7e04b7af82437453a691d5607d98133c9..d193b083755270ec0ff3865c27c76a2505aa5f4d
100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -15421,6 +15421,7 @@ aarch64_init_builtins ()
{
aarch64_general_init_builtins ();
aarch64_sve::init_builtins ();
+ aarch64_sve::init_neon_sve_builtins ();
#ifdef SUBTARGET_INIT_BUILTINS
SUBTARGET_INIT_BUILTINS;
#endif
diff --git a/gcc/config/aarch64/arm_neon_sve_bridge.h
b/gcc/config/aarch64/arm_neon_sve_bridge.h
new file mode 100644
index
0000000000000000000000000000000000000000..8f526eae86b94f615d22fe8de52583bb403e102e
--- /dev/null
+++ b/gcc/config/aarch64/arm_neon_sve_bridge.h
@@ -0,0 +1,38 @@
+/* AArch64 NEON-SVE Bridge intrinsics include file.
+ Copyright (C) 2023 Free Software Foundation, Inc.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published
+ by the Free Software Foundation; either version 3, or (at your
+ option) any later version.
+
+ GCC is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
+ License for more details.
+
+ Under Section 7 of GPL version 3, you are granted additional
+ permissions described in the GCC Runtime Library Exception, version
+ 3.1, as published by the Free Software Foundation.
+
+ You should have received a copy of the GNU General Public License and
+ a copy of the GCC Runtime Library Exception along with this program;
+ see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _ARM_NEON_SVE_BRIDGE_H_
+#define _ARM_NEON_SVE_BRIDGE_H_
+
+#include <arm_neon.h>
+#include <arm_sve.h>
+
+/* NOTE: This implementation of arm_neon_sve_bridge.h is intentionally short.
It does
+ not define the types and intrinsic functions directly in C and C++
+ code, but instead uses the following pragma to tell GCC to insert the
+ necessary type and function definitions itself. The net effect is the
+ same, and the file is a complete implementation of arm_neon_sve_bridge.h.
*/
+#pragma GCC aarch64 "arm_neon_sve_bridge.h"
+
+#endif
\ No newline at end of file
diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
index
9398d713044433cd89b2a83db5ae7969feb1dcf7..0285e620f5e5507d0923f7348809d5f49887f12f
100644
--- a/gcc/config/aarch64/iterators.md
+++ b/gcc/config/aarch64/iterators.md
@@ -749,6 +749,7 @@
UNSPEC_FTSMUL ; Used in aarch64-sve.md.
UNSPEC_FTSSEL ; Used in aarch64-sve.md.
UNSPEC_SMATMUL ; Used in aarch64-sve.md.
+ UNSPEC_SET_NEONQ ; Used in aarch64-sve.md.
UNSPEC_UMATMUL ; Used in aarch64-sve.md.
UNSPEC_USMATMUL ; Used in aarch64-sve.md.
UNSPEC_TRN1Q ; Used in aarch64-sve.md.
diff --git a/gcc/testsuite/gcc.dg/torture/neon-sve-bridge.c
b/gcc/testsuite/gcc.dg/torture/neon-sve-bridge.c
new file mode 100644
index
0000000000000000000000000000000000000000..4f8f26cec1b44cdba222757a2988089b5ed4acc4
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/torture/neon-sve-bridge.c
@@ -0,0 +1,64 @@
+// { dg-options "-march=armv8.2-a+sve" }
+// { dg-do run { target aarch64_sve_hw } }
+
+#include <arm_neon_sve_bridge.h>
+
+extern void abort (void);
+
+int
+svget_neonq_test ()
+{
+ int64_t val1 = 987654321;
+ int64_t val2 = 123456789;
+ svint64_t sveInput = svdupq_n_s64 (val1, val2);
+ int64x2_t neonReturn = svget_neonq_s64 (sveInput);
+ int64_t val1Return = vgetq_lane_s64 (neonReturn, 0);
+ int64_t val2Return = vgetq_lane_s64 (neonReturn, 1);
+ if (val1 == val1Return && val2 == val2Return)
+ return 0;
+ return 1;
+}
+
+int
+svset_neonq_test ()
+{
+ int64_t val1 = 987654321;
+ int64_t val2 = 123456789;
+ int64x2_t NeonInput;
+ NeonInput = vsetq_lane_s64 (val1, NeonInput, 0);
+ NeonInput = vsetq_lane_s64 (val2, NeonInput, 1);
+ svint64_t sveReturn = svset_neonq_s64 (sveReturn, NeonInput);
+ int64_t val1Return = svlasta_s64 (svptrue_b64(), sveReturn);
+ int64_t val2Return = svlastb_s64 (svptrue_pat_b8(SV_VL16), sveReturn);
+ if (val1 == val1Return && val2 == val2Return)
+ return 0;
+ return 1;
+}
+
+int
+svdup_neonq_test ()
+{
+ int64_t val1 = 987654321;
+ int64_t val2 = 123456789;
+ int64x2_t NeonInput;
+ NeonInput = vsetq_lane_s64 (val1, NeonInput, 0);
+ NeonInput = vsetq_lane_s64 (val2, NeonInput, 1);
+ svint64_t sveReturn = svdup_neonq_s64 (NeonInput);
+ int64_t val1Return = svlasta_s64 (svptrue_b64(), sveReturn);
+ int64_t val2Return = svlastb_s64 (svptrue_b64(), sveReturn);
+ if (val1 == val1Return && val2 == val2Return)
+ return 0;
+ return 1;
+}
+
+int
+main ()
+{
+ if (svget_neonq_test () == 1)
+ abort ();
+ if (svset_neonq_test () == 1)
+ abort ();
+ if (svdup_neonq_test () == 1)
+ abort ();
+ return 0;
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_bf16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_bf16.c
new file mode 100644
index
0000000000000000000000000000000000000000..9b4b762bf13383c812b907debfd04ccb107b7769
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_bf16.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_bf16_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_bf16_z0, bfloat16x8_t, svbfloat16_t,
+ z0 = svdup_neonq_bf16 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_bf16_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_bf16_z4, bfloat16x8_t, svbfloat16_t,
+ z4_res = svdup_neonq_bf16 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_bf16_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_bf16_z5, bfloat16x8_t, svbfloat16_t,
+ z5_res = svdup_neonq_bf16 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f16.c
new file mode 100644
index
0000000000000000000000000000000000000000..699e966c2af17ed051a72f38f5d4b1de9a58322c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f16.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_f16_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f16_z0, float16x8_t, svfloat16_t,
+ z0 = svdup_neonq_f16 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_f16_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f16_z4, float16x8_t, svfloat16_t,
+ z4_res = svdup_neonq_f16 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_f16_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f16_z5, float16x8_t, svfloat16_t,
+ z5_res = svdup_neonq_f16 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f32.c
new file mode 100644
index
0000000000000000000000000000000000000000..cd606ef0d0c4f00c2f3c2057148f724f60a6c785
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f32.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_f32_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f32_z0, float32x4_t, svfloat32_t,
+ z0 = svdup_neonq_f32 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_f32_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f32_z4, float32x4_t, svfloat32_t,
+ z4_res = svdup_neonq_f32 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_f32_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f32_z5, float32x4_t, svfloat32_t,
+ z5_res = svdup_neonq_f32 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f64.c
new file mode 100644
index
0000000000000000000000000000000000000000..4dd3e507ea33d51a1f3dd6ea79951acab76c37ef
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_f64.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_f64_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f64_z0, float64x2_t, svfloat64_t,
+ z0 = svdup_neonq_f64 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_f64_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f64_z4, float64x2_t, svfloat64_t,
+ z4_res = svdup_neonq_f64 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_f64_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_f64_z5, float64x2_t, svfloat64_t,
+ z5_res = svdup_neonq_f64 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s16.c
new file mode 100644
index
0000000000000000000000000000000000000000..e70e454a160295af11ad7b610235903a41011e21
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s16.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_s16_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s16_z0, int16x8_t, svint16_t,
+ z0 = svdup_neonq_s16 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_s16_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s16_z4, int16x8_t, svint16_t,
+ z4_res = svdup_neonq_s16 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_s16_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s16_z5, int16x8_t, svint16_t,
+ z5_res = svdup_neonq_s16 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s32.c
new file mode 100644
index
0000000000000000000000000000000000000000..9ef9228d5298e85cd8b41393c2ef03cade0fcf9e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s32.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_s32_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s32_z0, int32x4_t, svint32_t,
+ z0 = svdup_neonq_s32 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_s32_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s32_z4, int32x4_t, svint32_t,
+ z4_res = svdup_neonq_s32 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_s32_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s32_z5, int32x4_t, svint32_t,
+ z5_res = svdup_neonq_s32 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s64.c
new file mode 100644
index
0000000000000000000000000000000000000000..d3ed22220fae280bd0a828e8d3afe1df03cc70be
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s64.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_s64_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s64_z0, int64x2_t, svint64_t,
+ z0 = svdup_neonq_s64 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_s64_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s64_z4, int64x2_t, svint64_t,
+ z4_res = svdup_neonq_s64 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_s64_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s64_z5, int64x2_t, svint64_t,
+ z5_res = svdup_neonq_s64 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s8.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s8.c
new file mode 100644
index
0000000000000000000000000000000000000000..6437f858a4ce3200dfafb647de686fb83e43bae3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_s8.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_s8_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s8_z0, int8x16_t, svint8_t,
+ z0 = svdup_neonq_s8 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_s8_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s8_z4, int8x16_t, svint8_t,
+ z4_res = svdup_neonq_s8 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_s8_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_s8_z5, int8x16_t, svint8_t,
+ z5_res = svdup_neonq_s8 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u16.c
new file mode 100644
index
0000000000000000000000000000000000000000..5e7b2230c19ce0e763e1af63baf28743c34fb76a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u16.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_u16_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u16_z0, uint16x8_t, svuint16_t,
+ z0 = svdup_neonq_u16 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_u16_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u16_z4, uint16x8_t, svuint16_t,
+ z4_res = svdup_neonq_u16 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_u16_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u16_z5, uint16x8_t, svuint16_t,
+ z5_res = svdup_neonq_u16 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u32.c
new file mode 100644
index
0000000000000000000000000000000000000000..ce34d4f213f7cab19729463c4e3498d49fb8c47a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u32.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_u32_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u32_z0, uint32x4_t, svuint32_t,
+ z0 = svdup_neonq_u32 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_u32_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u32_z4, uint32x4_t, svuint32_t,
+ z4_res = svdup_neonq_u32 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_u32_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u32_z5, uint32x4_t, svuint32_t,
+ z5_res = svdup_neonq_u32 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u64.c
new file mode 100644
index
0000000000000000000000000000000000000000..1008740890b6759525078240c9647612bae91a74
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u64.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_u64_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u64_z0, uint64x2_t, svuint64_t,
+ z0 = svdup_neonq_u64 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_u64_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u64_z4, uint64x2_t, svuint64_t,
+ z4_res = svdup_neonq_u64 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_u64_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u64_z5, uint64x2_t, svuint64_t,
+ z5_res = svdup_neonq_u64 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u8.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u8.c
new file mode 100644
index
0000000000000000000000000000000000000000..c27ff8f34a75f426923e77dc05ac9eff3f0fa248
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/dup_neonq_u8.c
@@ -0,0 +1,30 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** dup_neonq_u8_z0:
+** dup z0.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u8_z0, uint8x16_t, svuint8_t,
+ z0 = svdup_neonq_u8 (z4),
+ z0 = svdup_neonq (z4))
+
+/*
+** dup_neonq_u8_z4:
+** dup z4.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u8_z4, uint8x16_t, svuint8_t,
+ z4_res = svdup_neonq_u8 (z4),
+ z4_res = svdup_neonq (z4))
+
+/*
+** dup_neonq_u8_z5:
+** dup z5.q, z4.q\[0\]
+** ret
+*/
+TEST_DUP_NEONQ (dup_neonq_u8_z5, uint8x16_t, svuint8_t,
+ z5_res = svdup_neonq_u8 (z4),
+ z5_res = svdup_neonq (z4))
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_bf16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_bf16.c
new file mode 100644
index
0000000000000000000000000000000000000000..1d43a25fb568b4ba494fcca4f530b126d0ed3c3b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_bf16.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_bf16_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_bf16_z0, svbfloat16_t, bfloat16x8_t,
+ z0 = svget_neonq_bf16 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_bf16_z4:
+** ret
+*/
+TEST_GET (get_neonq_bf16_z4, svbfloat16_t, bfloat16x8_t,
+ z4_res = svget_neonq_bf16 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_bf16_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_bf16_z5, svbfloat16_t, bfloat16x8_t,
+ z5_res = svget_neonq_bf16 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f16.c
new file mode 100644
index
0000000000000000000000000000000000000000..d2994d55fcf9deb1a28d8dbb6479669a84425e69
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f16.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_f16_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_f16_z0, svfloat16_t, float16x8_t,
+ z0 = svget_neonq_f16 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_f16_z4:
+** ret
+*/
+TEST_GET (get_neonq_f16_z4, svfloat16_t, float16x8_t,
+ z4_res = svget_neonq_f16 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_f16_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_f16_z5, svfloat16_t, float16x8_t,
+ z5_res = svget_neonq_f16 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f32.c
new file mode 100644
index
0000000000000000000000000000000000000000..1ef5244ffdd3df21048580fb8d303a178de3dd7e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f32.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_f32_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_f32_z0, svfloat32_t, float32x4_t,
+ z0 = svget_neonq_f32 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_f32_z4:
+** ret
+*/
+TEST_GET (get_neonq_f32_z4, svfloat32_t, float32x4_t,
+ z4_res = svget_neonq_f32 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_f32_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_f32_z5, svfloat32_t, float32x4_t,
+ z5_res = svget_neonq_f32 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f64.c
new file mode 100644
index
0000000000000000000000000000000000000000..6c8aafc9dc8552906ecba824ce46819a1ccf20a6
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_f64.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_f64_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_f64_z0, svfloat64_t, float64x2_t,
+ z0 = svget_neonq_f64 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_f64_z4:
+** ret
+*/
+TEST_GET (get_neonq_f64_z4, svfloat64_t, float64x2_t,
+ z4_res = svget_neonq_f64 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_f64_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_f64_z5, svfloat64_t, float64x2_t,
+ z5_res = svget_neonq_f64 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s16.c
new file mode 100644
index
0000000000000000000000000000000000000000..65082a8d1b4a7659141aac1641849787f1d11ed4
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s16.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_s16_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_s16_z0, svint16_t, int16x8_t,
+ z0 = svget_neonq_s16 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_s16_z4:
+** ret
+*/
+TEST_GET (get_neonq_s16_z4, svint16_t, int16x8_t,
+ z4_res = svget_neonq_s16 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_s16_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_s16_z5, svint16_t, int16x8_t,
+ z5_res = svget_neonq_s16 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s32.c
new file mode 100644
index
0000000000000000000000000000000000000000..509514649678aaa9d37add72e53efffff1d6b633
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s32.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_s32_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_s32_z0, svint32_t, int32x4_t,
+ z0 = svget_neonq_s32 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_s32_z4:
+** ret
+*/
+TEST_GET (get_neonq_s32_z4, svint32_t, int32x4_t,
+ z4_res = svget_neonq_s32 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_s32_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_s32_z5, svint32_t, int32x4_t,
+ z5_res = svget_neonq_s32 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s64.c
new file mode 100644
index
0000000000000000000000000000000000000000..5b3a455d65713e4ebab2fc48f61ec3d77b861cac
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s64.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_s64_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_s64_z0, svint64_t, int64x2_t,
+ z0 = svget_neonq_s64 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_s64_z4:
+** ret
+*/
+TEST_GET (get_neonq_s64_z4, svint64_t, int64x2_t,
+ z4_res = svget_neonq_s64 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_s64_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_s64_z5, svint64_t, int64x2_t,
+ z5_res = svget_neonq_s64 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s8.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s8.c
new file mode 100644
index
0000000000000000000000000000000000000000..aa40ed4499006d028d00fe7120a9e66ff9493b08
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_s8.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_s8_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_s8_z0, svint8_t, int8x16_t,
+ z0 = svget_neonq_s8 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_s8_z4:
+** ret
+*/
+TEST_GET (get_neonq_s8_z4, svint8_t, int8x16_t,
+ z4_res = svget_neonq_s8 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_s8_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_s8_z5, svint8_t, int8x16_t,
+ z5_res = svget_neonq_s8 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u16.c
new file mode 100644
index
0000000000000000000000000000000000000000..a1db23ff9281aae53567f95ce25a35eef72b8677
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u16.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_u16_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_u16_z0, svuint16_t, uint16x8_t,
+ z0 = svget_neonq_u16 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_u16_z4:
+** ret
+*/
+TEST_GET (get_neonq_u16_z4, svuint16_t, uint16x8_t,
+ z4_res = svget_neonq_u16 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_u16_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_u16_z5, svuint16_t, uint16x8_t,
+ z5_res = svget_neonq_u16 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u32.c
new file mode 100644
index
0000000000000000000000000000000000000000..808bfb87ca7323a08ecf1eb6a56ba3b6060a700e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u32.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_u32_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_u32_z0, svuint32_t, uint32x4_t,
+ z0 = svget_neonq_u32 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_u32_z4:
+** ret
+*/
+TEST_GET (get_neonq_u32_z4, svuint32_t, uint32x4_t,
+ z4_res = svget_neonq_u32 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_u32_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_u32_z5, svuint32_t, uint32x4_t,
+ z5_res = svget_neonq_u32 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u64.c
new file mode 100644
index
0000000000000000000000000000000000000000..2cc5a93888f4b145c7e6d1d1a541229dfd4e8163
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u64.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_u64_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_u64_z0, svuint64_t, uint64x2_t,
+ z0 = svget_neonq_u64 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_u64_z4:
+** ret
+*/
+TEST_GET (get_neonq_u64_z4, svuint64_t, uint64x2_t,
+ z4_res = svget_neonq_u64 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_u64_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_u64_z5, svuint64_t, uint64x2_t,
+ z5_res = svget_neonq_u64 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u8.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u8.c
new file mode 100644
index
0000000000000000000000000000000000000000..6877d0fa08692f741a1e9e2457da5d5167bc298d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/get_neonq_u8.c
@@ -0,0 +1,29 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** get_neonq_u8_z0:
+** mov v0.16b, v4.16b
+** ret
+*/
+TEST_GET (get_neonq_u8_z0, svuint8_t, uint8x16_t,
+ z0 = svget_neonq_u8 (z4),
+ z0 = svget_neonq (z4))
+
+/*
+** get_neonq_u8_z4:
+** ret
+*/
+TEST_GET (get_neonq_u8_z4, svuint8_t, uint8x16_t,
+ z4_res = svget_neonq_u8 (z4),
+ z4_res = svget_neonq (z4))
+
+/*
+** get_neonq_u8_z5:
+** mov z5.d, z4.d
+** ret
+*/
+TEST_GET (get_neonq_u8_z5, svuint8_t, uint8x16_t,
+ z5_res = svget_neonq_u8 (z4),
+ z5_res = svget_neonq (z4))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_bf16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_bf16.c
new file mode 100644
index
0000000000000000000000000000000000000000..50f0ef292ff2def75b2fb09c4d3744e9f2775c33
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_bf16.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_bf16_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_bf16_z23, svbfloat16_t, bfloat16x8_t,
+ z23 = svset_neonq_bf16 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_bf16_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** mov z30.h, #0
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_bf16_z4, svbfloat16_t, bfloat16x8_t,
+ z4 = svset_neonq_bf16 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f16.c
new file mode 100644
index
0000000000000000000000000000000000000000..eefa6f7d3680558baba7e4abe7265dba18dfbb01
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f16.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_f16_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_f16_z23, svfloat16_t, float16x8_t,
+ z23 = svset_neonq_f16 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_f16_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** mov z30.h, #0
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_f16_z4, svfloat16_t, float16x8_t,
+ z4 = svset_neonq_f16 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f32.c
new file mode 100644
index
0000000000000000000000000000000000000000..4dabb9376cc4f50798e6118693e3946ec007b70b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f32.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_f32_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1w z4.s, p3, \[sp\]
+** str q0, \[sp\]
+** ld1w z31.s, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_f32_z23, svfloat32_t, float32x4_t,
+ z23 = svset_neonq_f32 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_f32_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1w z4.s, p3, \[sp\]
+** mov z30.s, #0
+** str q0, \[sp\]
+** ld1w z31.s, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_f32_z4, svfloat32_t, float32x4_t,
+ z4 = svset_neonq_f32 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f64.c
new file mode 100644
index
0000000000000000000000000000000000000000..106bc2445305cd15f28164064af05258df2b5cdc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_f64.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_f64_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1d z4.d, p3, \[sp\]
+** str q0, \[sp\]
+** ld1d z31.d, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_f64_z23, svfloat64_t, float64x2_t,
+ z23 = svset_neonq_f64 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_f64_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1d z4.d, p3, \[sp\]
+** mov z30.d, #0
+** str q0, \[sp\]
+** ld1d z31.d, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_f64_z4, svfloat64_t, float64x2_t,
+ z4 = svset_neonq_f64 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s16.c
new file mode 100644
index
0000000000000000000000000000000000000000..baf2b8b1588552417257abe9e7bd2cfa5cb074e2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s16.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_s16_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s16_z23, svint16_t, int16x8_t,
+ z23 = svset_neonq_s16 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_s16_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s16_z4, svint16_t, int16x8_t,
+ z4 = svset_neonq_s16 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s32.c
new file mode 100644
index
0000000000000000000000000000000000000000..28b1711544b075f034f6a205560617a8e4073fd7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s32.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_s32_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1w z4.s, p3, \[sp\]
+** str q0, \[sp\]
+** ld1w z31.s, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s32_z23, svint32_t, int32x4_t,
+ z23 = svset_neonq_s32 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_s32_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1w z4.s, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1w z31.s, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s32_z4, svint32_t, int32x4_t,
+ z4 = svset_neonq_s32 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s64.c
new file mode 100644
index
0000000000000000000000000000000000000000..acff7a5c2ffae0a60b34b6accd48472896bd9e50
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s64.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_s64_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1d z4.d, p3, \[sp\]
+** str q0, \[sp\]
+** ld1d z31.d, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s64_z23, svint64_t, int64x2_t,
+ z23 = svset_neonq_s64 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_s64_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1d z4.d, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1d z31.d, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s64_z4, svint64_t, int64x2_t,
+ z4 = svset_neonq_s64 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s8.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s8.c
new file mode 100644
index
0000000000000000000000000000000000000000..d656400cc10c78b3d0120f83f91e0a5885a25a5c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_s8.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_s8_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1b z4.b, p3, \[sp\]
+** str q0, \[sp\]
+** ld1b z31.b, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s8_z23, svint8_t, int8x16_t,
+ z23 = svset_neonq_s8 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_s8_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1b z4.b, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1b z31.b, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_s8_z4, svint8_t, int8x16_t,
+ z4 = svset_neonq_s8 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u16.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u16.c
new file mode 100644
index
0000000000000000000000000000000000000000..203d010dbd1b66426719ce64f4e130c8d861f0f5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u16.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_u16_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u16_z23, svuint16_t, uint16x8_t,
+ z23 = svset_neonq_u16 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_u16_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1h z4.h, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1h z31.h, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u16_z4, svuint16_t, uint16x8_t,
+ z4 = svset_neonq_u16 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u32.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u32.c
new file mode 100644
index
0000000000000000000000000000000000000000..f95491e84514754433b579fb9e8d0c5e21607645
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u32.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_u32_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1w z4.s, p3, \[sp\]
+** str q0, \[sp\]
+** ld1w z31.s, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u32_z23, svuint32_t, uint32x4_t,
+ z23 = svset_neonq_u32 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_u32_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1w z4.s, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1w z31.s, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u32_z4, svuint32_t, uint32x4_t,
+ z4 = svset_neonq_u32 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u64.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u64.c
new file mode 100644
index
0000000000000000000000000000000000000000..206c1fe37fc28deef547e6aee6e5ce1c44bd26d3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u64.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_u64_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1d z4.d, p3, \[sp\]
+** str q0, \[sp\]
+** ld1d z31.d, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u64_z23, svuint64_t, uint64x2_t,
+ z23 = svset_neonq_u64 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_u64_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1d z4.d, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1d z31.d, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u64_z4, svuint64_t, uint64x2_t,
+ z4 = svset_neonq_u64 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u8.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u8.c
new file mode 100644
index
0000000000000000000000000000000000000000..aa66ca8351c9be72ca682bee83c669c420d0dc1a
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/set_neonq_u8.c
@@ -0,0 +1,32 @@
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sve_acle.h"
+
+/*
+** set_neonq_u8_z23:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1b z4.b, p3, \[sp\]
+** str q0, \[sp\]
+** ld1b z31.b, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u8_z23, svuint8_t, uint8x16_t,
+ z23 = svset_neonq_u8 (z4, z0),
+ z23 = svset_neonq (z4, z0))
+
+/*
+** set_neonq_u8_z4:
+** addvl sp, sp, #-1
+** ptrue p3.b, all
+** st1b z4.b, p3, \[sp\]
+** mov z30.b, #0
+** str q0, \[sp\]
+** ld1b z31.b, p3/z, \[sp\]
+** addvl sp, sp, #1
+** ret
+*/
+TEST_SET (set_neonq_u8_z4, svuint8_t, uint8x16_t,
+ z4 = svset_neonq_u8 (z4, z0),
+ z4 = svset_neonq (z4, z0))
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
index
fbf392b3ed49be37158790d3db6689bdb3ac72ef..bebcb295d2078c1f19f9f67f7e6cbdc33741a25c
100644
--- a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
@@ -1,7 +1,7 @@
#ifndef TEST_SVE_ACLE_H
#define TEST_SVE_ACLE_H 1
-#include <arm_sve.h>
+#include <arm_neon_sve_bridge.h>
#if defined (TEST_OVERLOADS)
#define INVOKE(CODE1, CODE2) CODE2
@@ -400,9 +400,22 @@
PROTO (NAME, void, (ZTYPE z0, ZTYPE z1, ZTYPE z2, ZTYPE z3, \
TTYPE z4)) \
{ \
+ TTYPE z23; \
register TTYPE z24 __asm ("z24"); \
INVOKE (CODE1, CODE2); \
- __asm volatile ("" :: "w" (z4), "w" (z24)); \
+ __asm volatile ("" :: "w" (z4), "w" (z24), "w" (z23)); \
+ }
+
+#define TEST_DUP_NEONQ(NAME, TTYPE, ZTYPE, CODE1, CODE2) \
+ PROTO (NAME, void, (ZTYPE unused0, ZTYPE unused1, \
+ ZTYPE unused2, ZTYPE unused3, TTYPE z4)) \
+ { \
+ register ZTYPE z0 __asm ("z0"); \
+ register ZTYPE z4_res __asm ("z4"); \
+ register ZTYPE z5_res __asm ("z5"); \
+ INVOKE (CODE1, CODE2); \
+ __asm volatile ("" :: "w" (z0), "w" (z4_res), \
+ "w" (z5_res)); \
}
#define TEST_TBL2(NAME, TTYPE, ZTYPE, UTYPE, CODE1, CODE2) \
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/dup_neonq_1.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/dup_neonq_1.c
new file mode 100644
index
0000000000000000000000000000000000000000..a07158f6ba8dc739ff6e3a5a14e44c619157e2b9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/dup_neonq_1.c
@@ -0,0 +1,20 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=armv8.2-a+sve -std=c99 -Wall -Wextra" } */
+
+#include <arm_neon_sve_bridge.h>
+
+float64x2_t
+f1 (int8x16_t s8, svint8_t sveS8, int64x2_t s64, int8x8x2_t s8x2)
+{
+ float64x2_t f64;
+
+ sveS8 = svdup_neonq (s8);
+ sveS8 = svdup_neonq (); /* { dg-error {too few arguments to function
'svdup_neonq'} } */
+ sveS8 = svdup_neonq (s8, 1); /* { dg-error {too many arguments to function
'svdup_neonq'} } */
+ sveS8 = svdup_neonq (sveS8); /* { dg-error {passing 'svint8_t' to argument 1
of 'svdup_neonq', which expects a 128 bit NEON vector type} } */
+ f64 = svdup_neonq (s8); /* { dg-error {incompatible types when assigning to
type 'float64x2_t' from type 'svint8_t'} } */
+ sveS8 = svdup_neonq (s8x2); /* { dg-error {passing 'int8x8x2_t' to argument
1 of 'svdup_neonq', which expects a 128 bit NEON vector type} } */
+ sveS8 = svdup_neonq (s64); /* { dg-error {incompatible types when assigning
to type 'svint8_t' from type 'svint64_t'} } */
+
+ return f64;
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/get_neonq_1.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/get_neonq_1.c
new file mode 100644
index
0000000000000000000000000000000000000000..9ccb47a3809d78c94d04a6a639163e86351e7813
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/get_neonq_1.c
@@ -0,0 +1,20 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=armv8.2-a+sve -std=c99 -Wall -Wextra" } */
+
+#include <arm_neon_sve_bridge.h>
+
+float64x2_t
+f1 (int8x16_t s8, svint8_t sveS8, svint8x2_t sveS8x2, svint64_t sveS64)
+{
+ float64x2_t f64;
+
+ s8 = svget_neonq (sveS8);
+ s8 = svget_neonq (); /* { dg-error {too few arguments to function
'svget_neonq'} } */
+ s8 = svget_neonq (sveS8, 1); /* { dg-error {too many arguments to function
'svget_neonq'} } */
+ s8 = svget_neonq (s8); /* { dg-error {passing 'int8x16_t' to argument 1 of
'svget_neonq', which expects an SVE vector type} } */
+ f64 = svget_neonq (sveS8); /* { dg-error {incompatible types when assigning
to type 'float64x2_t' from type '__Int8x16_t'} } */
+ s8 = svget_neonq (sveS8x2); /* { dg-error {passing 'svint8x2_t' to argument
1 of 'svget_neonq', which expects a single SVE vector rather than a tuple} } */
+ s8 = svget_neonq (sveS64); /* { dg-error {incompatible types when assigning
to type 'int8x16_t' from type '__Int64x2_t} } */
+
+ return f64;
+}
\ No newline at end of file
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/set_neonq_1.c
b/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/set_neonq_1.c
new file mode 100644
index
0000000000000000000000000000000000000000..f08172842a71d38a01857f3a880662cb990842d9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/general-c/set_neonq_1.c
@@ -0,0 +1,27 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-march=armv8.2-a+sve -std=c99 -Wall -Wextra" } */
+
+#include <arm_neon_sve_bridge.h>
+
+float64x2_t
+f1 (int8x16_t s8, svint8_t sveS8, svint8x2_t sveS8x2, svint64_t sveS64,
+ int64x2_t s64, svbfloat16_t sveBF16, bfloat16x8_t bf16, int8x8_t s8_64bit,
+ svbool_t svbool)
+{
+ float64x2_t f64;
+
+ sveS8 = svset_neonq (sveS8, s8);
+ sveS64 = svset_neonq (sveS64, s64);
+ sveBF16 = svset_neonq (sveBF16, bf16);
+ sveS8 = svset_neonq (); /* { dg-error {too few arguments to function
'svset_neonq'} } */
+ sveS8 = svset_neonq (sveS8, s8, 1); /* { dg-error {too many arguments to
function 'svset_neonq'} } */
+ sveS8 = svset_neonq (s8, s8); /* { dg-error {incompatible type for argument
1 of 'svset_neonq_s8'} } */
+ f64 = svset_neonq (sveS8, s8); /* { dg-error {incompatible types when
assigning to type 'float64x2_t' from type 'svint8_t'} } */
+ sveS8 = svset_neonq (sveS8x2, s8); /* { dg-error {incompatible type for
argument 1 of 'svset_neonq_s8'} } */
+ sveS8 = svset_neonq (sveS8, sveS8); /* { dg-error {passing 'svint8_t' to
argument 2 of 'svset_neonq', which expects a 128 bit NEON vector type} } */
+ sveS8 = svset_neonq (sveS8, s8_64bit); /* { dg-error {passing 'int8x8_t' to
argument 2 of 'svset_neonq', which expects a 128 bit NEON vector type} } */
+ sveS8 = svset_neonq (sveS64, s64); /* { dg-error {incompatible types when
assigning to type 'svint8_t' from type 'svint64_t} } */
+ sveS8 = svset_neonq (svbool, svbool); /* { dg-error {passing 'svbool_t' to
argument 2 of 'svset_neonq', which expects a 128 bit NEON vector type} } */
+
+ return f64;
+}
\ No newline at end of file
