This patch adds support for `target_version` function multiversioning to
the C frontend, specifically intended for enabling this for Aarch64
targets.
The functionality and behavior matches the CPP frontend. Which is to
say it is not ACLE compliant, as I have another patch achieving this.
The majority of the logic for this patch is in `pushdecl` and works as
follows (for a given set of FMV functions):
- When the first FMV function is pushed, a binding is created as
for any function declaration.
- When a second FMV function with distinct target is pushed, colliding
with the first, the first declaration is replaced with the dispatched
symbol declaration, and the two functions are stored in the cgraph
function_version structure of this funciton
- When subsequent declarations are pushed, they are added to the
cgraph function version structure also.
Additionally to this, the logic for analyzing functions has been
modified so that when the dispatcher function is analyzed it queues all
the function versions in its structure.
Note, while this should change little for C++ FMV, it does requires
a change in frontend hooks to enable the C FMV frontend. This is to allow
the creation of the FMV dispatched symbol declaration before all
versions are processed and in the structure. Notably, this means the
default version may not be present yet. This breaks previous assumptions.
This has been regression tested for both Aarch64 and X68_64.
Bootstrapped for aarch64-none-linux-gnu and x86_64-unknown-linux-gnu.
gcc/ChangeLog:
* attribs.cc (is_function_default_version): Add option for unannotated.
* attribs.h (is_function_default_version): Support option for
unannotated.
* calls.cc (get_c_function_version_dispatcher): New function
* calls.h (get_c_function_version_dispatcher): New function
* cgraph.cc (cgraph_node::record_function_versions): Change logic
(delete_function_version): Add new case.
* cgraphunit.cc (cgraph_node::analyze): Change logic
(analyze_functions): Enqueue FMV versions.
* config/aarch64/aarch64.cc (aarch64_get_function_versions_dispatcher):
Change logic
gcc/c-family/ChangeLog:
* c-gimplify.cc (c_gimplify_expr): Change logic.
gcc/c/ChangeLog:
* c-decl.cc (maybe_mark_function_versioned): New function.
(diagnose_mismatched_decls): Add allow duplicates parameter.
(pushdecl): Add FMV logic.
(start_function): Add error case.
(c_parse_final_cleanups): Add call to process_same_body_aliases.
gcc/testsuite/ChangeLog:
* g++.target/aarch64/mv-symbols10.C: New test.
* g++.target/aarch64/mv-symbols6.C: New test.
* g++.target/aarch64/mv-symbols8.C: New test.
* g++.target/aarch64/mv-symbols9.C: New test.
* gcc.target/aarch64/mv-1.c: New test.
* gcc.target/aarch64/mv-symbols-error1.c: New test.
* gcc.target/aarch64/mv-symbols-error10.c: New test.
* gcc.target/aarch64/mv-symbols-error2.c: New test.
* gcc.target/aarch64/mv-symbols-error3.c: New test.
* gcc.target/aarch64/mv-symbols-error4.c: New test.
* gcc.target/aarch64/mv-symbols-error5.c: New test.
* gcc.target/aarch64/mv-symbols-error6.c: New test.
* gcc.target/aarch64/mv-symbols-error7.c: New test.
* gcc.target/aarch64/mv-symbols-error8.c: New test.
* gcc.target/aarch64/mv-symbols-error9.c: New test.
* gcc.target/aarch64/mv-symbols1.c: New test.
* gcc.target/aarch64/mv-symbols10.c: New test.
* gcc.target/aarch64/mv-symbols11.c: New test.
* gcc.target/aarch64/mv-symbols2.c: New test.
* gcc.target/aarch64/mv-symbols3.c: New test.
* gcc.target/aarch64/mv-symbols4.c: New test.
* gcc.target/aarch64/mv-symbols5.c: New test.
* gcc.target/aarch64/mv-symbols6.c: New test.
* gcc.target/aarch64/mv-symbols7.c: New test.
* gcc.target/aarch64/mv-symbols8.c: New test.
* gcc.target/aarch64/mv-symbols9.c: New test.
* gcc.target/aarch64/mvc-symbols1.c: New test.
* gcc.target/aarch64/mvc-symbols2.c: New test.
* gcc.target/aarch64/mvc-symbols3.c: New test.
* gcc.target/aarch64/mvc-symbols4.c: New test.
---
gcc/attribs.cc | 21 ++-
gcc/attribs.h | 3 +-
gcc/c-family/c-gimplify.cc | 13 ++
gcc/c/c-decl.cc | 162 ++++++++++++++++--
gcc/calls.cc | 17 ++
gcc/calls.h | 2 +
gcc/cgraph.cc | 61 ++++++-
gcc/cgraphunit.cc | 23 +++
gcc/config/aarch64/aarch64.cc | 29 +---
.../g++.target/aarch64/mv-symbols10.C | 42 +++++
.../g++.target/aarch64/mv-symbols6.C | 16 ++
.../g++.target/aarch64/mv-symbols8.C | 47 +++++
.../g++.target/aarch64/mv-symbols9.C | 44 +++++
gcc/testsuite/gcc.target/aarch64/mv-1.c | 40 +++++
.../gcc.target/aarch64/mv-symbols-error1.c | 11 ++
.../gcc.target/aarch64/mv-symbols-error10.c | 11 ++
.../gcc.target/aarch64/mv-symbols-error2.c | 8 +
.../gcc.target/aarch64/mv-symbols-error3.c | 8 +
.../gcc.target/aarch64/mv-symbols-error4.c | 8 +
.../gcc.target/aarch64/mv-symbols-error5.c | 11 ++
.../gcc.target/aarch64/mv-symbols-error6.c | 8 +
.../gcc.target/aarch64/mv-symbols-error7.c | 12 ++
.../gcc.target/aarch64/mv-symbols-error8.c | 11 ++
.../gcc.target/aarch64/mv-symbols-error9.c | 10 ++
.../gcc.target/aarch64/mv-symbols1.c | 38 ++++
.../gcc.target/aarch64/mv-symbols10.c | 42 +++++
.../gcc.target/aarch64/mv-symbols11.c | 16 ++
.../gcc.target/aarch64/mv-symbols2.c | 28 +++
.../gcc.target/aarch64/mv-symbols3.c | 27 +++
.../gcc.target/aarch64/mv-symbols4.c | 31 ++++
.../gcc.target/aarch64/mv-symbols5.c | 36 ++++
.../gcc.target/aarch64/mv-symbols6.c | 16 ++
.../gcc.target/aarch64/mv-symbols7.c | 47 +++++
.../gcc.target/aarch64/mv-symbols8.c | 47 +++++
.../gcc.target/aarch64/mv-symbols9.c | 44 +++++
.../gcc.target/aarch64/mvc-symbols1.c | 25 +++
.../gcc.target/aarch64/mvc-symbols2.c | 15 ++
.../gcc.target/aarch64/mvc-symbols3.c | 19 ++
.../gcc.target/aarch64/mvc-symbols4.c | 12 ++
39 files changed, 1011 insertions(+), 50 deletions(-)
create mode 100644 gcc/testsuite/g++.target/aarch64/mv-symbols10.C
create mode 100644 gcc/testsuite/g++.target/aarch64/mv-symbols6.C
create mode 100644 gcc/testsuite/g++.target/aarch64/mv-symbols8.C
create mode 100644 gcc/testsuite/g++.target/aarch64/mv-symbols9.C
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error10.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error4.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error5.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error6.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error7.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error8.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols-error9.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols10.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols11.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols4.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols5.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols6.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols7.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols8.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mv-symbols9.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mvc-symbols1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mvc-symbols2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mvc-symbols3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/mvc-symbols4.c
diff --git a/gcc/attribs.cc b/gcc/attribs.cc
index 1d6589835a1..3eceeecd387 100644
--- a/gcc/attribs.cc
+++ b/gcc/attribs.cc
@@ -1277,17 +1277,26 @@ make_dispatcher_decl (const tree decl)
return func_decl;
}
-/* Returns true if DECL is multi-versioned using the target attribute, and this
- is the default version. This function can only be used for targets that do
- not support the "target_version" attribute. */
+/* If allow_unversioned is false, returns true if DECL has been marked as
+ multiversioned, is multi-versioned using the an attribute, and this is
+ the default version.
+ If allow_unversioned is true, then does not require the DECL is marked as
+ versioned and returns true if the function could be a default function,
+ ie could be unannotated. */
bool
-is_function_default_version (const tree decl)
+is_function_default_version (const tree decl, bool allow_unversioned)
{
if (TREE_CODE (decl) != FUNCTION_DECL
- || !DECL_FUNCTION_VERSIONED (decl))
+ || (!allow_unversioned && !DECL_FUNCTION_VERSIONED (decl)))
return false;
- tree attr = lookup_attribute ("target", DECL_ATTRIBUTES (decl));
+ tree attr
+ = lookup_attribute (TARGET_HAS_FMV_TARGET_ATTRIBUTE ? "target"
+ : "target_version",
+ DECL_ATTRIBUTES (decl));
+ /* An unannotated function can be default */
+ if (allow_unversioned && !attr)
+ return true;
gcc_assert (attr);
attr = TREE_VALUE (TREE_VALUE (attr));
return (TREE_CODE (attr) == STRING_CST
diff --git a/gcc/attribs.h b/gcc/attribs.h
index 00a83a785b4..701336d6562 100644
--- a/gcc/attribs.h
+++ b/gcc/attribs.h
@@ -56,7 +56,8 @@ extern struct scoped_attributes *
extern char *sorted_attr_string (tree);
extern bool common_function_versions (tree, tree);
extern tree make_dispatcher_decl (const tree);
-extern bool is_function_default_version (const tree);
+extern bool
+is_function_default_version (const tree, bool allow_unversioned = false);
extern void handle_ignored_attributes_option (vec<char *> *);
/* Return a type like TTYPE except that its TYPE_ATTRIBUTES
diff --git a/gcc/c-family/c-gimplify.cc b/gcc/c-family/c-gimplify.cc
index 7885e295033..bfd8f88724a 100644
--- a/gcc/c-family/c-gimplify.cc
+++ b/gcc/c-family/c-gimplify.cc
@@ -44,6 +44,7 @@ along with GCC; see the file COPYING3. If not see
#include "context.h"
#include "tree-pass.h"
#include "internal-fn.h"
+#include "calls.h"
/* The gimplification pass converts the language-dependent trees
(ld-trees) emitted by the parser into language-independent trees
@@ -905,6 +906,18 @@ c_gimplify_expr (tree *expr_p, gimple_seq *pre_p ATTRIBUTE_UNUSED,
case CALL_EXPR:
{
tree fndecl = get_callee_fndecl (*expr_p);
+
+ /* Change any calls to a multiversioned function to instead
+ * be a call to the dispatched symbol. */
+ if (fndecl && DECL_FUNCTION_VERSIONED (fndecl))
+ {
+ tree dis = cgraph_node::get (fndecl)
+ ->function_version ()
+ ->dispatcher_resolver;
+ dis = build_fold_addr_expr_loc (EXPR_LOCATION (dis), dis);
+ CALL_EXPR_FN (*expr_p) = dis;
+ }
+
if (fndecl
&& fndecl_built_in_p (fndecl, BUILT_IN_CLZG, BUILT_IN_CTZG)
&& call_expr_nargs (*expr_p) == 2
diff --git a/gcc/c/c-decl.cc b/gcc/c/c-decl.cc
index 07c18a34072..0ce0ac7a113 100644
--- a/gcc/c/c-decl.cc
+++ b/gcc/c/c-decl.cc
@@ -62,6 +62,8 @@ along with GCC; see the file COPYING3. If not see
#include "omp-general.h"
#include "omp-offload.h" /* For offload_vars. */
#include "c-parser.h"
+#include "tree.h"
+#include "calls.h"
#include "tree-pretty-print.h"
@@ -2087,6 +2089,22 @@ previous_tag (tree type)
return NULL_TREE;
}
+/* Subroutine to mark functions as versioned when using the attribute
+ 'target_version'. */
+
+static void
+maybe_mark_function_versioned (tree decl)
+{
+ if (!DECL_FUNCTION_VERSIONED (decl))
+ {
+ DECL_FUNCTION_VERSIONED (decl) = true;
+
+ tree mangled_name
+ = targetm.mangle_decl_assembler_name (decl, DECL_NAME (decl));
+ SET_DECL_ASSEMBLER_NAME (decl, mangled_name);
+ }
+}
+
/* Subroutine of duplicate_decls. Compare NEWDECL to OLDDECL.
Returns true if the caller should proceed to merge the two, false
if OLDDECL should simply be discarded. As a side effect, issues
@@ -2096,8 +2114,8 @@ previous_tag (tree type)
TREE_TYPE (NEWDECL, OLDDECL) respectively. */
static bool
-diagnose_mismatched_decls (tree newdecl, tree olddecl,
- tree *newtypep, tree *oldtypep)
+diagnose_mismatched_decls (tree newdecl, tree olddecl, tree *newtypep,
+ tree *oldtypep, bool allow_duplicates = false)
{
tree newtype, oldtype;
bool retval = true;
@@ -2407,6 +2425,21 @@ diagnose_mismatched_decls (tree newdecl, tree olddecl,
}
}
+ /* Disallow declaring a function which is already multiversioned. */
+ if ((DECL_FUNCTION_VERSIONED (olddecl)
+ || lookup_attribute (TARGET_HAS_FMV_TARGET_ATTRIBUTE
+ ? "target"
+ : "target_version",
+ DECL_ATTRIBUTES (olddecl)))
+ && current_scope != file_scope)
+ {
+ error_at (DECL_SOURCE_LOCATION (newdecl),
+ "cannot redeclare multiversioned functions at block scope");
+ inform (input_location, "previous multiversioned defintion of %q+D",
+ olddecl);
+ return false;
+ }
+
if (DECL_INITIAL (newdecl))
{
if (DECL_INITIAL (olddecl))
@@ -2414,15 +2447,17 @@ diagnose_mismatched_decls (tree newdecl, tree olddecl,
/* If the new declaration isn't overriding an extern inline
reject the new decl. In c99, no overriding is allowed
in the same translation unit. */
- if (!DECL_EXTERN_INLINE (olddecl)
- || DECL_EXTERN_INLINE (newdecl)
- || (!flag_gnu89_inline
- && (!DECL_DECLARED_INLINE_P (olddecl)
- || !lookup_attribute ("gnu_inline",
- DECL_ATTRIBUTES (olddecl)))
- && (!DECL_DECLARED_INLINE_P (newdecl)
- || !lookup_attribute ("gnu_inline",
- DECL_ATTRIBUTES (newdecl)))))
+ if (!allow_duplicates
+ && (!DECL_EXTERN_INLINE (olddecl)
+ || DECL_EXTERN_INLINE (newdecl)
+ || (!flag_gnu89_inline
+ && (!DECL_DECLARED_INLINE_P (olddecl)
+ || !lookup_attribute ("gnu_inline",
+ DECL_ATTRIBUTES (olddecl)))
+ && (!DECL_DECLARED_INLINE_P (newdecl)
+ || !lookup_attribute (
+ "gnu_inline",
+ DECL_ATTRIBUTES (newdecl))))))
{
auto_diagnostic_group d;
error ("redefinition of %q+D", newdecl);
@@ -2541,11 +2576,10 @@ diagnose_mismatched_decls (tree newdecl, tree olddecl,
definitions only if they also used static, but without saying
explicitly whether or not other cases count as
definitions at all. */
- if ((DECL_INITIAL (newdecl) && DECL_INITIAL (olddecl))
- || (flag_isoc23
- && DECL_THREAD_LOCAL_P (newdecl)
- && !DECL_EXTERNAL (newdecl)
- && !DECL_EXTERNAL (olddecl)))
+ if (!allow_duplicates
+ && ((DECL_INITIAL (newdecl) && DECL_INITIAL (olddecl))
+ || (flag_isoc23 && DECL_THREAD_LOCAL_P (newdecl)
+ && !DECL_EXTERNAL (newdecl) && !DECL_EXTERNAL (olddecl))))
{
auto_diagnostic_group d;
error ("redefinition of %q+D", newdecl);
@@ -3354,6 +3388,89 @@ pushdecl (tree x)
b->inner_comp = false;
b_use = b;
b_ext = b;
+
+ /* Check if the two functions are in a multiversioned set that we haven't
+ setup yet. If so, replace the old declaration with the dispatched
+ symbol declaration, and store both declarations in the FMV
+ structure. */
+ if (b && TREE_CODE (b->decl) == FUNCTION_DECL
+ && TREE_CODE (x) == FUNCTION_DECL && DECL_FILE_SCOPE_P (b->decl)
+ && DECL_FILE_SCOPE_P (x)
+ && !(cgraph_node::get (b->decl)
+ && cgraph_node::get (b->decl)->dispatcher_function)
+ && targetm.target_option.function_versions (x, b->decl))
+ {
+ /* Check for incompatible duplicates. */
+ if (!diagnose_mismatched_decls (x, b->decl, &TREE_TYPE (x),
+ &TREE_TYPE (b->decl), true))
+ return x;
+
+ /* Mark funcitons and start building FMV structrue. */
+ maybe_mark_function_versioned (b->decl);
+ maybe_mark_function_versioned (x);
+ cgraph_node::record_function_versions (b->decl, x);
+
+ /* Replace the declaration with the dispatched symbol declaration. */
+ tree dispatched_symbol_decl = get_c_function_version_dispatcher (x);
+ gcc_assert (dispatched_symbol_decl);
+ b->decl = dispatched_symbol_decl;
+
+ /* Return and do not record the duplicate binding. */
+ return x;
+ }
+
+ /* Check if the existing binding is a dispatched symbol and the new
+ decl is compatible with the existing nodes. In this case add the
+ existing structure to the FMV structure and merge the types of
+ the previous dispatcher declaration and this declaration. */
+ if (b && TREE_CODE (b->decl) == FUNCTION_DECL
+ && TREE_CODE (x) == FUNCTION_DECL && DECL_FILE_SCOPE_P (b->decl)
+ && DECL_FILE_SCOPE_P (x)
+ && (cgraph_node::get (b->decl)
+ && cgraph_node::get (b->decl)->dispatcher_function))
+ {
+ maybe_mark_function_versioned (x);
+
+ /* Check for incompatibility, and merge types. */
+ if (!diagnose_mismatched_decls (x, b->decl, &TREE_TYPE (x),
+ &TREE_TYPE (b->decl), true))
+ return x;
+
+ cgraph_node *dispatched_symbol_node = cgraph_node::get (b->decl);
+ gcc_assert (dispatched_symbol_node);
+
+ cgraph_function_version_info *dispatched_symbol_node_fmv
+ = dispatched_symbol_node->function_version ();
+ gcc_assert (dispatched_symbol_node_fmv);
+
+ cgraph_function_version_info *first_node
+ = dispatched_symbol_node_fmv->next;
+ gcc_assert (first_node);
+
+ cgraph_function_version_info *node = first_node;
+ cgraph_function_version_info *last_node = first_node;
+
+ /* Check if there are any multiverioned funcitons in this set with
+ the same target. In which case, merge the definitions. */
+ for (; node; node = node->next)
+ {
+ last_node = node;
+
+ if (!targetm.target_option.function_versions (
+ x, node->this_node->decl))
+ {
+ duplicate_decls (x, node->this_node->decl);
+ return node->this_node->decl;
+ }
+ }
+
+ /* Record the function into the function structure. */
+ cgraph_node::record_function_versions (last_node->this_node->decl, x);
+
+ /* Return and do not record the duplicate binding. */
+ return x;
+ }
+
/* If this is an external linkage declaration, we should check
for compatibility with the type in the external scope before
setting the type at this scope based on the visible
@@ -10827,6 +10944,15 @@ start_function (struct c_declspecs *declspecs, struct c_declarator *declarator,
warn_parm_array_mismatch (origloc, old_decl, parms);
}
+ /* Error for any nested FMV declarations. */
+ if (lookup_attribute (TARGET_HAS_FMV_TARGET_ATTRIBUTE ? "target"
+ : "target_version",
+ DECL_ATTRIBUTES (decl1))
+ && current_scope != file_scope)
+ error_at (DECL_SOURCE_LOCATION (decl1),
+ "declarations with %s attributes are only allowed at file scope",
+ TARGET_HAS_FMV_TARGET_ATTRIBUTE ? "target" : "target_version");
+
/* Record the decl so that the function name is defined.
If we already have a decl for this name, and it is a FUNCTION_DECL,
use the old decl. */
@@ -13677,6 +13803,10 @@ c_parse_final_cleanups (void)
c_write_global_declarations_1 (BLOCK_VARS (DECL_INITIAL (t)));
c_write_global_declarations_1 (BLOCK_VARS (ext_block));
+ /* Call this to set cpp_implicit_aliases_done on all nodes. This is
+ important for function multiversioning aliases to get resolved. */
+ symtab->process_same_body_aliases ();
+
if (!in_lto_p)
free_attr_access_data ();
diff --git a/gcc/calls.cc b/gcc/calls.cc
index f67067acad4..3bff6b4e947 100644
--- a/gcc/calls.cc
+++ b/gcc/calls.cc
@@ -5387,3 +5387,20 @@ cxx17_empty_base_field_p (const_tree field)
&& RECORD_OR_UNION_TYPE_P (TREE_TYPE (field))
&& !lookup_attribute ("no_unique_address", DECL_ATTRIBUTES (field)));
}
+
+/* Returns the decl of the dispatcher function if FN is a function version. */
+
+tree
+get_c_function_version_dispatcher (tree fn)
+{
+ tree dispatcher_decl = NULL;
+
+ gcc_assert (TREE_CODE (fn) == FUNCTION_DECL && DECL_FUNCTION_VERSIONED (fn));
+
+ gcc_assert (targetm.get_function_versions_dispatcher);
+ dispatcher_decl = targetm.get_function_versions_dispatcher (fn);
+
+ gcc_assert(dispatcher_decl);
+
+ return dispatcher_decl;
+}
diff --git a/gcc/calls.h b/gcc/calls.h
index 464a4e34e33..6d076f0f810 100644
--- a/gcc/calls.h
+++ b/gcc/calls.h
@@ -134,5 +134,7 @@ extern void maybe_complain_about_tail_call (tree, const char *);
extern rtx rtx_for_static_chain (const_tree, bool);
extern bool cxx17_empty_base_field_p (const_tree);
+extern tree
+get_c_function_version_dispatcher (tree fn);
#endif // GCC_CALLS_H
diff --git a/gcc/cgraph.cc b/gcc/cgraph.cc
index d4c35341ab5..82f679f107f 100644
--- a/gcc/cgraph.cc
+++ b/gcc/cgraph.cc
@@ -215,6 +215,16 @@ delete_function_version (cgraph_function_version_info *decl_v)
if (cgraph_fnver_htab != NULL)
cgraph_fnver_htab->remove_elt (decl_v);
+
+ /* If this is the first declaration in the dispatcher resolver node, update
+ it to use the next. */
+ if (decl_v->dispatcher_resolver
+ && cgraph_node::get(decl_v->dispatcher_resolver)
+ && cgraph_node::get(decl_v->dispatcher_resolver)->function_version()
+ && cgraph_node::get(decl_v->dispatcher_resolver)->function_version()
+ ->next == decl_v)
+ cgraph_node::get(decl_v->dispatcher_resolver)->function_version()->next
+ = decl_v->next;
}
/* Remove the cgraph_function_version_info and cgraph_node for DECL. This
@@ -237,12 +247,17 @@ cgraph_node::delete_function_version_by_decl (tree decl)
void
cgraph_node::record_function_versions (tree decl1, tree decl2)
{
- cgraph_node *decl1_node = cgraph_node::get_create (decl1);
- cgraph_node *decl2_node = cgraph_node::get_create (decl2);
+ cgraph_node *decl1_node;
+ cgraph_node *decl2_node;
cgraph_function_version_info *decl1_v = NULL;
cgraph_function_version_info *decl2_v = NULL;
cgraph_function_version_info *before;
cgraph_function_version_info *after;
+ cgraph_function_version_info *first_node;
+ cgraph_function_version_info *temp_node;
+
+ decl1_node = cgraph_node::get_create (decl1);
+ decl2_node = cgraph_node::get_create (decl2);
gcc_assert (decl1_node != NULL && decl2_node != NULL);
decl1_v = decl1_node->function_version ();
@@ -263,14 +278,54 @@ cgraph_node::record_function_versions (tree decl1, tree decl2)
before = decl1_v;
after = decl2_v;
+ // Go to first after node
+ while (after->prev != NULL)
+ after = after->prev;
+
+ /* Possibly reorder to make sure the default node gets to the front. */
+ if (is_function_default_version (after->this_node->decl, true))
+ {
+ temp_node = after;
+ after = before;
+ before = temp_node;
+ }
+
while (before->next != NULL)
before = before->next;
while (after->prev != NULL)
- after= after->prev;
+ after = after->prev;
before->next = after;
after->prev = before;
+
+ /* If the dispatch symbol has already been created, make sure all nodes have
+ a link to it and that is points to the correct first node. */
+ if (before->dispatcher_resolver)
+ {
+ gcc_assert (!after->dispatcher_resolver);
+ while (after)
+ {
+ after->dispatcher_resolver = before->dispatcher_resolver;
+ after = after->next;
+ }
+ }
+ else if (after->dispatcher_resolver)
+ {
+ gcc_assert (!before->dispatcher_resolver);
+
+ first_node = before;
+ while (before)
+ {
+ first_node = before;
+ before->dispatcher_resolver = after->dispatcher_resolver;
+ before = before->prev;
+ }
+
+ /* Point the dipatcher resolver to the first node. */
+ cgraph_node::get (after->dispatcher_resolver)->function_version ()->next
+ = first_node;
+ }
}
/* Initialize callgraph dump file. */
diff --git a/gcc/cgraphunit.cc b/gcc/cgraphunit.cc
index 9dbee9b6fae..63269deb3ec 100644
--- a/gcc/cgraphunit.cc
+++ b/gcc/cgraphunit.cc
@@ -661,6 +661,16 @@ cgraph_node::analyze (void)
resolve_alias (cgraph_node::get (alias_target), transparent_alias);
else if (dispatcher_function)
{
+ /* Check for the default version. */
+ if (!is_function_default_version (
+ function_version ()->next->this_node->decl, true))
+ {
+ error_at (callers->call_stmt->location,
+ "Call to multiversioned function without a default version "
+ "declaration in scope.");
+ return;
+ }
+
/* Generate the dispatcher body of multi-versioned functions. */
cgraph_function_version_info *dispatcher_version_info
= function_version ();
@@ -1281,6 +1291,19 @@ analyze_functions (bool first_time)
}
}
+
+ /* Enqueue all the functions in the FMV set to be analyzed. */
+ if (cnode->dispatcher_function)
+ {
+ cgraph_function_version_info *fvi
+ = cnode->function_version ();
+ cgraph_function_version_info *node;
+ for (node = fvi->next; node; node = node->next)
+ {
+ enqueue_node (node->this_node);
+ }
+ }
+
/* If decl is a clone of an abstract function,
mark that abstract function so that we don't release its body.
The DECL_INITIAL() of that abstract function declaration
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index f0672420aed..d5c583671d2 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -20802,34 +20802,15 @@ aarch64_get_function_versions_dispatcher (void *decl)
if (node_v->dispatcher_resolver != NULL)
return node_v->dispatcher_resolver;
- /* Find the default version and make it the first node. */
- first_v = node_v;
/* Go to the beginning of the chain. */
+ first_v = node_v;
while (first_v->prev != NULL)
first_v = first_v->prev;
- default_version_info = first_v;
- while (default_version_info != NULL)
- {
- if (get_feature_mask_for_version
- (default_version_info->this_node->decl) == 0ULL)
- break;
- default_version_info = default_version_info->next;
- }
- /* If there is no default node, just return NULL. */
- if (default_version_info == NULL)
- return NULL;
-
- /* Make default info the first node. */
- if (first_v != default_version_info)
- {
- default_version_info->prev->next = default_version_info->next;
- if (default_version_info->next)
- default_version_info->next->prev = default_version_info->prev;
- first_v->prev = default_version_info;
- default_version_info->next = first_v;
- default_version_info->prev = NULL;
- }
+ /* If there is a default version it should be at the front of the chain.
+ It is possible for the C front end call this routine before the default
+ declaration has been processed. */
+ default_version_info = first_v;
default_node = default_version_info->this_node;
diff --git a/gcc/testsuite/g++.target/aarch64/mv-symbols10.C b/gcc/testsuite/g++.target/aarch64/mv-symbols10.C
new file mode 100644
index 00000000000..8675437ea2c
--- /dev/null
+++ b/gcc/testsuite/g++.target/aarch64/mv-symbols10.C
@@ -0,0 +1,42 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+int
+foo ();
+
+int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\n_Z3foov\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n_Z3foov\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\t_Z3foov\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\t_Z3foov, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\t_Z3foov,_Z3foov\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/g++.target/aarch64/mv-symbols6.C b/gcc/testsuite/g++.target/aarch64/mv-symbols6.C
new file mode 100644
index 00000000000..a0a612e7770
--- /dev/null
+++ b/gcc/testsuite/g++.target/aarch64/mv-symbols6.C
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+/* It is not overly clear what the correct behaviour is in this case.
+ This test serves more as a test of consistency for this case rather
+ than a test of correctness. */
+
+/* { dg-final { scan-assembler-times "\n_Z3foov:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\.default:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\.resolver:\n" 0 } } */
diff --git a/gcc/testsuite/g++.target/aarch64/mv-symbols8.C b/gcc/testsuite/g++.target/aarch64/mv-symbols8.C
new file mode 100644
index 00000000000..a65ec60e2da
--- /dev/null
+++ b/gcc/testsuite/g++.target/aarch64/mv-symbols8.C
@@ -0,0 +1,47 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+__attribute__ ((target_version ("default"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\n_Z3foov\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n_Z3foov\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\t_Z3foov\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\t_Z3foov, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\t_Z3foov,_Z3foov\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/g++.target/aarch64/mv-symbols9.C b/gcc/testsuite/g++.target/aarch64/mv-symbols9.C
new file mode 100644
index 00000000000..5a5ae02971f
--- /dev/null
+++ b/gcc/testsuite/g++.target/aarch64/mv-symbols9.C
@@ -0,0 +1,44 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\n_Z3foov\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n_Z3foov\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n_Z3foov\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., _Z3foov\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\t_Z3foov\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\t_Z3foov, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\t_Z3foov,_Z3foov\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-1.c b/gcc/testsuite/gcc.target/aarch64/mv-1.c
new file mode 100644
index 00000000000..856bdd9cb08
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-1.c
@@ -0,0 +1,40 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("rng"))) int
+foo ()
+{
+ return 2;
+}
+
+__attribute__ ((target_version ("flagm"))) int
+foo ()
+{
+ return 3;
+}
+
+__attribute__ ((target_version ("rng+flagm"))) int
+foo ()
+{
+ return 4;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* Check usage of the first two FMV features, in case of off-by-one errors. */
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mrng:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MrngMflagm:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mflagm:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error1.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error1.c
new file mode 100644
index 00000000000..2501cd0060b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error1.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo ();
+
+__attribute__ ((target_version ("default"))) int
+foo () { return 1; } /* { dg-message "note: previous definition of 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) float
+foo () { return 3; } /* { dg-error "conflicting types for 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error10.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error10.c
new file mode 100644
index 00000000000..b61aca172cb
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error10.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo (int a, int (*b)[4]) { return 1; }
+
+__attribute__ ((target_version ("sve"))) int
+foo (int a, int (*b)[4]) { return 1; } /* { dg-message "note: previous definition of" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo (int a, int (*b)[5]) { return 3; } /* { dg-error "conflicting types for 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error2.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error2.c
new file mode 100644
index 00000000000..537f9a682c2
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error2.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) float
+foo () { return 3; } /* { dg-message "note: previous definition of 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) float
+foo () { return 3; } /* { dg-error "redefinition of 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error3.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error3.c
new file mode 100644
index 00000000000..733c965d9a7
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error3.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo (int a, int b) { return 1; } /* { dg-message "note: previous definition of 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo (int a, float c) { return 3; } /* { dg-error "conflicting types for 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error4.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error4.c
new file mode 100644
index 00000000000..9b1b7cf2be1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error4.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo (int a, int b); /* { dg-message "note: previous declaration of 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo (int a, float c) { return 3; } /* { dg-error "conflicting types for 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error5.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error5.c
new file mode 100644
index 00000000000..70311cf0154
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error5.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo () { return 1; } /* { dg-message "note: previous definition of 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) float
+foo (); /* { dg-error "conflicting types for 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) float
+foo () { return 3.0; } /* { dg-error "conflicting types for 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error6.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error6.c
new file mode 100644
index 00000000000..908cd9f6490
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error6.c
@@ -0,0 +1,8 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo (int a, int (*b)[4]) { return 1; } /* { dg-message "note: previous definition of 'foo'" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo (int a, int (*b)[5]) { return 3; } /* { dg-error "conflicting types for 'foo'" } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error7.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error7.c
new file mode 100644
index 00000000000..19e62b4b47e
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error7.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo (int a, int (*b)[4]) { return 1; }
+
+int bar(void) {
+ __attribute__ ((target_version ("dotprod"))) int
+ foo (int a, int (*b)[5]) { return 3; } /* { dg-error "declarations with target_version attributes are only allowed at file scope" } */
+
+ return 1;
+}
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error8.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error8.c
new file mode 100644
index 00000000000..e87744bb685
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error8.c
@@ -0,0 +1,11 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo (int a, int (*b)[4]) { return 1; } /* { dg-message "note: previous multiversioned defintion of 'foo'" } */
+
+int bar(void) {
+ int foo (int a, int (*b)[4]); /* { dg-error "cannot redeclare multiversioned functions at block scope" } */
+
+ return 1;
+}
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols-error9.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error9.c
new file mode 100644
index 00000000000..a2c396240b3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols-error9.c
@@ -0,0 +1,10 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+int bar () { return foo (); } /* { dg-error "Call to multiversioned function without a default version declaration in scope." } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols1.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols1.c
new file mode 100644
index 00000000000..798227826e5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols1.c
@@ -0,0 +1,38 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+// Basic case of fmv correctness with all functions and use in one TU.
+
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* When updating any of the symbol names in these tests, make sure to also
+ update any tests for their absence in mv-symbolsN.C */
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols10.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols10.c
new file mode 100644
index 00000000000..d5256389d7b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols10.c
@@ -0,0 +1,42 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+int
+foo ();
+
+int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols11.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols11.c
new file mode 100644
index 00000000000..274640fc21f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols11.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+// Check that types can be combined
+
+__attribute__ ((target_version ("default"))) int
+foo (int a, int (*b)[4]) { return 1; }
+
+__attribute__ ((target_version ("dotprod"))) int
+foo (int a, int (*b)[]) { return 3; }
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 0 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols2.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols2.c
new file mode 100644
index 00000000000..2e59ea9fb86
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols2.c
@@ -0,0 +1,28 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+// FMV correctness with definitions but no call
+
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 0 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols3.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols3.c
new file mode 100644
index 00000000000..29bf17cac95
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols3.c
@@ -0,0 +1,27 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+// FMV correctness with declarations but no implementation
+
+__attribute__ ((target_version ("default"))) int
+foo ();
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols4.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols4.c
new file mode 100644
index 00000000000..a476800b2c5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols4.c
@@ -0,0 +1,31 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+// FMV correctness with a default implementation and declarations of other
+// versions
+
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols5.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols5.c
new file mode 100644
index 00000000000..344d1e6fe34
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols5.c
@@ -0,0 +1,36 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+// FMV correctness with default declaration, and implementations of other
+// versions.
+
+__attribute__ ((target_version ("default"))) int
+foo ();
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* When updating any of the symbol names in these tests, make sure to also
+ update any tests for their absence in mvc-symbolsN.C */
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols6.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols6.c
new file mode 100644
index 00000000000..97e552689fc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols6.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+/* It is not overly clear what the correct behaviour is in this case.
+ This test serves more as a test of consistency for this case rather
+ than a test of correctness. */
+
+/* { dg-final { scan-assembler-times "\nfoo:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 0 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols7.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols7.c
new file mode 100644
index 00000000000..0360de9aafc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols7.c
@@ -0,0 +1,47 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+
+int
+bar ()
+{
+ return foo ();
+}
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+__attribute__ ((target_version ("default"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols8.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols8.c
new file mode 100644
index 00000000000..3e3eaf21aa9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols8.c
@@ -0,0 +1,47 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+__attribute__ ((target_version ("default"))) int
+foo ();
+
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("default"))) int
+foo ()
+{
+ return 1;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mv-symbols9.c b/gcc/testsuite/gcc.target/aarch64/mv-symbols9.c
new file mode 100644
index 00000000000..8e0864f1663
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mv-symbols9.c
@@ -0,0 +1,44 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ();
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ();
+
+int
+foo ()
+{
+ return 1;
+}
+
+__attribute__ ((target_version ("dotprod"))) int
+foo ()
+{
+ return 3;
+}
+__attribute__ ((target_version ("sve+sve2"))) int
+foo ()
+{
+ return 5;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._MsveMsve2\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\._Mdotprod\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tadrp\tx., foo\.default\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mvc-symbols1.c b/gcc/testsuite/gcc.target/aarch64/mvc-symbols1.c
new file mode 100644
index 00000000000..3ad15e5bb73
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mvc-symbols1.c
@@ -0,0 +1,25 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_clones ("default", "dotprod", "sve+sve2"))) int
+foo ()
+{
+ return 1;
+}
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* When updating any of the symbol names in these tests, make sure to also
+ update any tests for their absence in mvc-symbolsN.C */
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mvc-symbols2.c b/gcc/testsuite/gcc.target/aarch64/mvc-symbols2.c
new file mode 100644
index 00000000000..78385ed904b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mvc-symbols2.c
@@ -0,0 +1,15 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_clones ("default", "dotprod", "sve+sve2"))) int
+foo ()
+{
+ return 1;
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mvc-symbols3.c b/gcc/testsuite/gcc.target/aarch64/mvc-symbols3.c
new file mode 100644
index 00000000000..d24293e8660
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mvc-symbols3.c
@@ -0,0 +1,19 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_clones ("default", "dotprod", "sve+sve2"))) int
+foo ();
+
+int
+bar ()
+{
+ return foo ();
+}
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\tbl\tfoo\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 1 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/mvc-symbols4.c b/gcc/testsuite/gcc.target/aarch64/mvc-symbols4.c
new file mode 100644
index 00000000000..abaf60f91c3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/mvc-symbols4.c
@@ -0,0 +1,12 @@
+/* { dg-do compile } */
+/* { dg-options "-O0" } */
+
+__attribute__ ((target_clones ("default", "dotprod", "sve+sve2"))) int
+foo ();
+
+/* { dg-final { scan-assembler-times "\nfoo\.default:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._Mdotprod:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\._MsveMsve2:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\nfoo\.resolver:\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n\t\.type\tfoo, %gnu_indirect_function\n" 0 } } */
+/* { dg-final { scan-assembler-times "\n\t\.set\tfoo,foo\.resolver\n" 0 } } */
