Hi, currently, an instance of cgraph_indirect_call_info is allocated for each indirect call and it contains fields which are only usable for some kinds of calls. The most special are polymorphic calls representing calls of virtual methods through an OBJ_TYPE_REF and which need the ipa_polymorphic_context structure, among other data from the O_T_R itself for devirtualization, which are not needed for other types of calls.
This patch splits the class into three. A common base which is also used for calls which we know we cannot do anything about (when the call statement is not known or when it is a constant integer or something like that), a (simple) type for calls of SSA_NAMEs and a type for polymorphic calls. This means we no longer allocate memory for members which we do not need in a particular situation. Part of the motivation to write this is also that I have a patch adding fields to the simple variant and this reorganization makes it doable in a clean fashion. The base class retains the param_index field even though it really only is meaningful in the derived classes. This made conversion of some of the functions operating on it easier (and I expect that vast majority of actual instances will fall into one of the two categories anyway). We already stream information stored in indirect information at two places, once in call graph streaming and once in ipa-prop streaming. I am not sure what the reason is (call graph streaming cannot do trees so we need the ipa-prop one) but this patch preserves this. The patch also removes a part of the comment in ipa_make_edge_direct_to_target which said the check for member_ptr was not necessary. We even have a test in our testsuite showing it is and we produce wrong code without it. Bootsstrapped and LTO-bootstrapped and tested on x86_64-linux. OK for master? thanks, Martin gcc/ChangeLog: 2025-10-17 Martin Jambor <[email protected]> * cgraph.h (cgraph_node): Adjust the comment of member function create_indirect_edge. (enum cgraph_indirect_info_kind): New. (cgraph_indirect_call_info): Convert into a base class. (cgraph_simple_indirect_info): New. (cgraph_polymorphic_indirect_info): Likewise. (usable_polymorphic_info_p): Likewise. (is_a_helper <cgraph_simple_indirect_info *>::test): Likewise. (is_a_helper <cgraph_polymorphic_indirect_info *>::test): Likewise. (cgraph_allocate_init_indirect_info): Remove declaration. (ipa_polymorphic_call_context::ipa_polymorphic_call_context): Use the appropriate derived type of indirect info. * cgraph.cc (cgraph_allocate_init_indirect_info): Removed. (cgraph_node::create_indirect_edge): Create an appropriate type of indirect_info. (cgraph_node::dump): Dump indirect info using its dump function. (cgraph_indirect_call_info::dump): New function. (cgraph_indirect_call_info::debug): Likewise. * cgraphclones.cc (cgraph_edge::clone): Create an appropriate type of indirect_info. * cgraphunit.cc (analyze_functions): Use the appropriate derived type of indirect info. * ipa-cp.cc (initialize_node_lattices): Adjust the check for polymorphic indirect info. (ipa_get_indirect_edge_target_1): Use the appropriate derived types of indirect info. (ipcp_discover_new_direct_edges): Likewise. * ipa-devirt.cc (ipa_devirt): Use the polymorphis derived type of indirect info and check that it is usable. * ipa-inline.cc (dump_inline_stats): Adjust the check for polymorphic indirect info. * ipa-profile.cc (ipa_profile): Likewise and check usability. * ipa-prop.cc (ipa_print_node_jump_functions): Dump indirect info using its dumping member function. (ipa_note_param_call): Removed. (ipa_analyze_indirect_call_uses): Use the appropriate derived type of indirect info, set all fields of indirect info separately rather than relying on ipa_note_param_call. (ipa_analyze_virtual_call_uses): Use the polymorphis derived type of indirect info and check that it is usable, set all fields of indirect info separately rather than relying on ipa_note_param_call. (ipa_analyze_call_uses): Use the appropriate derived type of indirect info. (ipa_make_edge_direct_to_target): Use the appropriate derived type of indirect info. Remove wrong note that member_ptr check was not needed. Adjust check for polymorphic call when dumping. (try_make_edge_direct_simple_call): Use the appropriate derived type of indirect info. (try_make_edge_direct_virtual_call): Use the polymorphis derived type of indirect info and check that it is usable. (update_indirect_edges_after_inlining): Use the appropriate derived type of indirect info. Define local variables only before their first use. (ipa_write_indirect_edge_info): Also stream indirect info kind. Use the appropriate derived type of indirect info. (ipa_read_indirect_edge_info): Check that the streamed in indirect info kind matches rthe structure at hand. Use the appropriate derived type of indirect info. * ipa-utils.h (possible_polymorphic_call_targets): Use the polymorphis derived type of indirect info. Assert it is usable. (dump_possible_polymorphic_call_targets): Use the polymorphis derived type of indirect info and check it is usable. (possible_polymorphic_call_target_p): Likewise. * ipa.cc (symbol_table::remove_unreachable_nodes): Use usable_polymorphic_info_p. * lto-cgraph.cc (lto_output_edge): Stream indirect info kind. (compute_ltrans_boundary): Use usable_polymorphic_info_p. (input_edge): Move definition of ecf_flags before its first use. Pass true as the last parameter to create_indirect_edge. Stream indirect info kind and create a corresponding type to hold the information. * trans-mem.cc (ipa_tm_insert_gettmclone_call): Use the polymorphis derived type of indirect info. --- gcc/cgraph.cc | 145 +++++++++------- gcc/cgraph.h | 191 ++++++++++++++++++--- gcc/cgraphclones.cc | 17 +- gcc/cgraphunit.cc | 3 +- gcc/ipa-cp.cc | 94 +++++------ gcc/ipa-devirt.cc | 12 +- gcc/ipa-inline.cc | 2 +- gcc/ipa-profile.cc | 2 +- gcc/ipa-prop.cc | 403 +++++++++++++++++++++++--------------------- gcc/ipa-utils.h | 23 ++- gcc/ipa.cc | 2 +- gcc/lto-cgraph.cc | 24 ++- gcc/trans-mem.cc | 10 +- 13 files changed, 573 insertions(+), 355 deletions(-) diff --git a/gcc/cgraph.cc b/gcc/cgraph.cc index d1b2e2a162c..c640dd106c9 100644 --- a/gcc/cgraph.cc +++ b/gcc/cgraph.cc @@ -1161,56 +1161,44 @@ cgraph_node::create_edge (cgraph_node *callee, return edge; } -/* Allocate cgraph_indirect_call_info and set its fields to default values. */ - -cgraph_indirect_call_info * -cgraph_allocate_init_indirect_info (void) -{ - cgraph_indirect_call_info *ii; - - ii = ggc_cleared_alloc<cgraph_indirect_call_info> (); - ii->param_index = -1; - return ii; -} - -/* Create an indirect edge with a yet-undetermined callee where the call - statement destination is a formal parameter of the caller with index - PARAM_INDEX. CLONING_P should be set if properties that are copied from an - original edge should not be calculated and indirect_info structure should - not be calculated. */ +/* Create an indirect edge to a (yet-)undetermined callee. CALL_STMT is the + corresponding statement, if available, ECF_FLAGS and COUNT are corresponding + gimple call flags and profiling count respectively. CLONING_P should be set + if properties that are copied from an original edge should not be + calculated. */ cgraph_edge * cgraph_node::create_indirect_edge (gcall *call_stmt, int ecf_flags, - profile_count count, - bool cloning_p) + profile_count count, bool cloning_p) { cgraph_edge *edge = symtab->create_edge (this, NULL, call_stmt, count, true, cloning_p); - tree target; if (!cloning_p) - initialize_inline_failed (edge); - - edge->indirect_info = cgraph_allocate_init_indirect_info (); - edge->indirect_info->ecf_flags = ecf_flags; - edge->indirect_info->vptr_changed = true; - - /* Record polymorphic call info. */ - if (!cloning_p - && call_stmt - && (target = gimple_call_fn (call_stmt)) - && virtual_method_call_p (target)) { - ipa_polymorphic_call_context context (decl, target, call_stmt); + initialize_inline_failed (edge); - /* Only record types can have virtual calls. */ - edge->indirect_info->polymorphic = true; - edge->indirect_info->param_index = -1; - edge->indirect_info->otr_token - = tree_to_uhwi (OBJ_TYPE_REF_TOKEN (target)); - edge->indirect_info->otr_type = obj_type_ref_class (target); - gcc_assert (TREE_CODE (edge->indirect_info->otr_type) == RECORD_TYPE); - edge->indirect_info->context = context; + tree target = NULL_TREE; + if (call_stmt) + target = gimple_call_fn (call_stmt); + if (target && virtual_method_call_p (target)) + { + ipa_polymorphic_call_context context (decl, target, call_stmt); + HOST_WIDE_INT token = tree_to_shwi (OBJ_TYPE_REF_TOKEN (target)); + tree type = obj_type_ref_class (target); + edge->indirect_info + = (new (ggc_alloc<cgraph_polymorphic_indirect_info> ()) + cgraph_polymorphic_indirect_info (ecf_flags, context, token, + type)); + } + else if (target && TREE_CODE (target) == SSA_NAME) + edge->indirect_info + = (new (ggc_alloc<cgraph_simple_indirect_info> ()) + cgraph_simple_indirect_info (ecf_flags)); + else + edge->indirect_info + = (new (ggc_alloc<cgraph_indirect_call_info> ()) + cgraph_indirect_call_info(CIIK_UNSPECIFIED, ecf_flags)); } edge->next_callee = indirect_calls; @@ -2668,30 +2656,8 @@ cgraph_node::dump (FILE *f) for (edge = indirect_calls; edge; edge = edge->next_callee) { - if (edge->indirect_info->polymorphic) - { - fprintf (f, " Polymorphic indirect call of type "); - print_generic_expr (f, edge->indirect_info->otr_type, TDF_SLIM); - fprintf (f, " token:%i", (int) edge->indirect_info->otr_token); - } - else - fprintf (f, " Indirect call"); - edge->dump_edge_flags (f); - if (edge->indirect_info->param_index != -1) - { - fprintf (f, "of param:%i ", edge->indirect_info->param_index); - if (edge->indirect_info->agg_contents) - fprintf (f, "loaded from %s %s at offset %i ", - edge->indirect_info->member_ptr ? "member ptr" : "aggregate", - edge->indirect_info->by_ref ? "passed by reference" : "", - (int)edge->indirect_info->offset); - if (edge->indirect_info->vptr_changed) - fprintf (f, "(vptr maybe changed) "); - } - fprintf (f, "num speculative call targets: %i\n", - edge->indirect_info->num_speculative_call_targets); - if (edge->indirect_info->polymorphic) - edge->indirect_info->context.dump (f); + fprintf (f, " "); + edge->indirect_info->dump (f); } } @@ -2731,6 +2697,57 @@ cgraph_node::dump_cgraph (FILE *f) node->dump (f); } +/* Dump human readable information about the indirect call to F. If NEWLINE + is true, it will be terminated by a newline. */ + +void +cgraph_indirect_call_info::dump (FILE *f, bool newline) const +{ + if (const cgraph_polymorphic_indirect_info *pii + = dyn_cast <const cgraph_polymorphic_indirect_info *> (this)) + { + fprintf (f, " indirect polymorphic callsite, %s, " + "calling param %i, offset " HOST_WIDE_INT_PRINT_DEC + "otr_token " HOST_WIDE_INT_PRINT_DEC ", otr_type ", + pii->vptr_changed ? "vptr_changed" : "vptr not changed", + pii->param_index, pii->offset, pii->otr_token); + print_generic_expr (f, pii->otr_type); + fprintf (f, ", context "); + pii->context.dump (f, false); + } + else if (const cgraph_simple_indirect_info *sii + = dyn_cast <const cgraph_simple_indirect_info *> (this)) + { + if (sii->agg_contents) + fprintf (f, " indirect %s callsite, calling param %i, " + "offset " HOST_WIDE_INT_PRINT_DEC ", %s", + sii->member_ptr ? "member ptr" : "aggregate", + sii->param_index, sii->offset, + sii->by_ref ? "by reference" : "by_value"); + else if (sii->param_index >= 0) + fprintf (f, " indirect simple callsite, calling param %i", + sii->param_index); + else + fprintf (f, " indirect simple callsite, not calling a known " + "parameter"); + } + else + fprintf (f, " indirect callsite"); + + fprintf (f, ", flags %i, num speculative call targets: %i", ecf_flags, + num_speculative_call_targets); + if (newline) + fprintf (f, "\n"); +} + +/* Dump human readable information about the indirect call to stderr. */ + +void +cgraph_indirect_call_info::debug () const +{ + dump (stderr); +} + /* Return true when the DECL can possibly be inlined. */ bool diff --git a/gcc/cgraph.h b/gcc/cgraph.h index 069e007ab71..c1329015342 100644 --- a/gcc/cgraph.h +++ b/gcc/cgraph.h @@ -1152,9 +1152,12 @@ struct GTY((tag ("SYMTAB_FUNCTION"))) cgraph_node : public symtab_node gcall *call_stmt, profile_count count, bool cloning_p = false); - /* Create an indirect edge with a yet-undetermined callee where the call - statement destination is a formal parameter of the caller with index - PARAM_INDEX. */ + /* Create an indirect edge to a (yet-)undetermined callee. CALL_STMT is the + corresponding statement, if available, ECF_FLAGS and COUNT are + corresponding gimple call flags and profiling count respectively. + CLONING_P should be set if properties that are copied from an original + edge should not be calculated. */ + cgraph_edge *create_indirect_edge (gcall *call_stmt, int ecf_flags, profile_count count, bool cloning_p = false); @@ -1685,31 +1688,72 @@ private: void make_speculative (tree otr_type = NULL); }; -/* Structure containing additional information about an indirect call. */ +/* Denotes the kind of call that a particular cgraph_indirect_call_info + instance describes. */ + +enum cgraph_indirect_info_kind { + /* Unspecified kind. Only to be used when no information about the call + statement is available or it does not fall into any of the other + categories. */ + CIIK_UNSPECIFIED, + /* A normal indirect call when the target is an SSA_NAME. */ + CIIK_SIMPLE, + /* Call of a virtual method when the target is an OBJ_TYPE_REF which conforms + to virtual_method_call_p. */ + CIIK_POLYMORPHIC, + /* Must be last */ + CIIK_N_KINDS +}; + +/* The base class containing additional information about all kinds of indirect + calls. It can also be used when no information about the call statement is + available or it does not fall into any of the other categories. */ -class GTY(()) cgraph_indirect_call_info +class GTY((desc ("%h.kind"), tag ("CIIK_UNSPECIFIED"))) + cgraph_indirect_call_info { public: - /* When agg_content is set, an offset where the call pointer is located - within the aggregate. */ - HOST_WIDE_INT offset; - /* Context of the polymorphic call; use only when POLYMORPHIC flag is set. */ - ipa_polymorphic_call_context context; - /* OBJ_TYPE_REF_TOKEN of a polymorphic call (if polymorphic is set). */ - HOST_WIDE_INT otr_token; - /* Type of the object from OBJ_TYPE_REF_OBJECT. */ - tree otr_type; - /* Index of the parameter that is called. */ - int param_index; + cgraph_indirect_call_info (int flags) + : ecf_flags (flags), param_index (-1), kind (CIIK_UNSPECIFIED), + num_speculative_call_targets (0) {} + cgraph_indirect_call_info (enum cgraph_indirect_info_kind k, int flags) + : ecf_flags (flags), param_index (-1), kind (k), + num_speculative_call_targets (0) {} + + /* Dump human readable information about the indirect call to F. If NEWLINE + is true, it will be terminated by a newline. */ + void dump (FILE *f, bool newline = true) const; + void DEBUG_FUNCTION debug () const; + /* ECF flags determined from the caller. */ int ecf_flags; + /* If we can relate this call target to a specific formal parameter of the + caller, then this is its index. Otherwise set to -1. */ + int param_index; - /* Number of speculative call targets, it's less than GCOV_TOPN_VALUES. */ + /* Identifier of the specific type of indirect info this actually is. */ + enum cgraph_indirect_info_kind kind : 2; + /* Number of speculative call targets. */ unsigned num_speculative_call_targets : 16; +}; + +/* Structure containing additional information about non-virtual indirect calls + where the target is an SSA_NAME. */ + +class GTY((tag ("CIIK_SIMPLE"))) + cgraph_simple_indirect_info : public cgraph_indirect_call_info +{ +public: + cgraph_simple_indirect_info (int flags) + : cgraph_indirect_call_info (CIIK_SIMPLE, flags), offset (0), + agg_contents (false), member_ptr (false), by_ref (false), + guaranteed_unmodified (false) + {} + + /* When agg_content is set, an offset where the call pointer is located + within the aggregate. */ + HOST_WIDE_INT offset; - /* Set when the call is a virtual call with the parameter being the - associated object pointer rather than a simple direct call. */ - unsigned polymorphic : 1; /* Set when the call is a call of a pointer loaded from contents of an aggregate at offset. */ unsigned agg_contents : 1; @@ -1723,11 +1767,69 @@ public: never modified between the invocation of the function and the load point. */ unsigned guaranteed_unmodified : 1; +}; + +/* Structure containing additional information about non-virtual indirect calls + when the target is an OBJ_TYPE_REF which conforms to + virtual_method_call_p. */ + +class GTY((tag ("CIIK_POLYMORPHIC"))) + cgraph_polymorphic_indirect_info : public cgraph_indirect_call_info +{ +public: + cgraph_polymorphic_indirect_info (int flags) + : cgraph_indirect_call_info (CIIK_POLYMORPHIC, flags), context (), + otr_token (0), otr_type (nullptr), offset (0), vptr_changed (true) + {} + cgraph_polymorphic_indirect_info (int flags, + const ipa_polymorphic_call_context &ctx, + HOST_WIDE_INT token, tree type) + : cgraph_indirect_call_info (CIIK_POLYMORPHIC, flags), context (ctx), + otr_token (token), otr_type (type), offset (0), vptr_changed (true) + {} + + /* Return true if the information is usable for devirtualization. This can + happen if part of the required information is not streamed in yet and for + some cases we determine it is no longer useful to attempt to use the + information too. */ + bool usable_p () const + { + return !!otr_type; + } + /* Mark this information as not useful for devirtualization. Return true if + it was considered useful until now. */ + bool mark_unusable () + { + bool r = !!otr_type; + otr_type = NULL_TREE; + return r; + } + + /* Context of the polymorphic call; use only when POLYMORPHIC flag is set. */ + ipa_polymorphic_call_context context; + /* OBJ_TYPE_REF_TOKEN of a polymorphic call (if polymorphic is set). */ + HOST_WIDE_INT otr_token; + /* Type of the object from OBJ_TYPE_REF_OBJECT. */ + tree otr_type; + /* The offset from the point where the parameter identified by param_index to + the point where the corresponding object appears. */ + HOST_WIDE_INT offset; + /* For polymorphic calls this specify whether the virtual table pointer may have changed in between function entry and the call. */ unsigned vptr_changed : 1; }; +/* Return true if ii is a cgraph_polymorphic_indirect_info that is usable_p. */ + +inline bool +usable_polymorphic_info_p (cgraph_indirect_call_info *ii) +{ + cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (ii); + return pii && pii->usable_p (); +} + class GTY((chain_next ("%h.next_caller"), chain_prev ("%h.prev_caller"), for_user)) cgraph_edge { @@ -2253,6 +2355,49 @@ is_a_helper <varpool_node *>::test (symtab_node *p) return p && p->type == SYMTAB_VARIABLE; } +/* Report whether or not THIS indirect info is a known simple one. */ + +template <> +template <> +inline bool +is_a_helper <cgraph_simple_indirect_info *>::test (cgraph_indirect_call_info *p) +{ + return p && p->kind == CIIK_SIMPLE; +} + +/* Likewise, but const qualified. */ + +template <> +template <> +inline bool +is_a_helper <const cgraph_simple_indirect_info *> +::test (const cgraph_indirect_call_info *p) +{ + return p && p->kind == CIIK_SIMPLE; +} + +/* Report whether or not THIS indirect info is a known polymorphic one. */ + +template <> +template <> +inline bool +is_a_helper <cgraph_polymorphic_indirect_info *> +::test (cgraph_indirect_call_info *p) +{ + return p && p->kind == CIIK_POLYMORPHIC; +} + +/* Likewise, but const qualified. */ + +template <> +template <> +inline bool +is_a_helper <const cgraph_polymorphic_indirect_info *> +::test (const cgraph_indirect_call_info *p) +{ + return p && p->kind == CIIK_POLYMORPHIC; +} + typedef void (*cgraph_edge_hook)(cgraph_edge *, void *); typedef void (*cgraph_node_hook)(cgraph_node *, void *); typedef void (*varpool_node_hook)(varpool_node *, void *); @@ -2676,7 +2821,6 @@ asmname_hasher::equal (symtab_node *n, const_tree t) /* In cgraph.cc */ void cgraph_cc_finalize (void); void release_function_body (tree); -cgraph_indirect_call_info *cgraph_allocate_init_indirect_info (void); void cgraph_update_edges_for_call_stmt (gimple *, tree, gimple *); bool cgraph_function_possibly_inlined_p (tree); @@ -3575,8 +3719,9 @@ ipa_ref::address_matters_p () inline ipa_polymorphic_call_context::ipa_polymorphic_call_context (cgraph_edge *e) { - gcc_checking_assert (e->indirect_info->polymorphic); - *this = e->indirect_info->context; + cgraph_polymorphic_indirect_info *pii + = as_a <cgraph_polymorphic_indirect_info *> (e->indirect_info); + *this = pii->context; } /* Build empty "I know nothing" context. */ diff --git a/gcc/cgraphclones.cc b/gcc/cgraphclones.cc index 49f0e58fa1e..3f85c15df6e 100644 --- a/gcc/cgraphclones.cc +++ b/gcc/cgraphclones.cc @@ -121,7 +121,22 @@ cgraph_edge::clone (cgraph_node *n, gcall *call_stmt, unsigned stmt_uid, new_edge = n->create_indirect_edge (call_stmt, indirect_info->ecf_flags, prof_count, true); - *new_edge->indirect_info = *indirect_info; + + if (indirect_info->kind == CIIK_POLYMORPHIC) + new_edge->indirect_info + = (new (ggc_alloc<cgraph_polymorphic_indirect_info> ()) + cgraph_polymorphic_indirect_info ( + *(const cgraph_polymorphic_indirect_info *) indirect_info)); + else if (indirect_info->kind == CIIK_SIMPLE) + new_edge->indirect_info + = (new (ggc_alloc<cgraph_simple_indirect_info> ()) + cgraph_simple_indirect_info ( + *(const cgraph_simple_indirect_info *) indirect_info)); + else + new_edge->indirect_info + = (new (ggc_alloc<cgraph_indirect_call_info> ()) + cgraph_indirect_call_info( + *(const cgraph_indirect_call_info *) indirect_info)); } } else diff --git a/gcc/cgraphunit.cc b/gcc/cgraphunit.cc index a81f685654f..cc94a144f2a 100644 --- a/gcc/cgraphunit.cc +++ b/gcc/cgraphunit.cc @@ -1297,7 +1297,8 @@ analyze_functions (bool first_time) for (edge = cnode->indirect_calls; edge; edge = next) { next = edge->next_callee; - if (edge->indirect_info->polymorphic) + if (is_a <cgraph_polymorphic_indirect_info *> + (edge->indirect_info)) walk_polymorphic_call_targets (&reachable_call_targets, edge); } diff --git a/gcc/ipa-cp.cc b/gcc/ipa-cp.cc index 2105c9a2ef7..74ec8a7c4b2 100644 --- a/gcc/ipa-cp.cc +++ b/gcc/ipa-cp.cc @@ -1482,13 +1482,10 @@ initialize_node_lattices (struct cgraph_node *node) } for (ie = node->indirect_calls; ie; ie = ie->next_callee) - if (ie->indirect_info->polymorphic - && ie->indirect_info->param_index >= 0) - { - gcc_checking_assert (ie->indirect_info->param_index >= 0); - ipa_get_parm_lattices (info, - ie->indirect_info->param_index)->virt_call = 1; - } + if (ie->indirect_info->param_index >= 0 + && is_a <cgraph_polymorphic_indirect_info *> (ie->indirect_info)) + ipa_get_parm_lattices (info, + ie->indirect_info->param_index)->virt_call = 1; } /* Return VALUE if it is NULL_TREE or if it can be directly safely IPA-CP @@ -3106,32 +3103,28 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, bool *speculative) { int param_index = ie->indirect_info->param_index; - HOST_WIDE_INT anc_offset; - tree t = NULL; - tree target = NULL; - *speculative = false; if (param_index == -1) return NULL_TREE; - if (!ie->indirect_info->polymorphic) + if (cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info)) { tree t = NULL; - if (ie->indirect_info->agg_contents) + if (sii->agg_contents) { t = NULL; if ((unsigned) param_index < known_csts.length () && known_csts[param_index]) t = ipa_find_agg_cst_from_init (known_csts[param_index], - ie->indirect_info->offset, - ie->indirect_info->by_ref); + sii->offset, + sii->by_ref); - if (!t && ie->indirect_info->guaranteed_unmodified) - t = avs.get_value (param_index, - ie->indirect_info->offset / BITS_PER_UNIT, - ie->indirect_info->by_ref); + if (!t && sii->guaranteed_unmodified) + t = avs.get_value (param_index, sii->offset / BITS_PER_UNIT, + sii->by_ref); } else if ((unsigned) param_index < known_csts.length ()) t = known_csts[param_index]; @@ -3147,23 +3140,22 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, if (!opt_for_fn (ie->caller->decl, flag_devirtualize)) return NULL_TREE; - gcc_assert (!ie->indirect_info->agg_contents); - gcc_assert (!ie->indirect_info->by_ref); - anc_offset = ie->indirect_info->offset; - - t = NULL; + cgraph_polymorphic_indirect_info *pii + = as_a <cgraph_polymorphic_indirect_info *> (ie->indirect_info); + if (!pii->usable_p ()) + return NULL_TREE; + HOST_WIDE_INT anc_offset = pii->offset; + tree t = NULL; + tree target = NULL; if ((unsigned) param_index < known_csts.length () && known_csts[param_index]) - t = ipa_find_agg_cst_from_init (known_csts[param_index], - ie->indirect_info->offset, true); + t = ipa_find_agg_cst_from_init (known_csts[param_index], anc_offset, true); /* Try to work out value of virtual table pointer value in replacements. */ /* or known aggregate values. */ if (!t) - t = avs.get_value (param_index, - ie->indirect_info->offset / BITS_PER_UNIT, - true); + t = avs.get_value (param_index, anc_offset / BITS_PER_UNIT, true); /* If we found the virtual table pointer, lookup the target. */ if (t) @@ -3173,8 +3165,8 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, if (vtable_pointer_value_to_vtable (t, &vtable, &offset)) { bool can_refer; - target = gimple_get_virt_method_for_vtable (ie->indirect_info->otr_token, - vtable, offset, &can_refer); + target = gimple_get_virt_method_for_vtable (pii->otr_token, vtable, + offset, &can_refer); if (can_refer) { if (!target @@ -3183,11 +3175,11 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, (ie, cgraph_node::get (target))) { /* Do not speculate builtin_unreachable, it is stupid! */ - if (ie->indirect_info->vptr_changed) + if (pii->vptr_changed) return NULL; target = ipa_impossible_devirt_target (ie, target); } - *speculative = ie->indirect_info->vptr_changed; + *speculative = pii->vptr_changed; if (!*speculative) return target; } @@ -3198,31 +3190,28 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, if (!t && (unsigned) param_index < known_csts.length ()) t = known_csts[param_index]; - gcc_checking_assert (!t || TREE_CODE (t) != TREE_BINFO); - ipa_polymorphic_call_context context; if (known_contexts.length () > (unsigned int) param_index) { context = known_contexts[param_index]; context.offset_by (anc_offset); - if (ie->indirect_info->vptr_changed) + if (pii->vptr_changed) context.possible_dynamic_type_change (ie->in_polymorphic_cdtor, - ie->indirect_info->otr_type); + pii->otr_type); if (t) { - ipa_polymorphic_call_context ctx2 = ipa_polymorphic_call_context - (t, ie->indirect_info->otr_type, anc_offset); + ipa_polymorphic_call_context ctx2 + = ipa_polymorphic_call_context (t, pii->otr_type, anc_offset); if (!ctx2.useless_p ()) - context.combine_with (ctx2, ie->indirect_info->otr_type); + context.combine_with (ctx2, pii->otr_type); } } else if (t) { - context = ipa_polymorphic_call_context (t, ie->indirect_info->otr_type, - anc_offset); - if (ie->indirect_info->vptr_changed) + context = ipa_polymorphic_call_context (t, pii->otr_type, anc_offset); + if (pii->vptr_changed) context.possible_dynamic_type_change (ie->in_polymorphic_cdtor, - ie->indirect_info->otr_type); + pii->otr_type); } else return NULL_TREE; @@ -3230,10 +3219,8 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, vec <cgraph_node *>targets; bool final; - targets = possible_polymorphic_call_targets - (ie->indirect_info->otr_type, - ie->indirect_info->otr_token, - context, &final); + targets = possible_polymorphic_call_targets (pii->otr_type, pii->otr_token, + context, &final); if (!final || targets.length () > 1) { struct cgraph_node *node; @@ -3242,8 +3229,7 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, if (!opt_for_fn (ie->caller->decl, flag_devirtualize_speculatively) || ie->speculative || !ie->maybe_hot_p ()) return NULL; - node = try_speculative_devirtualization (ie->indirect_info->otr_type, - ie->indirect_info->otr_token, + node = try_speculative_devirtualization (pii->otr_type, pii->otr_token, context); if (node) { @@ -4142,8 +4128,12 @@ ipcp_discover_new_direct_edges (struct cgraph_node *node, avs, &speculative); if (target) { - bool agg_contents = ie->indirect_info->agg_contents; - bool polymorphic = ie->indirect_info->polymorphic; + cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (ie->indirect_info); + cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info); + bool agg_contents = sii && sii->agg_contents; + bool polymorphic = !!pii; int param_index = ie->indirect_info->param_index; struct cgraph_edge *cs = ipa_make_edge_direct_to_target (ie, target, speculative); diff --git a/gcc/ipa-devirt.cc b/gcc/ipa-devirt.cc index c10d67f1e67..69bd73f4240 100644 --- a/gcc/ipa-devirt.cc +++ b/gcc/ipa-devirt.cc @@ -3692,8 +3692,12 @@ ipa_devirt (void) fprintf (dump_file, "\n\nProcesing function %s\n", n->dump_name ()); for (e = n->indirect_calls; e; e = e->next_callee) - if (e->indirect_info->polymorphic) + if (cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (e->indirect_info)) { + if (!pii->usable_p ()) + continue; + void *cache_token; bool final; @@ -3725,12 +3729,12 @@ ipa_devirt (void) This may need to be revisited once we add further ways to use the may edges, but it is a reasonable thing to do right now. */ - if ((e->indirect_info->param_index == -1 + if ((pii->param_index == -1 || (!opt_for_fn (n->decl, flag_devirtualize_speculatively) - && e->indirect_info->vptr_changed)) + && pii->vptr_changed)) && !flag_ltrans_devirtualize) { - e->indirect_info->polymorphic = false; + pii->mark_unusable (); ndropped++; if (dump_file) fprintf (dump_file, "Dropping polymorphic call info;" diff --git a/gcc/ipa-inline.cc b/gcc/ipa-inline.cc index 1f2287da896..f6c375289fe 100644 --- a/gcc/ipa-inline.cc +++ b/gcc/ipa-inline.cc @@ -2758,7 +2758,7 @@ dump_inline_stats (void) } } for (e = node->indirect_calls; e; e = e->next_callee) - if (e->indirect_info->polymorphic + if (is_a <cgraph_polymorphic_indirect_info *> (e->indirect_info) & e->count.ipa ().initialized_p ()) indirect_poly_cnt += e->count.ipa ().to_gcov_type (); else if (e->count.ipa ().initialized_p ()) diff --git a/gcc/ipa-profile.cc b/gcc/ipa-profile.cc index e4c0ff5fe31..8326ccba69b 100644 --- a/gcc/ipa-profile.cc +++ b/gcc/ipa-profile.cc @@ -921,7 +921,7 @@ ipa_profile (void) "Not speculating: " "parameter count mismatch\n"); } - else if (e->indirect_info->polymorphic + else if (usable_polymorphic_info_p (e->indirect_info) && !opt_for_fn (n->decl, flag_devirtualize) && !possible_polymorphic_call_target_p (e, n2)) { diff --git a/gcc/ipa-prop.cc b/gcc/ipa-prop.cc index c8438d67ee4..3d2a4c279b6 100644 --- a/gcc/ipa-prop.cc +++ b/gcc/ipa-prop.cc @@ -599,21 +599,8 @@ ipa_print_node_jump_functions (FILE *f, struct cgraph_node *node) for (cs = node->indirect_calls; cs; cs = cs->next_callee) { - class cgraph_indirect_call_info *ii; - - ii = cs->indirect_info; - if (ii->agg_contents) - fprintf (f, " indirect %s callsite, calling param %i, " - "offset " HOST_WIDE_INT_PRINT_DEC ", %s", - ii->member_ptr ? "member ptr" : "aggregate", - ii->param_index, ii->offset, - ii->by_ref ? "by reference" : "by_value"); - else - fprintf (f, " indirect %s callsite, calling param %i, " - "offset " HOST_WIDE_INT_PRINT_DEC, - ii->polymorphic ? "polymorphic" : "simple", ii->param_index, - ii->offset); - + fprintf (f, " "); + cs->indirect_info->dump (f, false); if (cs->call_stmt) { fprintf (f, ", for stmt "); @@ -621,8 +608,6 @@ ipa_print_node_jump_functions (FILE *f, struct cgraph_node *node) } else fprintf (f, "\n"); - if (ii->polymorphic) - ii->context.dump (f); if (!ipa_edge_args_info_available_for_edge_p (cs)) fprintf (f, " no arg info\n"); else @@ -2789,29 +2774,6 @@ ipa_is_ssa_with_stmt_def (tree t) return false; } -/* Find the indirect call graph edge corresponding to STMT and mark it as a - call to a parameter number PARAM_INDEX. NODE is the caller. Return the - indirect call graph edge. - If POLYMORPHIC is true record is as a destination of polymorphic call. */ - -static struct cgraph_edge * -ipa_note_param_call (struct cgraph_node *node, int param_index, - gcall *stmt, bool polymorphic) -{ - struct cgraph_edge *cs; - - cs = node->get_edge (stmt); - cs->indirect_info->param_index = param_index; - cs->indirect_info->agg_contents = 0; - cs->indirect_info->member_ptr = 0; - cs->indirect_info->guaranteed_unmodified = 0; - ipa_node_params *info = ipa_node_params_sum->get (node); - ipa_set_param_used_by_indirect_call (info, param_index, true); - if (cs->indirect_info->polymorphic || polymorphic) - ipa_set_param_used_by_polymorphic_call (info, param_index, true); - return cs; -} - /* Analyze the CALL and examine uses of formal parameters of the caller NODE (described by INFO). PARMS_AINFO is a pointer to a vector containing intermediate information about each formal parameter. Currently it checks @@ -2885,7 +2847,14 @@ ipa_analyze_indirect_call_uses (struct ipa_func_body_info *fbi, gcall *call, tree var = SSA_NAME_VAR (target); int index = ipa_get_param_decl_index (info, var); if (index >= 0) - ipa_note_param_call (fbi->node, index, call, false); + { + cgraph_edge *cs = fbi->node->get_edge (call); + cgraph_simple_indirect_info *sii = + as_a <cgraph_simple_indirect_info *> (cs->indirect_info); + sii->param_index = index; + gcc_assert (!sii->agg_contents && !sii->member_ptr); + ipa_set_param_used_by_indirect_call (info, index, true); + } return; } @@ -2897,12 +2866,16 @@ ipa_analyze_indirect_call_uses (struct ipa_func_body_info *fbi, gcall *call, gimple_assign_rhs1 (def), &index, &offset, NULL, &by_ref, &guaranteed_unmodified)) { - struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, - call, false); - cs->indirect_info->offset = offset; - cs->indirect_info->agg_contents = 1; - cs->indirect_info->by_ref = by_ref; - cs->indirect_info->guaranteed_unmodified = guaranteed_unmodified; + cgraph_edge *cs = fbi->node->get_edge (call); + cgraph_simple_indirect_info *sii = + as_a <cgraph_simple_indirect_info *> (cs->indirect_info); + sii->param_index = index; + sii->offset = offset; + sii->agg_contents = 1; + sii->by_ref = by_ref; + sii->guaranteed_unmodified = guaranteed_unmodified; + gcc_assert (!sii->member_ptr); + ipa_set_param_used_by_indirect_call (info, index, true); return; } @@ -3026,14 +2999,16 @@ ipa_analyze_indirect_call_uses (struct ipa_func_body_info *fbi, gcall *call, by_ref = false; } - struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, - call, false); - cs->indirect_info->offset = offset; - cs->indirect_info->agg_contents = 1; - cs->indirect_info->member_ptr = 1; - cs->indirect_info->by_ref = by_ref; - cs->indirect_info->guaranteed_unmodified = 1; - + cgraph_edge *cs = fbi->node->get_edge (call); + cgraph_simple_indirect_info *sii = + as_a <cgraph_simple_indirect_info *> (cs->indirect_info); + sii->param_index = index; + sii->offset = offset; + sii->agg_contents = 1; + sii->member_ptr = 1; + sii->by_ref = by_ref; + sii->guaranteed_unmodified = 1; + ipa_set_param_used_by_indirect_call (info, index, true); return; } @@ -3085,13 +3060,15 @@ ipa_analyze_virtual_call_uses (struct ipa_func_body_info *fbi, return; } - struct cgraph_edge *cs = ipa_note_param_call (fbi->node, index, - call, true); - class cgraph_indirect_call_info *ii = cs->indirect_info; - ii->offset = anc_offset; - ii->otr_token = tree_to_uhwi (OBJ_TYPE_REF_TOKEN (target)); - ii->otr_type = obj_type_ref_class (target); - ii->polymorphic = 1; + cgraph_edge *cs = fbi->node->get_edge (call); + cgraph_polymorphic_indirect_info *pii = + as_a <cgraph_polymorphic_indirect_info *> (cs->indirect_info); + pii->param_index = index; + pii->offset = anc_offset; + gcc_assert (pii->otr_token == tree_to_shwi (OBJ_TYPE_REF_TOKEN (target))); + gcc_assert (pii->otr_type = obj_type_ref_class (target)); + ipa_set_param_used_by_indirect_call (info, index, true); + ipa_set_param_used_by_polymorphic_call (info, index, true); } /* Analyze a call statement CALL whether and how it utilizes formal parameters @@ -3114,24 +3091,26 @@ ipa_analyze_call_uses (struct ipa_func_body_info *fbi, gcall *call) if (cs && !cs->indirect_unknown_callee) return; - if (cs->indirect_info->polymorphic && flag_devirtualize) + cgraph_polymorphic_indirect_info *pii; + if (flag_devirtualize + && (pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (cs->indirect_info))) { tree instance; tree target = gimple_call_fn (call); ipa_polymorphic_call_context context (current_function_decl, target, call, &instance); - gcc_checking_assert (cs->indirect_info->otr_type - == obj_type_ref_class (target)); - gcc_checking_assert (cs->indirect_info->otr_token + gcc_checking_assert (pii->otr_type == obj_type_ref_class (target)); + gcc_checking_assert (pii->otr_token == tree_to_shwi (OBJ_TYPE_REF_TOKEN (target))); - cs->indirect_info->vptr_changed + pii->vptr_changed = !context.get_dynamic_type (instance, OBJ_TYPE_REF_OBJECT (target), obj_type_ref_class (target), call, &fbi->aa_walk_budget); - cs->indirect_info->context = context; + pii->context = context; } if (TREE_CODE (target) == SSA_NAME) @@ -3754,16 +3733,17 @@ ipa_make_edge_direct_to_target (struct cgraph_edge *ie, tree target, target = canonicalize_constructor_val (target, NULL); if (!target || TREE_CODE (target) != FUNCTION_DECL) { + cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info); /* Member pointer call that goes through a VMT lookup. */ - if (ie->indirect_info->member_ptr + if ((sii && sii->member_ptr) /* Or if target is not an invariant expression and we do not know if it will evaulate to function at runtime. This can happen when folding through &VAR, where &VAR is IP invariant, but VAR itself is not. - TODO: Revisit this when GCC 5 is branched. It seems that - member_ptr check is not needed and that we may try to fold - the expression and see if VAR is readonly. */ + TODO: It seems that we may try to fold the expression and see + if VAR is readonly. */ || !is_gimple_ip_invariant (target)) { if (dump_enabled_p ()) @@ -3856,7 +3836,8 @@ ipa_make_edge_direct_to_target (struct cgraph_edge *ie, tree target, { fprintf (dump_file, "ipa-prop: Discovered %s call to a %s target " "(%s -> %s), for stmt ", - ie->indirect_info->polymorphic ? "a virtual" : "an indirect", + is_a <cgraph_polymorphic_indirect_info *> (ie->indirect_info) + ? "a virtual" : "an indirect", speculative ? "speculative" : "known", ie->caller->dump_name (), callee->dump_name ()); @@ -4113,26 +4094,25 @@ try_make_edge_direct_simple_call (struct cgraph_edge *ie, struct cgraph_node *new_root, class ipa_node_params *new_root_info) { - struct cgraph_edge *cs; tree target = NULL_TREE; - bool agg_contents = ie->indirect_info->agg_contents; + cgraph_simple_indirect_info *sii + = as_a <cgraph_simple_indirect_info *> (ie->indirect_info); + bool agg_contents = sii->agg_contents; tree scalar = ipa_value_from_jfunc (new_root_info, jfunc, target_type); if (agg_contents) { if (scalar) - target = ipa_find_agg_cst_from_init (scalar, ie->indirect_info->offset, - ie->indirect_info->by_ref); - if (!target && ie->indirect_info->guaranteed_unmodified) + target = ipa_find_agg_cst_from_init (scalar, sii->offset, sii->by_ref); + if (!target && sii->guaranteed_unmodified) target = ipa_find_agg_cst_from_jfunc_items (&jfunc->agg, new_root_info, - new_root, - ie->indirect_info->offset, - ie->indirect_info->by_ref); + new_root, sii->offset, + sii->by_ref); } else target = scalar; if (!target) return NULL; - cs = ipa_make_edge_direct_to_target (ie, target); + cgraph_edge *cs = ipa_make_edge_direct_to_target (ie, target); if (cs && !agg_contents) { @@ -4192,11 +4172,13 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, if (!opt_for_fn (ie->caller->decl, flag_devirtualize)) return NULL; - - gcc_assert (!ie->indirect_info->by_ref); + cgraph_polymorphic_indirect_info *pii + = as_a <cgraph_polymorphic_indirect_info *> (ie->indirect_info); + if (!pii->usable_p ()) + return nullptr; /* Try to do lookup via known virtual table pointer value. */ - if (!ie->indirect_info->vptr_changed + if (!pii->vptr_changed || opt_for_fn (ie->caller->decl, flag_devirtualize_speculatively)) { tree vtable; @@ -4204,16 +4186,15 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, tree t = NULL_TREE; if (jfunc->type == IPA_JF_CONST) t = ipa_find_agg_cst_from_init (ipa_get_jf_constant (jfunc), - ie->indirect_info->offset, true); + pii->offset, true); if (!t) t = ipa_find_agg_cst_from_jfunc_items (&jfunc->agg, new_root_info, - new_root, - ie->indirect_info->offset, true); + new_root, pii->offset, true); if (t && vtable_pointer_value_to_vtable (t, &vtable, &offset)) { bool can_refer; - t = gimple_get_virt_method_for_vtable (ie->indirect_info->otr_token, - vtable, offset, &can_refer); + t = gimple_get_virt_method_for_vtable (pii->otr_token, vtable, offset, + &can_refer); if (can_refer) { if (!t @@ -4223,7 +4204,7 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, (ie, cgraph_node::get (t))) { /* Do not speculate builtin_unreachable, it is stupid! */ - if (!ie->indirect_info->vptr_changed) + if (!pii->vptr_changed) target = ipa_impossible_devirt_target (ie, target); else target = NULL; @@ -4231,7 +4212,7 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, else { target = t; - speculative = ie->indirect_info->vptr_changed; + speculative = pii->vptr_changed; } } } @@ -4241,15 +4222,13 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, vec <cgraph_node *>targets; bool final; - ctx.offset_by (ie->indirect_info->offset); - if (ie->indirect_info->vptr_changed) + ctx.offset_by (pii->offset); + if (pii->vptr_changed) ctx.possible_dynamic_type_change (ie->in_polymorphic_cdtor, - ie->indirect_info->otr_type); - ctx.combine_with (ie_context, ie->indirect_info->otr_type); - targets = possible_polymorphic_call_targets - (ie->indirect_info->otr_type, - ie->indirect_info->otr_token, - ctx, &final); + pii->otr_type); + ctx.combine_with (ie_context, pii->otr_type); + targets = possible_polymorphic_call_targets (pii->otr_type, pii->otr_token, + ctx, &final); if (final && targets.length () <= 1) { speculative = false; @@ -4258,13 +4237,13 @@ try_make_edge_direct_virtual_call (struct cgraph_edge *ie, else target = ipa_impossible_devirt_target (ie, NULL_TREE); } - else if (!target && opt_for_fn (ie->caller->decl, flag_devirtualize_speculatively) + else if (!target && opt_for_fn (ie->caller->decl, + flag_devirtualize_speculatively) && !ie->speculative && ie->maybe_hot_p ()) { cgraph_node *n; - n = try_speculative_devirtualization (ie->indirect_info->otr_type, - ie->indirect_info->otr_token, - ie->indirect_info->context); + n = try_speculative_devirtualization (pii->otr_type, pii->otr_token, + pii->context); if (n) { target = n->decl; @@ -4298,39 +4277,36 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, struct cgraph_node *node, vec<cgraph_edge *> *new_edges) { - class ipa_edge_args *top; - struct cgraph_edge *ie, *next_ie, *new_direct_edge; - struct cgraph_node *new_root; - class ipa_node_params *new_root_info, *inlined_node_info; bool res = false; ipa_check_create_edge_args (); - top = ipa_edge_args_sum->get (cs); - new_root = cs->caller->inlined_to - ? cs->caller->inlined_to : cs->caller; - new_root_info = ipa_node_params_sum->get (new_root); - inlined_node_info = ipa_node_params_sum->get (cs->callee->function_symbol ()); + class ipa_edge_args *top = ipa_edge_args_sum->get (cs); + if (!top) + return res; + cgraph_node *new_root + = cs->caller->inlined_to ? cs->caller->inlined_to : cs->caller; + ipa_node_params *new_root_info = ipa_node_params_sum->get (new_root); + ipa_node_params *inlined_node_info + = ipa_node_params_sum->get (cs->callee->function_symbol ()); - for (ie = node->indirect_calls; ie; ie = next_ie) + cgraph_edge *next_ie; + for (cgraph_edge *ie = node->indirect_calls; ie; ie = next_ie) { - class cgraph_indirect_call_info *ici = ie->indirect_info; - struct ipa_jump_func *jfunc; - int param_index; - next_ie = ie->next_callee; - if (ici->param_index == -1) - continue; - - /* We must check range due to calls with variable number of arguments: */ - if (!top || ici->param_index >= ipa_get_cs_argument_count (top)) + if (ie->indirect_info->param_index < 0 + || ie->indirect_info->param_index >= ipa_get_cs_argument_count (top)) { - ici->param_index = -1; + ie->indirect_info->param_index = -1; continue; } - param_index = ici->param_index; - jfunc = ipa_get_ith_jump_func (top, param_index); + int param_index = ie->indirect_info->param_index; + cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (ie->indirect_info); + cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info); + struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (top, param_index); auto_vec<cgraph_node *, 4> spec_targets; if (ie->speculative) @@ -4339,9 +4315,10 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, direct = direct->next_speculative_call_target ()) spec_targets.safe_push (direct->callee); + cgraph_edge *new_direct_edge; if (!opt_for_fn (node->decl, flag_indirect_inlining)) new_direct_edge = NULL; - else if (ici->polymorphic) + else if (pii) { ipa_polymorphic_call_context ctx; ctx = ipa_context_from_jfunc (new_root_info, cs, param_index, jfunc); @@ -4349,7 +4326,7 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, new_root, new_root_info); } - else + else if (sii) { tree target_type = ipa_get_type (inlined_node_info, param_index); new_direct_edge = try_make_edge_direct_simple_call (ie, jfunc, @@ -4357,6 +4334,8 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, new_root, new_root_info); } + else + gcc_unreachable (); /* If speculation was removed, then we need to do nothing. */ if (new_direct_edge && new_direct_edge != ie @@ -4383,46 +4362,52 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, if (jfunc->type == IPA_JF_PASS_THROUGH && ipa_get_jf_pass_through_operation (jfunc) == NOP_EXPR) { - if (ici->agg_contents - && !ipa_get_jf_pass_through_agg_preserved (jfunc) - && !ici->polymorphic) - ici->param_index = -1; + if (!pii + && sii->agg_contents + && !ipa_get_jf_pass_through_agg_preserved (jfunc)) + ie->indirect_info->param_index = -1; else { - ici->param_index = ipa_get_jf_pass_through_formal_id (jfunc); - if (ici->polymorphic - && !ipa_get_jf_pass_through_type_preserved (jfunc)) - ici->vptr_changed = true; - ipa_set_param_used_by_indirect_call (new_root_info, - ici->param_index, true); - if (ici->polymorphic) - ipa_set_param_used_by_polymorphic_call (new_root_info, - ici->param_index, true); + param_index = ipa_get_jf_pass_through_formal_id (jfunc); + ie->indirect_info->param_index = param_index; + ipa_set_param_used_by_indirect_call (new_root_info, param_index, + true); + if (pii) + { + if (!ipa_get_jf_pass_through_type_preserved (jfunc)) + pii->vptr_changed = true; + ipa_set_param_used_by_polymorphic_call (new_root_info, + param_index, true); + } } } else if (jfunc->type == IPA_JF_ANCESTOR) { - if (ici->agg_contents - && !ipa_get_jf_ancestor_agg_preserved (jfunc) - && !ici->polymorphic) - ici->param_index = -1; + if (!pii + && sii->agg_contents + && !ipa_get_jf_ancestor_agg_preserved (jfunc)) + ie->indirect_info->param_index = -1; else { - ici->param_index = ipa_get_jf_ancestor_formal_id (jfunc); - ici->offset += ipa_get_jf_ancestor_offset (jfunc); - if (ici->polymorphic - && !ipa_get_jf_ancestor_type_preserved (jfunc)) - ici->vptr_changed = true; - ipa_set_param_used_by_indirect_call (new_root_info, - ici->param_index, true); - if (ici->polymorphic) - ipa_set_param_used_by_polymorphic_call (new_root_info, - ici->param_index, true); + param_index = ipa_get_jf_ancestor_formal_id (jfunc); + ie->indirect_info->param_index = param_index; + ipa_set_param_used_by_indirect_call (new_root_info, param_index, + true); + if (pii) + { + pii->offset += ipa_get_jf_ancestor_offset (jfunc); + if (!ipa_get_jf_ancestor_type_preserved (jfunc)) + pii->vptr_changed = true; + ipa_set_param_used_by_polymorphic_call (new_root_info, + param_index, true); + } + else + sii->offset += ipa_get_jf_ancestor_offset (jfunc); } } else /* Either we can find a destination for this edge now or never. */ - ici->param_index = -1; + ie->indirect_info->param_index = -1; } return res; @@ -5311,29 +5296,44 @@ static void ipa_write_indirect_edge_info (struct output_block *ob, struct cgraph_edge *cs) { - class cgraph_indirect_call_info *ii = cs->indirect_info; struct bitpack_d bp; - streamer_write_hwi (ob, ii->param_index); bp = bitpack_create (ob->main_stream); - bp_pack_value (&bp, ii->polymorphic, 1); - bp_pack_value (&bp, ii->agg_contents, 1); - bp_pack_value (&bp, ii->member_ptr, 1); - bp_pack_value (&bp, ii->by_ref, 1); - bp_pack_value (&bp, ii->guaranteed_unmodified, 1); - bp_pack_value (&bp, ii->vptr_changed, 1); + bp_pack_enum (&bp, cgraph_indirect_info_kind, CIIK_N_KINDS, + cs->indirect_info->kind); streamer_write_bitpack (&bp); - if (ii->agg_contents || ii->polymorphic) - streamer_write_hwi (ob, ii->offset); - else - gcc_assert (ii->offset == 0); - if (ii->polymorphic) + if (cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (cs->indirect_info)) + { + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, pii->vptr_changed, 1); + streamer_write_bitpack (&bp); + + streamer_write_hwi (ob, pii->param_index); + pii->context.stream_out (ob); + streamer_write_hwi (ob, pii->otr_token); + stream_write_tree (ob, pii->otr_type, true); + streamer_write_hwi (ob, pii->offset); + } + else if (cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (cs->indirect_info)) { - streamer_write_hwi (ob, ii->otr_token); - stream_write_tree (ob, ii->otr_type, true); - ii->context.stream_out (ob); + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, sii->agg_contents, 1); + bp_pack_value (&bp, sii->member_ptr, 1); + bp_pack_value (&bp, sii->by_ref, 1); + bp_pack_value (&bp, sii->guaranteed_unmodified, 1); + streamer_write_bitpack (&bp); + + streamer_write_hwi (ob, sii->param_index); + if (sii->agg_contents) + streamer_write_hwi (ob, sii->offset); + else + gcc_assert (sii->offset == 0); } + else + gcc_assert (cs->indirect_info->param_index == -1); } /* Read in parts of cgraph_indirect_call_info corresponding to CS that are @@ -5345,35 +5345,50 @@ ipa_read_indirect_edge_info (class lto_input_block *ib, struct cgraph_edge *cs, class ipa_node_params *info) { - class cgraph_indirect_call_info *ii = cs->indirect_info; struct bitpack_d bp; - ii->param_index = (int) streamer_read_hwi (ib); bp = streamer_read_bitpack (ib); - ii->polymorphic = bp_unpack_value (&bp, 1); - ii->agg_contents = bp_unpack_value (&bp, 1); - ii->member_ptr = bp_unpack_value (&bp, 1); - ii->by_ref = bp_unpack_value (&bp, 1); - ii->guaranteed_unmodified = bp_unpack_value (&bp, 1); - ii->vptr_changed = bp_unpack_value (&bp, 1); - if (ii->agg_contents || ii->polymorphic) - ii->offset = (HOST_WIDE_INT) streamer_read_hwi (ib); - else - ii->offset = 0; - if (ii->polymorphic) + enum cgraph_indirect_info_kind ii_kind + = bp_unpack_enum (&bp, cgraph_indirect_info_kind, CIIK_N_KINDS); + gcc_assert (ii_kind == cs->indirect_info->kind); + + if (cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (cs->indirect_info)) { - ii->otr_token = (HOST_WIDE_INT) streamer_read_hwi (ib); - ii->otr_type = stream_read_tree (ib, data_in); - ii->context.stream_in (ib, data_in); + bp = streamer_read_bitpack (ib); + pii->vptr_changed = bp_unpack_value (&bp, 1); + + pii->param_index = (int) streamer_read_hwi (ib); + pii->context.stream_in (ib, data_in); + pii->otr_token = (HOST_WIDE_INT) streamer_read_hwi (ib); + pii->otr_type = stream_read_tree (ib, data_in); + pii->offset = (HOST_WIDE_INT) streamer_read_hwi (ib); + + if (info && pii->param_index >= 0) + { + ipa_set_param_used_by_polymorphic_call (info, pii->param_index, true); + ipa_set_param_used_by_indirect_call (info, pii->param_index, true); + } } - if (info && ii->param_index >= 0) + else if (cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (cs->indirect_info)) { - if (ii->polymorphic) - ipa_set_param_used_by_polymorphic_call (info, - ii->param_index , true); - ipa_set_param_used_by_indirect_call (info, - ii->param_index, true); + bp = streamer_read_bitpack (ib); + sii->agg_contents = bp_unpack_value (&bp, 1); + sii->member_ptr = bp_unpack_value (&bp, 1); + sii->by_ref = bp_unpack_value (&bp, 1); + sii->guaranteed_unmodified = bp_unpack_value (&bp, 1); + + sii->param_index = (int) streamer_read_hwi (ib); + if (sii->agg_contents) + sii->offset = (HOST_WIDE_INT) streamer_read_hwi (ib); + else + sii->offset = 0; + if (info && sii->param_index >= 0) + ipa_set_param_used_by_indirect_call (info, sii->param_index, true); } + else + cs->indirect_info->param_index = -1; } /* Stream out NODE info to OB. */ diff --git a/gcc/ipa-utils.h b/gcc/ipa-utils.h index f33d21e64ce..56d12d1209f 100644 --- a/gcc/ipa-utils.h +++ b/gcc/ipa-utils.h @@ -134,8 +134,11 @@ possible_polymorphic_call_targets (struct cgraph_edge *e, { ipa_polymorphic_call_context context(e); - return possible_polymorphic_call_targets (e->indirect_info->otr_type, - e->indirect_info->otr_token, + cgraph_polymorphic_indirect_info *pii + = as_a <cgraph_polymorphic_indirect_info *> (e->indirect_info); + gcc_checking_assert (pii->usable_p ()); + return possible_polymorphic_call_targets (pii->otr_type, + pii->otr_token, context, completep, cache_token, speculative); @@ -166,8 +169,12 @@ dump_possible_polymorphic_call_targets (FILE *f, struct cgraph_edge *e, { ipa_polymorphic_call_context context(e); - dump_possible_polymorphic_call_targets (f, e->indirect_info->otr_type, - e->indirect_info->otr_token, + cgraph_polymorphic_indirect_info *pii + = as_a <cgraph_polymorphic_indirect_info *> (e->indirect_info); + if (!pii->usable_p ()) + return; + dump_possible_polymorphic_call_targets (f, pii->otr_type, + pii->otr_token, context, verbose); } @@ -180,8 +187,12 @@ possible_polymorphic_call_target_p (struct cgraph_edge *e, { ipa_polymorphic_call_context context(e); - return possible_polymorphic_call_target_p (e->indirect_info->otr_type, - e->indirect_info->otr_token, + cgraph_polymorphic_indirect_info *pii + = as_a <cgraph_polymorphic_indirect_info *> (e->indirect_info); + if (!pii->usable_p ()) + return true; + return possible_polymorphic_call_target_p (pii->otr_type, + pii->otr_token, context, n); } diff --git a/gcc/ipa.cc b/gcc/ipa.cc index dea22ea0b49..2c8565eba25 100644 --- a/gcc/ipa.cc +++ b/gcc/ipa.cc @@ -428,7 +428,7 @@ symbol_table::remove_unreachable_nodes (FILE *file) for (e = cnode->indirect_calls; e; e = next) { next = e->next_callee; - if (e->indirect_info->polymorphic) + if (usable_polymorphic_info_p (e->indirect_info)) walk_polymorphic_call_targets (&reachable_call_targets, e, &first, &reachable); } diff --git a/gcc/lto-cgraph.cc b/gcc/lto-cgraph.cc index 5708ba046c9..ee4a84932f8 100644 --- a/gcc/lto-cgraph.cc +++ b/gcc/lto-cgraph.cc @@ -284,6 +284,8 @@ lto_output_edge (struct lto_simple_output_block *ob, struct cgraph_edge *edge, bp_pack_value (&bp, edge->in_polymorphic_cdtor, 1); if (edge->indirect_unknown_callee) { + bp_pack_enum (&bp, cgraph_indirect_info_kind, CIIK_N_KINDS, + edge->indirect_info->kind); int flags = edge->indirect_info->ecf_flags; bp_pack_value (&bp, (flags & ECF_CONST) != 0, 1); bp_pack_value (&bp, (flags & ECF_PURE) != 0, 1); @@ -930,7 +932,7 @@ compute_ltrans_boundary (lto_symtab_encoder_t in_encoder) /* Add all possible targets for late devirtualization. */ if (flag_ltrans_devirtualize || !flag_wpa) for (edge = node->indirect_calls; edge; edge = edge->next_callee) - if (edge->indirect_info->polymorphic) + if (usable_polymorphic_info_p (edge->indirect_info)) { unsigned int i; void *cache_token; @@ -1513,7 +1515,6 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes, profile_count count; cgraph_inline_failed_t inline_failed; struct bitpack_d bp; - int ecf_flags = 0; caller = dyn_cast<cgraph_node *> (nodes[streamer_read_hwi (ib)]); if (caller == NULL || caller->decl == NULL_TREE) @@ -1536,7 +1537,7 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes, speculative_id = bp_unpack_value (&bp, 16); if (indirect) - edge = caller->create_indirect_edge (NULL, 0, count); + edge = caller->create_indirect_edge (NULL, 0, count, true); else edge = caller->create_edge (callee, NULL, count); @@ -1553,6 +1554,9 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes, edge->in_polymorphic_cdtor = bp_unpack_value (&bp, 1); if (indirect) { + enum cgraph_indirect_info_kind ii_kind + = bp_unpack_enum (&bp, cgraph_indirect_info_kind, CIIK_N_KINDS); + int ecf_flags = 0; if (bp_unpack_value (&bp, 1)) ecf_flags |= ECF_CONST; if (bp_unpack_value (&bp, 1)) @@ -1565,7 +1569,19 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes, ecf_flags |= ECF_NOTHROW; if (bp_unpack_value (&bp, 1)) ecf_flags |= ECF_RETURNS_TWICE; - edge->indirect_info->ecf_flags = ecf_flags; + + if (ii_kind == CIIK_POLYMORPHIC) + edge->indirect_info + = (new (ggc_alloc<cgraph_polymorphic_indirect_info> ()) + cgraph_polymorphic_indirect_info (ecf_flags)); + else if (ii_kind == CIIK_SIMPLE) + edge->indirect_info + = (new (ggc_alloc<cgraph_simple_indirect_info> ()) + cgraph_simple_indirect_info (ecf_flags)); + else + edge->indirect_info + = (new (ggc_alloc<cgraph_indirect_call_info> ()) + cgraph_indirect_call_info(CIIK_UNSPECIFIED, ecf_flags)); edge->indirect_info->num_speculative_call_targets = bp_unpack_value (&bp, 16); diff --git a/gcc/trans-mem.cc b/gcc/trans-mem.cc index 9fc1b2d54e3..a989efb431e 100644 --- a/gcc/trans-mem.cc +++ b/gcc/trans-mem.cc @@ -5163,9 +5163,13 @@ ipa_tm_insert_gettmclone_call (struct cgraph_node *node, update_stmt (stmt); cgraph_edge *e = cgraph_node::get (current_function_decl)->get_edge (stmt); - if (e && e->indirect_info) - e->indirect_info->polymorphic = false; - + if (e) + { + cgraph_polymorphic_indirect_info *pii + = dyn_cast <cgraph_polymorphic_indirect_info *> (e->indirect_info); + if (pii) + pii->mark_unusable (); + } return true; } -- 2.51.0
