On Thu, May 23, 2013 at 6:18 AM, Teresa Johnson <tejohn...@google.com> wrote: > On Wed, May 22, 2013 at 2:05 PM, Teresa Johnson <tejohn...@google.com> wrote: >> Revised patch included below. The spacing of my pasted in patch text >> looks funky again, let me know if you want the patch as an attachment >> instead. >> >> I addressed all of Steven's comments, except for the suggestion to use >> gcc_assert >> instead of error() in verify_hot_cold_block_grouping() to keep this >> consistent >> with the rest of the verify_flow_info subroutines (let me know if this is >> ok). > > I fixed this issue too, which was actually in > insert_section_boundary_note(), so that it gcc_asserts more > efficiently as suggested. Retested, latest patch below. > > Honza, would you be able to review the patch?
Ping. Still needs a global maintainer to review and approve. Also, I submitted a PR for the debug range issue: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57451 Thanks! Teresa > > Thanks! > Teresa > >> >> The other main changes: >> (1) Added several test cases (cloned from the torture subdirectories, >> where I manually >> built/ran with FDO and -freorder-blocks-and-partition with both the >> current trunk and >> my fixed trunk compiler, and was able to expose some failures I fixed. >> (2) Changed existing tree-prof tests that used >> -freorder-blocks-and-partition to be >> built with -O2 instead of -O, so that partitioning actually kicks in. >> (3) Fixed a couple of failures in the new >> verify_hot_cold_block_grouping() checks >> exposed by the torture tests I ran manually with splitting (2 of the >> tests cloned >> to tree-prof in this patch). One was in computed goto where we were >> too aggressive >> about cloning crossing edges, and the other was in rtl_split_edge >> called from the "stack" >> pass which was not correctly inserting the new bb in the correct partition >> since >> bb layout is complete at that point. >> >> Re-tested on x86_64-unknown-linux-gnu with bootstrap and profiledbootstrap >> builds and regression testing. Re-built/ran cpu2006int with profile >> feedback and -freorder-blocks-and-partition enabled. >> >> Ok for trunk? >> >> Thanks! >> Teresa > > 2013-05-23 Teresa Johnson <tejohn...@google.com> > > * ifcvt.c (find_if_case_1): Replace BB_COPY_PARTITION with assert > as this is now done by redirect_edge_and_branch_force. > * function.c (thread_prologue_and_epilogue_insns): Insert new bb after > barriers, and fix interaction with splitting. > * emit-rtl.c (try_split): Copy REG_CROSSING_JUMP notes. > * cfgcleanup.c (try_forward_edges): Fix early return value to properly > reflect changes made in the routine. > * bb-reorder.c (emit_barrier_after_bb): Move to cfgrtl.c. > (fix_up_fall_thru_edges): Remove incorrect check for bb layout order > since this is called in cfglayout mode, and replace partition fixup > with assert as that is now done by force_nonfallthru_and_redirect. > (add_reg_crossing_jump_notes): Handle the fact that some jumps may > already be marked with region crossing note. > (insert_section_boundary_note): Make non-static, gate on flag > has_bb_partition, rewrite to also check for multiple partitions. > (rest_of_handle_reorder_blocks): Remove call to > insert_section_boundary_note, now done later during free_cfg. > (duplicate_computed_gotos): Don't duplicate partition crossing edge. > * bb-reorder.h (insert_section_boundary_note): Declare. > * Makefile.in (cfgrtl.o): Depend on bb-reorder.h > * cfgrtl.c (rest_of_pass_free_cfg): If partitions exist > invoke insert_section_boundary_note. > (try_redirect_by_replacing_jump): Remove unnecessary > check for region crossing note. > (fixup_partition_crossing): New function. > (rtl_redirect_edge_and_branch): Fixup partition boundaries. > (emit_barrier_after_bb): Move here from bb-reorder.c, handle insertion > in non-cfglayout mode. > (force_nonfallthru_and_redirect): Fixup partition boundaries, > remove old code that tried to do this. Emit barrier correctly > when we are in cfglayout mode. > (last_bb_in_partition): New function. > (rtl_split_edge): Correctly fixup partition boundaries. > (commit_one_edge_insertion): Remove old code that tried to > fixup region crossing edge since this is now handled in > split_block, and set up insertion point correctly since > block may now end in a jump. > (verify_hot_cold_block_grouping): Guard against checking when not in > linearized RTL mode. > (rtl_verify_edges): Add checks for incorrect/missing REG_CROSSING_JUMP > notes. > (rtl_verify_flow_info_1): Move verify_hot_cold_block_grouping to > rtl_verify_flow_info, so not called in cfglayout mode. > (rtl_verify_flow_info): Move verify_hot_cold_block_grouping here. > (fixup_reorder_chain): Remove old code that attempted to fixup region > crossing note as this is now handled in > force_nonfallthru_and_redirect. > (duplicate_insn_chain): Don't duplicate switch section notes. > (rtl_can_remove_branch_p): Remove unnecessary check for region crossing > note. > * basic-block.h (emit_barrier_after_bb): Declare. > * testsuite/gcc.dg/tree-prof/va-arg-pack-1.c: Cloned from c-torture, made > into -freorder-blocks-and-partition test. > * testsuite/gcc.dg/tree-prof/comp-goto-1.c: Ditto. > * testsuite/gcc.dg/tree-prof/20041218-1.c: Ditto. > * testsuite/gcc.dg/tree-prof/pr52027.c: Use -O2. > * testsuite/gcc.dg/tree-prof/pr50907.c: Ditto. > * testsuite/gcc.dg/tree-prof/pr45354.c: Ditto. > * testsuite/g++.dg/tree-prof/partition2.C: Ditto. > * testsuite/g++.dg/tree-prof/partition3.C: Ditto. > > Index: ifcvt.c > =================================================================== > --- ifcvt.c (revision 199014) > +++ ifcvt.c (working copy) > @@ -3905,10 +3905,9 @@ find_if_case_1 (basic_block test_bb, edge then_edg > if (new_bb) > { > df_bb_replace (then_bb_index, new_bb); > - /* Since the fallthru edge was redirected from test_bb to new_bb, > - we need to ensure that new_bb is in the same partition as > - test bb (you can not fall through across section boundaries). */ > - BB_COPY_PARTITION (new_bb, test_bb); > + /* This should have been done above via force_nonfallthru_and_redirect > + (possibly called from redirect_edge_and_branch_force). */ > + gcc_checking_assert (BB_PARTITION (new_bb) == BB_PARTITION (test_bb)); > } > > num_true_changes++; > Index: function.c > =================================================================== > --- function.c (revision 199014) > +++ function.c (working copy) > @@ -6270,8 +6270,10 @@ thread_prologue_and_epilogue_insns (void) > break; > if (e) > { > - copy_bb = create_basic_block (NEXT_INSN (BB_END (e->src)), > - NULL_RTX, e->src); > + /* Make sure we insert after any barriers. */ > + rtx end = get_last_bb_insn (e->src); > + copy_bb = create_basic_block (NEXT_INSN (end), > + NULL_RTX, e->src); > BB_COPY_PARTITION (copy_bb, e->src); > } > else > @@ -6538,7 +6540,7 @@ epilogue_done: > basic_block simple_return_block_cold = NULL; > edge pending_edge_hot = NULL; > edge pending_edge_cold = NULL; > - basic_block exit_pred = EXIT_BLOCK_PTR->prev_bb; > + basic_block exit_pred; > int i; > > gcc_assert (entry_edge != orig_entry_edge); > @@ -6566,6 +6568,12 @@ epilogue_done: > else > pending_edge_cold = e; > } > + > + /* Save a pointer to the exit's predecessor BB for use in > + inserting new BBs at the end of the function. Do this > + after the call to split_block above which may split > + the original exit pred. */ > + exit_pred = EXIT_BLOCK_PTR->prev_bb; > > FOR_EACH_VEC_ELT (unconverted_simple_returns, i, e) > { > Index: emit-rtl.c > =================================================================== > --- emit-rtl.c (revision 199014) > +++ emit-rtl.c (working copy) > @@ -3574,6 +3574,7 @@ try_split (rtx pat, rtx trial, int last) > break; > > case REG_NON_LOCAL_GOTO: > + case REG_CROSSING_JUMP: > for (insn = insn_last; insn != NULL_RTX; insn = PREV_INSN (insn)) > { > if (JUMP_P (insn)) > Index: cfgcleanup.c > =================================================================== > --- cfgcleanup.c (revision 199014) > +++ cfgcleanup.c (working copy) > @@ -456,7 +456,7 @@ try_forward_edges (int mode, basic_block b) > > if (first != EXIT_BLOCK_PTR > && find_reg_note (BB_END (first), REG_CROSSING_JUMP, NULL_RTX)) > - return false; > + return changed; > > while (counter < n_basic_blocks) > { > Index: bb-reorder.c > =================================================================== > --- bb-reorder.c (revision 199014) > +++ bb-reorder.c (working copy) > @@ -1380,15 +1380,6 @@ get_uncond_jump_length (void) > return length; > } > > -/* Emit a barrier into the footer of BB. */ > - > -static void > -emit_barrier_after_bb (basic_block bb) > -{ > - rtx barrier = emit_barrier_after (BB_END (bb)); > - BB_FOOTER (bb) = unlink_insn_chain (barrier, barrier); > -} > - > /* The landing pad OLD_LP, in block OLD_BB, has edges from both partitions. > Duplicate the landing pad and split the edges so that no EH edge > crosses partitions. */ > @@ -1720,8 +1711,7 @@ fix_up_fall_thru_edges (void) > (i.e. fix it so the fall through does not cross and > the cond jump does). */ > > - if (!cond_jump_crosses > - && cur_bb->aux == cond_jump->dest) > + if (!cond_jump_crosses) > { > /* Find label in fall_thru block. We've already added > any missing labels, so there must be one. */ > @@ -1765,10 +1755,10 @@ fix_up_fall_thru_edges (void) > new_bb->aux = cur_bb->aux; > cur_bb->aux = new_bb; > > - /* Make sure new fall-through bb is in same > - partition as bb it's falling through from. */ > + /* This is done by force_nonfallthru_and_redirect. */ > + gcc_assert (BB_PARTITION (new_bb) > + == BB_PARTITION (cur_bb)); > > - BB_COPY_PARTITION (new_bb, cur_bb); > single_succ_edge (new_bb)->flags |= EDGE_CROSSING; > } > else > @@ -2064,7 +2054,10 @@ add_reg_crossing_jump_notes (void) > FOR_EACH_BB (bb) > FOR_EACH_EDGE (e, ei, bb->succs) > if ((e->flags & EDGE_CROSSING) > - && JUMP_P (BB_END (e->src))) > + && JUMP_P (BB_END (e->src)) > + /* Some notes were added during fix_up_fall_thru_edges, via > + force_nonfallthru_and_redirect. */ > + && !find_reg_note (BB_END (e->src), REG_CROSSING_JUMP, NULL_RTX)) > add_reg_note (BB_END (e->src), REG_CROSSING_JUMP, NULL_RTX); > } > > @@ -2133,23 +2126,26 @@ reorder_basic_blocks (void) > encountering this note will make the compiler switch between the > hot and cold text sections. */ > > -static void > +void > insert_section_boundary_note (void) > { > basic_block bb; > - int first_partition = 0; > + bool switched_sections = false; > + int current_partition = 0; > > - if (!flag_reorder_blocks_and_partition) > + if (!crtl->has_bb_partition) > return; > > FOR_EACH_BB (bb) > { > - if (!first_partition) > - first_partition = BB_PARTITION (bb); > - if (BB_PARTITION (bb) != first_partition) > + if (!current_partition) > + current_partition = BB_PARTITION (bb); > + if (BB_PARTITION (bb) != current_partition) > { > - emit_note_before (NOTE_INSN_SWITCH_TEXT_SECTIONS, BB_HEAD (bb)); > - break; > + gcc_assert (!switched_sections); > + switched_sections = true; > + emit_note_before (NOTE_INSN_SWITCH_TEXT_SECTIONS, BB_HEAD (bb)); > + current_partition = BB_PARTITION (bb); > } > } > } > @@ -2180,8 +2176,6 @@ rest_of_handle_reorder_blocks (void) > bb->aux = bb->next_bb; > cfg_layout_finalize (); > > - /* Add NOTE_INSN_SWITCH_TEXT_SECTIONS notes. */ > - insert_section_boundary_note (); > return 0; > } > > @@ -2315,6 +2309,11 @@ duplicate_computed_gotos (void) > if (!bitmap_bit_p (candidates, single_succ (bb)->index)) > continue; > > + /* Don't duplicate a partition crossing edge, which requires difficult > + fixup. */ > + if (find_reg_note (BB_END (bb), REG_CROSSING_JUMP, NULL_RTX)) > + continue; > + > new_bb = duplicate_block (single_succ (bb), single_succ_edge (bb), bb); > new_bb->aux = bb->aux; > bb->aux = new_bb; > Index: bb-reorder.h > =================================================================== > --- bb-reorder.h (revision 199014) > +++ bb-reorder.h (working copy) > @@ -35,4 +35,6 @@ extern struct target_bb_reorder *this_target_bb_re > > extern int get_uncond_jump_length (void); > > +extern void insert_section_boundary_note (void); > + > #endif > Index: Makefile.in > =================================================================== > --- Makefile.in (revision 199014) > +++ Makefile.in (working copy) > @@ -3151,7 +3151,7 @@ cfgrtl.o : cfgrtl.c $(CONFIG_H) $(SYSTEM_H) corety > $(FUNCTION_H) $(EXCEPT_H) $(TM_P_H) $(INSN_ATTR_H) \ > insn-config.h $(EXPR_H) \ > $(CFGLOOP_H) $(OBSTACK_H) $(TARGET_H) $(TREE_H) \ > - $(TREE_PASS_H) $(DF_H) $(GGC_H) $(COMMON_TARGET_H) gt-cfgrtl.h > + $(TREE_PASS_H) $(DF_H) $(GGC_H) $(COMMON_TARGET_H) gt-cfgrtl.h > bb-reorder.h > cfganal.o : cfganal.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(BASIC_BLOCK_H) \ > $(TIMEVAR_H) sbitmap.h $(BITMAP_H) > cfgbuild.o : cfgbuild.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(RTL_H) > \ > Index: cfgrtl.c > =================================================================== > --- cfgrtl.c (revision 199014) > +++ cfgrtl.c (working copy) > @@ -44,6 +44,7 @@ along with GCC; see the file COPYING3. If not see > #include "tree.h" > #include "hard-reg-set.h" > #include "basic-block.h" > +#include "bb-reorder.h" > #include "regs.h" > #include "flags.h" > #include "function.h" > @@ -451,6 +452,9 @@ rest_of_pass_free_cfg (void) > } > #endif > > + if (crtl->has_bb_partition) > + insert_section_boundary_note (); > + > free_bb_for_insn (); > return 0; > } > @@ -981,8 +985,7 @@ try_redirect_by_replacing_jump (edge e, basic_bloc > partition boundaries). See the comments at the top of > bb-reorder.c:partition_hot_cold_basic_blocks for complete details. */ > > - if (find_reg_note (insn, REG_CROSSING_JUMP, NULL_RTX) > - || BB_PARTITION (src) != BB_PARTITION (target)) > + if (BB_PARTITION (src) != BB_PARTITION (target)) > return NULL; > > /* We can replace or remove a complex jump only when we have exactly > @@ -1291,6 +1294,53 @@ redirect_branch_edge (edge e, basic_block target) > return e; > } > > +/* Called when edge E has been redirected to a new destination, > + in order to update the region crossing flag on the edge and > + jump. */ > + > +static void > +fixup_partition_crossing (edge e) > +{ > + rtx note; > + > + if (e->src == ENTRY_BLOCK_PTR || e->dest == EXIT_BLOCK_PTR) > + return; > + /* If we redirected an existing edge, it may already be marked > + crossing, even though the new src is missing a reg crossing note. > + But make sure reg crossing note doesn't already exist before > + inserting. */ > + if (BB_PARTITION (e->src) != BB_PARTITION (e->dest)) > + { > + e->flags |= EDGE_CROSSING; > + note = find_reg_note (BB_END (e->src), REG_CROSSING_JUMP, NULL_RTX); > + if (JUMP_P (BB_END (e->src)) > + && !note) > + add_reg_note (BB_END (e->src), REG_CROSSING_JUMP, NULL_RTX); > + } > + else if (BB_PARTITION (e->src) == BB_PARTITION (e->dest)) > + { > + e->flags &= ~EDGE_CROSSING; > + /* Remove the section crossing note from jump at end of > + src if it exists, and if no other successors are > + still crossing. */ > + note = find_reg_note (BB_END (e->src), REG_CROSSING_JUMP, NULL_RTX); > + if (note) > + { > + bool has_crossing_succ = false; > + edge e2; > + edge_iterator ei; > + FOR_EACH_EDGE (e2, ei, e->src->succs) > + { > + has_crossing_succ |= (e2->flags & EDGE_CROSSING); > + if (has_crossing_succ) > + break; > + } > + if (!has_crossing_succ) > + remove_note (BB_END (e->src), note); > + } > + } > +} > + > /* Attempt to change code to redirect edge E to TARGET. Don't do that on > expense of adding new instructions or reordering basic blocks. > > @@ -1307,16 +1357,18 @@ rtl_redirect_edge_and_branch (edge e, basic_block > { > edge ret; > basic_block src = e->src; > + basic_block dest = e->dest; > > if (e->flags & (EDGE_ABNORMAL_CALL | EDGE_EH)) > return NULL; > > - if (e->dest == target) > + if (dest == target) > return e; > > if ((ret = try_redirect_by_replacing_jump (e, target, false)) != NULL) > { > df_set_bb_dirty (src); > + fixup_partition_crossing (ret); > return ret; > } > > @@ -1325,9 +1377,22 @@ rtl_redirect_edge_and_branch (edge e, basic_block > return NULL; > > df_set_bb_dirty (src); > + fixup_partition_crossing (ret); > return ret; > } > > +/* Emit a barrier after BB, into the footer if we are in CFGLAYOUT mode. */ > + > +void > +emit_barrier_after_bb (basic_block bb) > +{ > + rtx barrier = emit_barrier_after (BB_END (bb)); > + gcc_assert (current_ir_type() == IR_RTL_CFGRTL > + || current_ir_type () == IR_RTL_CFGLAYOUT); > + if (current_ir_type () == IR_RTL_CFGLAYOUT) > + BB_FOOTER (bb) = unlink_insn_chain (barrier, barrier); > +} > + > /* Like force_nonfallthru below, but additionally performs redirection > Used by redirect_edge_and_branch_force. JUMP_LABEL is used only > when redirecting to the EXIT_BLOCK, it is either ret_rtx or > @@ -1492,12 +1557,6 @@ force_nonfallthru_and_redirect (edge e, basic_bloc > /* Make sure new block ends up in correct hot/cold section. */ > > BB_COPY_PARTITION (jump_block, e->src); > - if (flag_reorder_blocks_and_partition > - && targetm_common.have_named_sections > - && JUMP_P (BB_END (jump_block)) > - && !any_condjump_p (BB_END (jump_block)) > - && (EDGE_SUCC (jump_block, 0)->flags & EDGE_CROSSING)) > - add_reg_note (BB_END (jump_block), REG_CROSSING_JUMP, NULL_RTX); > > /* Wire edge in. */ > new_edge = make_edge (e->src, jump_block, EDGE_FALLTHRU); > @@ -1508,6 +1567,10 @@ force_nonfallthru_and_redirect (edge e, basic_bloc > redirect_edge_pred (e, jump_block); > e->probability = REG_BR_PROB_BASE; > > + /* If e->src was previously region crossing, it no longer is > + and the reg crossing note should be removed. */ > + fixup_partition_crossing (new_edge); > + > /* If asm goto has any label refs to target's label, > add also edge from asm goto bb to target. */ > if (asm_goto_edge) > @@ -1559,13 +1622,16 @@ force_nonfallthru_and_redirect (edge e, basic_bloc > LABEL_NUSES (label)++; > } > > - emit_barrier_after (BB_END (jump_block)); > + /* We might be in cfg layout mode, and if so, the following routine will > + insert the barrier correctly. */ > + emit_barrier_after_bb (jump_block); > redirect_edge_succ_nodup (e, target); > > if (abnormal_edge_flags) > make_edge (src, target, abnormal_edge_flags); > > df_mark_solutions_dirty (); > + fixup_partition_crossing (e); > return new_bb; > } > > @@ -1654,6 +1720,21 @@ rtl_move_block_after (basic_block bb ATTRIBUTE_UNU > return false; > } > > +/* Locate the last bb in the same partition as START_BB. */ > + > +static basic_block > +last_bb_in_partition (basic_block start_bb) > +{ > + basic_block bb; > + FOR_BB_BETWEEN (bb, start_bb, EXIT_BLOCK_PTR, next_bb) > + { > + if (BB_PARTITION (start_bb) != BB_PARTITION (bb->next_bb)) > + return bb; > + } > + /* Return bb before EXIT_BLOCK_PTR. */ > + return bb->prev_bb; > +} > + > /* Split a (typically critical) edge. Return the new block. > The edge must not be abnormal. > > @@ -1664,7 +1745,7 @@ rtl_move_block_after (basic_block bb ATTRIBUTE_UNU > static basic_block > rtl_split_edge (edge edge_in) > { > - basic_block bb; > + basic_block bb, new_bb; > rtx before; > > /* Abnormal edges cannot be split. */ > @@ -1696,13 +1777,50 @@ rtl_split_edge (edge edge_in) > } > else > { > - bb = create_basic_block (before, NULL, edge_in->dest->prev_bb); > - /* ??? Why not edge_in->dest->prev_bb here? */ > - BB_COPY_PARTITION (bb, edge_in->dest); > + if (edge_in->src == ENTRY_BLOCK_PTR) > + { > + bb = create_basic_block (before, NULL, edge_in->dest->prev_bb); > + BB_COPY_PARTITION (bb, edge_in->dest); > + } > + else > + { > + basic_block after = edge_in->dest->prev_bb; > + /* If this is post-bb reordering, and the edge crosses a partition > + boundary, the new block needs to be inserted in the bb chain > + at the end of the src partition (since we put the new bb into > + that partition, see below). Otherwise we may end up creating > + an extra partition crossing in the chain, which is illegal. > + It can't go after the src, because src may have a fall-through > + to a different block. */ > + if (crtl->bb_reorder_complete > + && (edge_in->flags & EDGE_CROSSING)) > + { > + after = last_bb_in_partition (edge_in->src); > + before = NEXT_INSN (BB_END (after)); > + /* The instruction following the last bb in partition should > + be a barrier, since it cannot end in a fall-through. */ > + gcc_checking_assert (BARRIER_P (before)); > + before = NEXT_INSN (before); > + } > + bb = create_basic_block (before, NULL, after); > + /* Put the split bb into the src partition, to avoid creating > + a situation where a cold bb dominates a hot bb, in the case > + where src is cold and dest is hot. The src will dominate > + the new bb (whereas it might not have dominated dest). */ > + BB_COPY_PARTITION (bb, edge_in->src); > + } > } > > make_single_succ_edge (bb, edge_in->dest, EDGE_FALLTHRU); > > + /* Can't allow a region crossing edge to be fallthrough. */ > + if (BB_PARTITION (bb) != BB_PARTITION (edge_in->dest) > + && edge_in->dest != EXIT_BLOCK_PTR) > + { > + new_bb = force_nonfallthru (single_succ_edge (bb)); > + gcc_assert (!new_bb); > + } > + > /* For non-fallthru edges, we must adjust the predecessor's > jump instruction to target our new block. */ > if ((edge_in->flags & EDGE_FALLTHRU) == 0) > @@ -1815,17 +1933,13 @@ commit_one_edge_insertion (edge e) > else > { > bb = split_edge (e); > - after = BB_END (bb); > > - if (flag_reorder_blocks_and_partition > - && targetm_common.have_named_sections > - && e->src != ENTRY_BLOCK_PTR > - && BB_PARTITION (e->src) == BB_COLD_PARTITION > - && !(e->flags & EDGE_CROSSING) > - && JUMP_P (after) > - && !any_condjump_p (after) > - && (single_succ_edge (bb)->flags & EDGE_CROSSING)) > - add_reg_note (after, REG_CROSSING_JUMP, NULL_RTX); > + /* If E crossed a partition boundary, we needed to make bb end in > + a region-crossing jump, even though it was originally fallthru. */ > + if (JUMP_P (BB_END (bb))) > + before = BB_END (bb); > + else > + after = BB_END (bb); > } > > /* Now that we've found the spot, do the insertion. */ > @@ -2071,7 +2185,11 @@ verify_hot_cold_block_grouping (void) > bool switched_sections = false; > int current_partition = BB_UNPARTITIONED; > > - if (!crtl->bb_reorder_complete) > + /* Even after bb reordering is complete, we go into cfglayout mode > + again (in compgoto). Ensure we don't call this before going back > + into linearized RTL when any layout fixes would have been committed. */ > + if (!crtl->bb_reorder_complete > + || current_ir_type() != IR_RTL_CFGRTL) > return err; > > FOR_EACH_BB (bb) > @@ -2116,6 +2234,7 @@ rtl_verify_edges (void) > edge e, fallthru = NULL; > edge_iterator ei; > rtx note; > + bool has_crossing_edge = false; > > if (JUMP_P (BB_END (bb)) > && (note = find_reg_note (BB_END (bb), REG_BR_PROB, NULL_RTX)) > @@ -2141,6 +2260,7 @@ rtl_verify_edges (void) > is_crossing = (BB_PARTITION (e->src) != BB_PARTITION (e->dest) > && e->src != ENTRY_BLOCK_PTR > && e->dest != EXIT_BLOCK_PTR); > + has_crossing_edge |= is_crossing; > if (e->flags & EDGE_CROSSING) > { > if (!is_crossing) > @@ -2160,6 +2280,13 @@ rtl_verify_edges (void) > e->src->index); > err = 1; > } > + if (JUMP_P (BB_END (bb)) > + && !find_reg_note (BB_END (bb), REG_CROSSING_JUMP, > NULL_RTX)) > + { > + error ("No region crossing jump at section boundary in bb %i", > + bb->index); > + err = 1; > + } > } > else if (is_crossing) > { > @@ -2188,6 +2315,15 @@ rtl_verify_edges (void) > n_abnormal++; > } > > + if (!has_crossing_edge > + && find_reg_note (BB_END (bb), REG_CROSSING_JUMP, NULL_RTX)) > + { > + print_rtl_with_bb (stderr, get_insns (), TDF_RTL | > TDF_BLOCKS | TDF_DETAILS); > + error ("Region crossing jump across same section in bb %i", > + bb->index); > + err = 1; > + } > + > if (n_eh && !find_reg_note (BB_END (bb), REG_EH_REGION, NULL_RTX)) > { > error ("missing REG_EH_REGION note at the end of bb %i", bb->index); > @@ -2395,8 +2531,6 @@ rtl_verify_flow_info_1 (void) > > err |= rtl_verify_edges (); > > - err |= verify_hot_cold_block_grouping(); > - > return err; > } > > @@ -2642,6 +2776,8 @@ rtl_verify_flow_info (void) > > err |= rtl_verify_bb_layout (); > > + err |= verify_hot_cold_block_grouping (); > + > return err; > } > > @@ -3343,7 +3479,7 @@ fixup_reorder_chain (void) > edge e_fall, e_taken, e; > rtx bb_end_insn; > rtx ret_label = NULL_RTX; > - basic_block nb, src_bb; > + basic_block nb; > edge_iterator ei; > > if (EDGE_COUNT (bb->succs) == 0) > @@ -3478,7 +3614,6 @@ fixup_reorder_chain (void) > /* We got here if we need to add a new jump insn. > Note force_nonfallthru can delete E_FALL and thus we have to > save E_FALL->src prior to the call to force_nonfallthru. */ > - src_bb = e_fall->src; > nb = force_nonfallthru_and_redirect (e_fall, e_fall->dest, ret_label); > if (nb) > { > @@ -3486,17 +3621,6 @@ fixup_reorder_chain (void) > bb->aux = nb; > /* Don't process this new block. */ > bb = nb; > - > - /* Make sure new bb is tagged for correct section (same as > - fall-thru source, since you cannot fall-thru across > - section boundaries). */ > - BB_COPY_PARTITION (src_bb, single_pred (bb)); > - if (flag_reorder_blocks_and_partition > - && targetm_common.have_named_sections > - && JUMP_P (BB_END (bb)) > - && !any_condjump_p (BB_END (bb)) > - && (EDGE_SUCC (bb, 0)->flags & EDGE_CROSSING)) > - add_reg_note (BB_END (bb), REG_CROSSING_JUMP, NULL_RTX); > } > } > > @@ -3796,10 +3920,11 @@ duplicate_insn_chain (rtx from, rtx to) > case NOTE_INSN_FUNCTION_BEG: > /* There is always just single entry to function. */ > case NOTE_INSN_BASIC_BLOCK: > + /* We should only switch text sections once. */ > + case NOTE_INSN_SWITCH_TEXT_SECTIONS: > break; > > case NOTE_INSN_EPILOGUE_BEG: > - case NOTE_INSN_SWITCH_TEXT_SECTIONS: > emit_note_copy (insn); > break; > > @@ -4611,8 +4736,7 @@ rtl_can_remove_branch_p (const_edge e) > if (e->flags & (EDGE_ABNORMAL_CALL | EDGE_EH)) > return false; > > - if (find_reg_note (insn, REG_CROSSING_JUMP, NULL_RTX) > - || BB_PARTITION (src) != BB_PARTITION (target)) > + if (BB_PARTITION (src) != BB_PARTITION (target)) > return false; > > if (!onlyjump_p (insn) > Index: basic-block.h > =================================================================== > --- basic-block.h (revision 199014) > +++ basic-block.h (working copy) > @@ -796,6 +796,7 @@ extern basic_block force_nonfallthru_and_redirect > extern bool contains_no_active_insn_p (const_basic_block); > extern bool forwarder_block_p (const_basic_block); > extern bool can_fallthru (basic_block, basic_block); > +extern void emit_barrier_after_bb (basic_block bb); > > /* In cfgbuild.c. */ > extern void find_many_sub_basic_blocks (sbitmap); > Index: testsuite/gcc.dg/tree-prof/va-arg-pack-1.c > =================================================================== > --- testsuite/gcc.dg/tree-prof/va-arg-pack-1.c (revision 0) > +++ testsuite/gcc.dg/tree-prof/va-arg-pack-1.c (revision 0) > @@ -0,0 +1,145 @@ > +/* __builtin_va_arg_pack () builtin tests. */ > +/* { dg-require-effective-target freorder } */ > +/* { dg-options "-O2 -freorder-blocks-and-partition" } */ > + > +#include <stdarg.h> > + > +extern void abort (void); > + > +int v1 = 8; > +long int v2 = 3; > +void *v3 = (void *) &v2; > +struct A { char c[16]; } v4 = { "foo" }; > +long double v5 = 40; > +char seen[20]; > +int cnt; > + > +__attribute__ ((noinline)) int > +foo1 (int x, int y, ...) > +{ > + int i; > + long int l; > + void *v; > + struct A a; > + long double ld; > + va_list ap; > + > + va_start (ap, y); > + if (x < 0 || x >= 20 || seen[x]) > + abort (); > + seen[x] = ++cnt; > + if (y != 6) > + abort (); > + i = va_arg (ap, int); > + if (i != 5) > + abort (); > + switch (x) > + { > + case 0: > + i = va_arg (ap, int); > + if (i != 9 || v1 != 9) > + abort (); > + a = va_arg (ap, struct A); > + if (__builtin_memcmp (a.c, v4.c, sizeof (a.c)) != 0) > + abort (); > + v = (void *) va_arg (ap, struct A *); > + if (v != (void *) &v4) > + abort (); > + l = va_arg (ap, long int); > + if (l != 3 || v2 != 4) > + abort (); > + break; > + case 1: > + ld = va_arg (ap, long double); > + if (ld != 41 || v5 != ld) > + abort (); > + i = va_arg (ap, int); > + if (i != 8) > + abort (); > + v = va_arg (ap, void *); > + if (v != &v2) > + abort (); > + break; > + case 2: > + break; > + default: > + abort (); > + } > + va_end (ap); > + return x; > +} > + > +__attribute__ ((noinline)) int > +foo2 (int x, int y, ...) > +{ > + long long int ll; > + void *v; > + struct A a, b; > + long double ld; > + va_list ap; > + > + va_start (ap, y); > + if (x < 0 || x >= 20 || seen[x]) > + abort (); > + seen[x] = ++cnt | 64; > + if (y != 10) > + abort (); > + switch (x) > + { > + case 11: > + break; > + case 12: > + ld = va_arg (ap, long double); > + if (ld != 41 || v5 != 40) > + abort (); > + a = va_arg (ap, struct A); > + if (__builtin_memcmp (a.c, v4.c, sizeof (a.c)) != 0) > + abort (); > + b = va_arg (ap, struct A); > + if (__builtin_memcmp (b.c, v4.c, sizeof (b.c)) != 0) > + abort (); > + v = va_arg (ap, void *); > + if (v != &v2) > + abort (); > + ll = va_arg (ap, long long int); > + if (ll != 16LL) > + abort (); > + break; > + case 2: > + break; > + default: > + abort (); > + } > + va_end (ap); > + return x + 8; > +} > + > +__attribute__ ((noinline)) int > +foo3 (void) > +{ > + return 6; > +} > + > +extern inline __attribute__ ((always_inline, gnu_inline)) int > +bar (int x, ...) > +{ > + if (x < 10) > + return foo1 (x, foo3 (), 5, __builtin_va_arg_pack ()); > + return foo2 (x, foo3 () + 4, __builtin_va_arg_pack ()); > +} > + > +int > +main (void) > +{ > + if (bar (0, ++v1, v4, &v4, v2++) != 0) > + abort (); > + if (bar (1, ++v5, 8, v3) != 1) > + abort (); > + if (bar (2) != 2) > + abort (); > + if (bar (v1 + 2) != 19) > + abort (); > + if (bar (v1 + 3, v5--, v4, v4, v3, 16LL) != 20) > + abort (); > + return 0; > +} > Index: testsuite/gcc.dg/tree-prof/comp-goto-1.c > =================================================================== > --- testsuite/gcc.dg/tree-prof/comp-goto-1.c (revision 0) > +++ testsuite/gcc.dg/tree-prof/comp-goto-1.c (revision 0) > @@ -0,0 +1,166 @@ > +/* { dg-require-effective-target freorder } */ > +/* { dg-options "-O2 -freorder-blocks-and-partition" } */ > +#include <stdlib.h> > + > +#if !defined(NO_LABEL_VALUES) && (!defined(STACK_SIZE) || STACK_SIZE >>= 4000) && __INT_MAX__ >= 2147483647 > +typedef unsigned int uint32; > +typedef signed int sint32; > + > +typedef uint32 reg_t; > + > +typedef unsigned long int host_addr_t; > +typedef uint32 target_addr_t; > +typedef sint32 target_saddr_t; > + > +typedef union > +{ > + struct > + { > + unsigned int offset:18; > + unsigned int ignore:4; > + unsigned int s1:8; > + int :2; > + signed int simm:14; > + unsigned int s3:8; > + unsigned int s2:8; > + int pad2:2; > + } f1; > + long long ll; > + double d; > +} insn_t; > + > +typedef struct > +{ > + target_addr_t vaddr_tag; > + unsigned long int rigged_paddr; > +} tlb_entry_t; > + > +typedef struct > +{ > + insn_t *pc; > + reg_t registers[256]; > + insn_t *program; > + tlb_entry_t tlb_tab[0x100]; > +} environment_t; > + > +enum operations > +{ > + LOAD32_RR, > + METAOP_DONE > +}; > + > +host_addr_t > +f () > +{ > + abort (); > +} > + > +reg_t > +simulator_kernel (int what, environment_t *env) > +{ > + register insn_t *pc = env->pc; > + register reg_t *regs = env->registers; > + register insn_t insn; > + register int s1; > + register reg_t r2; > + register void *base_addr = &&sim_base_addr; > + register tlb_entry_t *tlb = env->tlb_tab; > + > + if (what != 0) > + { > + int i; > + static void *op_map[] = > + { > + &&L_LOAD32_RR, > + &&L_METAOP_DONE, > + }; > + insn_t *program = env->program; > + for (i = 0; i < what; i++) > + program[i].f1.offset = op_map[program[i].f1.offset] - base_addr; > + } > + > + sim_base_addr:; > + > + insn = *pc++; > + r2 = (*(reg_t *) (((char *) regs) + (insn.f1.s2 << 2))); > + s1 = (insn.f1.s1 << 2); > + goto *(base_addr + insn.f1.offset); > + > + L_LOAD32_RR: > + { > + target_addr_t vaddr_page = r2 / 4096; > + unsigned int x = vaddr_page % 0x100; > + insn = *pc++; > + > + for (;;) > + { > + target_addr_t tag = tlb[x].vaddr_tag; > + host_addr_t rigged_paddr = tlb[x].rigged_paddr; > + > + if (tag == vaddr_page) > + { > + *(reg_t *) (((char *) regs) + s1) = *(uint32 *) (rigged_paddr + r2); > + r2 = *(reg_t *) (((char *) regs) + (insn.f1.s2 << 2)); > + s1 = insn.f1.s1 << 2; > + goto *(base_addr + insn.f1.offset); > + } > + > + if (((target_saddr_t) tag < 0)) > + { > + *(reg_t *) (((char *) regs) + s1) = *(uint32 *) f (); > + r2 = *(reg_t *) (((char *) regs) + (insn.f1.s2 << 2)); > + s1 = insn.f1.s1 << 2; > + goto *(base_addr + insn.f1.offset); > + } > + > + x = (x - 1) % 0x100; > + } > + > + L_METAOP_DONE: > + return (*(reg_t *) (((char *) regs) + s1)); > + } > +} > + > +insn_t program[2 + 1]; > + > +void *malloc (); > + > +int > +main () > +{ > + environment_t env; > + insn_t insn; > + int i, res; > + host_addr_t a_page = (host_addr_t) malloc (2 * 4096); > + target_addr_t a_vaddr = 0x123450; > + target_addr_t vaddr_page = a_vaddr / 4096; > + a_page = (a_page + 4096 - 1) & -4096; > + > + env.tlb_tab[((vaddr_page) % 0x100)].vaddr_tag = vaddr_page; > + env.tlb_tab[((vaddr_page) % 0x100)].rigged_paddr = a_page - > vaddr_page * 4096; > + insn.f1.offset = LOAD32_RR; > + env.registers[0] = 0; > + env.registers[2] = a_vaddr; > + *(sint32 *) (a_page + a_vaddr % 4096) = 88; > + insn.f1.s1 = 0; > + insn.f1.s2 = 2; > + > + for (i = 0; i < 2; i++) > + program[i] = insn; > + > + insn.f1.offset = METAOP_DONE; > + insn.f1.s1 = 0; > + program[2] = insn; > + > + env.pc = program; > + env.program = program; > + > + res = simulator_kernel (2 + 1, &env); > + > + if (res != 88) > + abort (); > + exit (0); > +} > +#else > +main(){ exit (0); } > +#endif > Index: testsuite/gcc.dg/tree-prof/pr52027.c > =================================================================== > --- testsuite/gcc.dg/tree-prof/pr52027.c (revision 199014) > +++ testsuite/gcc.dg/tree-prof/pr52027.c (working copy) > @@ -1,6 +1,6 @@ > /* PR debug/52027 */ > /* { dg-require-effective-target freorder } */ > -/* { dg-options "-O -freorder-blocks-and-partition -fno-reorder-functions" } > */ > +/* { dg-options "-O2 -freorder-blocks-and-partition > -fno-reorder-functions" } */ > > void > foo (int len) > Index: testsuite/gcc.dg/tree-prof/pr50907.c > =================================================================== > --- testsuite/gcc.dg/tree-prof/pr50907.c (revision 199014) > +++ testsuite/gcc.dg/tree-prof/pr50907.c (working copy) > @@ -1,5 +1,5 @@ > /* PR middle-end/50907 */ > /* { dg-require-effective-target freorder } */ > -/* { dg-options "-O -freorder-blocks-and-partition -fschedule-insns > -fselective-scheduling -fpic" { target { { powerpc*-*-* ia64-*-* > x86_64-*-* } && fpic } } } */ > +/* { dg-options "-O2 -freorder-blocks-and-partition -fschedule-insns > -fselective-scheduling -fpic" { target { { powerpc*-*-* ia64-*-* > x86_64-*-* } && fpic } } } */ > > #include "pr45354.c" > Index: testsuite/gcc.dg/tree-prof/pr45354.c > =================================================================== > --- testsuite/gcc.dg/tree-prof/pr45354.c (revision 199014) > +++ testsuite/gcc.dg/tree-prof/pr45354.c (working copy) > @@ -1,5 +1,5 @@ > /* { dg-require-effective-target freorder } */ > -/* { dg-options "-O -freorder-blocks-and-partition -fschedule-insns > -fselective-scheduling" { target powerpc*-*-* ia64-*-* x86_64-*-* } } > */ > +/* { dg-options "-O2 -freorder-blocks-and-partition -fschedule-insns > -fselective-scheduling" { target powerpc*-*-* ia64-*-* x86_64-*-* } } > */ > > extern void abort (void); > > Index: testsuite/gcc.dg/tree-prof/20041218-1.c > =================================================================== > --- testsuite/gcc.dg/tree-prof/20041218-1.c (revision 0) > +++ testsuite/gcc.dg/tree-prof/20041218-1.c (revision 0) > @@ -0,0 +1,119 @@ > +/* PR rtl-optimization/16968 */ > +/* Testcase by Jakub Jelinek <ja...@redhat.com> */ > +/* { dg-require-effective-target freorder } */ > +/* { dg-options "-O2 -freorder-blocks-and-partition" } */ > + > +struct T > +{ > + unsigned int b, c, *d; > + unsigned char e; > +}; > +struct S > +{ > + unsigned int a; > + struct T f; > +}; > +struct U > +{ > + struct S g, h; > +}; > +struct V > +{ > + unsigned int i; > + struct U j; > +}; > + > +extern void exit (int); > +extern void abort (void); > + > +void * > +dummy1 (void *x) > +{ > + return ""; > +} > + > +void * > +dummy2 (void *x, void *y) > +{ > + exit (0); > +} > + > +struct V * > +baz (unsigned int x) > +{ > + static struct V v; > + __builtin_memset (&v, 0x55, sizeof (v)); > + return &v; > +} > + > +int > +check (void *x, struct S *y) > +{ > + if (y->a || y->f.b || y->f.c || y->f.d || y->f.e) > + abort (); > + return 1; > +} > + > +static struct V * > +bar (unsigned int x, void *y) > +{ > + const struct T t = { 0, 0, (void *) 0, 0 }; > + struct V *u; > + void *v; > + v = dummy1 (y); > + if (!v) > + return (void *) 0; > + > + u = baz (sizeof (struct V)); > + u->i = x; > + u->j.g.a = 0; > + u->j.g.f = t; > + u->j.h.a = 0; > + u->j.h.f = t; > + > + if (!check (v, &u->j.g) || !check (v, &u->j.h)) > + return (void *) 0; > + return u; > +} > + > +int > +foo (unsigned int *x, unsigned int y, void **z) > +{ > + void *v; > + unsigned int i, j; > + > + *z = v = (void *) 0; > + > + for (i = 0; i < y; i++) > + { > + struct V *c; > + > + j = *x; > + > + switch (j) > + { > + case 1: > + c = bar (j, x); > + break; > + default: > + c = 0; > + break; > + } > + if (c) > + v = dummy2 (v, c); > + else > + return 1; > + } > + > + *z = v; > + return 0; > +} > + > +int > +main (void) > +{ > + unsigned int one = 1; > + void *p; > + foo (&one, 1, &p); > + abort (); > +} > Index: testsuite/g++.dg/tree-prof/partition2.C > =================================================================== > --- testsuite/g++.dg/tree-prof/partition2.C (revision 199014) > +++ testsuite/g++.dg/tree-prof/partition2.C (working copy) > @@ -1,6 +1,6 @@ > // PR middle-end/45458 > // { dg-require-effective-target freorder } > -// { dg-options "-fnon-call-exceptions -freorder-blocks-and-partition" } > +// { dg-options "-O2 -fnon-call-exceptions -freorder-blocks-and-partition" } > > int > main () > Index: testsuite/g++.dg/tree-prof/partition3.C > =================================================================== > --- testsuite/g++.dg/tree-prof/partition3.C (revision 199014) > +++ testsuite/g++.dg/tree-prof/partition3.C (working copy) > @@ -1,6 +1,6 @@ > // PR middle-end/45566 > // { dg-require-effective-target freorder } > -// { dg-options "-O -fnon-call-exceptions -freorder-blocks-and-partition" } > +// { dg-options "-O2 -fnon-call-exceptions -freorder-blocks-and-partition" } > > int k; > > > > > -- > Teresa Johnson | Software Engineer | tejohn...@google.com | 408-460-2413 -- Teresa Johnson | Software Engineer | tejohn...@google.com | 408-460-2413
