> Hi! > > Here is an updated patch for what Tobias has posted earlier: > http://gcc.gnu.org/ml/gcc-patches/2014-03/msg00043.html > While that version bootstrapped/regtested fine, most of the Fortran > tests ICEd, primarily because the 3 operand __builtin_expect wasn't being > removed from the IL and for expansion we only allow it for !optimize or > couple of similar cases. > > This (combined) patch fixes that, fixes a couple of if (*predictor) to > if (predictor),. The biggest change is to introduce IFN_BUILTIN_EXPECT, > because as we don't want to allow user code to specify 3+ argument > __builtin_expect form, we probably don't want to make the builtin prototype > a varargs function, but in that case it means e.g. gimple_builtin_p (stmt, > BUILT_IN_EXPECT) will never match the 3 operand __builtin_expect. > Also, predict.c would happily predict that &__gthrw___pthread_key_create != 0 > is PRED_UNCONDITIONALly true (and also that &__gthrw___pthread_key_create == 0 > is PRED_UNCONDITIONALly true), that is just wrong. > > I wanted to minimize the amount of changes for 4.9, so this patch only uses > the internal fn for the 3 operand __builtin_expect, after branching I'd like > to use it always and remove handling of non-internal __builtin_expect after > gimplification. The advantage could be e.g. that the argument/return value > doesn't have to be necessarily long, we could just fold it at gimplification > time. > > The predict.c changes affect inlining in libstdc++-v3/src/c++11/thread.cc > somewhat, so it is not inlining one ctor any longer, Jonathan has kindly > committed a gnu.ver fix for that yesterday. > > Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? > > 2014-03-15 Jakub Jelinek <ja...@redhat.com> > > PR ipa/58721 > gcc/ > * internal-fn.c: Include diagnostic-core.h. > (expand_BUILTIN_EXPECT): New function. > * gimplify.c (gimplify_call_expr): Use false instead of FALSE. > (gimplify_modify_expr): Gimplify 3 argument __builtin_expect into > IFN_BUILTIN_EXPECT call instead of __builtin_expect builtin call. > * ipa-inline-analysis.c (find_foldable_builtin_expect): Handle > IFN_BUILTIN_EXPECT. > * predict.c (expr_expected_value_1): Handle IFN_BUILTIN_EXPECT. > Revert 3 argument __builtin_expect code. > (strip_predict_hints): Handle IFN_BUILTIN_EXPECT. > * gimple-fold.c (gimple_fold_call): Likewise. > * tree.h (fold_builtin_expect): New prototype. > * builtins.c (build_builtin_expect_predicate): Add predictor > argument, if non-NULL, create 3 argument __builtin_expect. > (fold_builtin_expect): No longer static. Add ARG2 argument, > pass it through to build_builtin_expect_predicate. > (fold_builtin_2): Adjust caller. > (fold_builtin_3): Handle BUILT_IN_EXPECT. > * internal-fn.def (BUILTIN_EXPECT): New. > gcc/fortran/ > * trans.c (gfc_unlikely, gfc_likely): Don't add __builtin_expect > if !optimize.
Thank you for looking into this! The branch prediction changes are OK. I however wonder how the inlining decisions can be changed on libstdc++? Is it the compare_and_swap/unconditional change? Honza > > 2014-03-15 Tobias Burnus <bur...@net-b.de> > > PR ipa/58721 > gcc/ > * predict.def (PRED_FORTRAN_OVERFLOW, PRED_FORTRAN_FAIL_ALLOC, > PRED_FORTRAN_FAIL_IO, PRED_FORTRAN_WARN_ONCE, PRED_FORTRAN_SIZE_ZERO, > PRED_FORTRAN_INVALID_BOUND, PRED_FORTRAN_ABSENT_DUMMY): Add. > gcc/fortran/ > * trans.h (gfc_unlikely, gfc_likely): Add predictor as argument. > (gfc_trans_io_runtime_check): Remove. > * trans-io.c (gfc_trans_io_runtime_check): Make static; add has_iostat > as argument, add predictor to block. > (set_parameter_value, gfc_trans_open, gfc_trans_close, build_filepos, > gfc_trans_inquire, gfc_trans_wait, build_dt): Update calls. > * trans.c (gfc_unlikely, gfc_likely): Add predictor as argument. > (gfc_trans_runtime_check, gfc_allocate_using_malloc, > gfc_allocate_allocatable, gfc_deallocate_with_status): Set explicitly > branch predictor. > * trans-expr.c (gfc_conv_procedure_call): Ditto. > * trans-stmt.c (gfc_trans_allocate): Ditto. > * trans-array.c (gfc_array_init_size, gfc_array_allocate): Ditto. > > 2014-03-15 Jan Hubicka <hubi...@ucw.cz> > > PR ipa/58721 > gcc/ > * predict.c (combine_predictions_for_bb): Fix up formatting. > (expr_expected_value_1, expr_expected_value): Add predictor argument, > fill what it points to if non-NULL. > (tree_predict_by_opcode): Adjust caller, use the predictor. > * predict.def (PRED_COMPARE_AND_SWAP): Add. > > --- gcc/predict.c.jj 2014-01-03 11:40:46.957378605 +0100 > +++ gcc/predict.c 2014-03-14 13:16:15.246017052 +0100 > @@ -956,7 +956,8 @@ combine_predictions_for_bb (basic_block > struct edge_prediction *pred2; > int prob = probability; > > - for (pred2 = (struct edge_prediction *) *preds; pred2; pred2 = > pred2->ep_next) > + for (pred2 = (struct edge_prediction *) *preds; > + pred2; pred2 = pred2->ep_next) > if (pred2 != pred && pred2->ep_predictor == pred->ep_predictor) > { > int probability2 = pred->ep_probability; > @@ -1788,16 +1789,19 @@ guess_outgoing_edge_probabilities (basic > combine_predictions_for_insn (BB_END (bb), bb); > } > > -static tree expr_expected_value (tree, bitmap); > +static tree expr_expected_value (tree, bitmap, enum br_predictor *predictor); > > /* Helper function for expr_expected_value. */ > > static tree > expr_expected_value_1 (tree type, tree op0, enum tree_code code, > - tree op1, bitmap visited) > + tree op1, bitmap visited, enum br_predictor *predictor) > { > gimple def; > > + if (predictor) > + *predictor = PRED_UNCONDITIONAL; > + > if (get_gimple_rhs_class (code) == GIMPLE_SINGLE_RHS) > { > if (TREE_CONSTANT (op0)) > @@ -1822,6 +1826,7 @@ expr_expected_value_1 (tree type, tree o > for (i = 0; i < n; i++) > { > tree arg = PHI_ARG_DEF (def, i); > + enum br_predictor predictor2; > > /* If this PHI has itself as an argument, we cannot > determine the string length of this argument. However, > @@ -1832,7 +1837,12 @@ expr_expected_value_1 (tree type, tree o > if (arg == PHI_RESULT (def)) > continue; > > - new_val = expr_expected_value (arg, visited); > + new_val = expr_expected_value (arg, visited, &predictor2); > + > + /* It is difficult to combine value predictors. Simply assume > + that later predictor is weaker and take its prediction. */ > + if (predictor && *predictor < predictor2) > + *predictor = predictor2; > if (!new_val) > return NULL; > if (!val) > @@ -1851,14 +1861,34 @@ expr_expected_value_1 (tree type, tree o > gimple_assign_rhs1 (def), > gimple_assign_rhs_code (def), > gimple_assign_rhs2 (def), > - visited); > + visited, predictor); > } > > if (is_gimple_call (def)) > { > tree decl = gimple_call_fndecl (def); > if (!decl) > - return NULL; > + { > + if (gimple_call_internal_p (def) > + && gimple_call_internal_fn (def) == IFN_BUILTIN_EXPECT) > + { > + gcc_assert (gimple_call_num_args (def) == 3); > + tree val = gimple_call_arg (def, 0); > + if (TREE_CONSTANT (val)) > + return val; > + if (predictor) > + { > + *predictor = PRED_BUILTIN_EXPECT; > + tree val2 = gimple_call_arg (def, 2); > + gcc_assert (TREE_CODE (val2) == INTEGER_CST > + && tree_fits_uhwi_p (val2) > + && tree_to_uhwi (val2) < END_PREDICTORS); > + *predictor = (enum br_predictor) tree_to_uhwi (val2); > + } > + return gimple_call_arg (def, 1); > + } > + return NULL; > + } > if (DECL_BUILT_IN_CLASS (decl) == BUILT_IN_NORMAL) > switch (DECL_FUNCTION_CODE (decl)) > { > @@ -1870,6 +1900,8 @@ expr_expected_value_1 (tree type, tree o > val = gimple_call_arg (def, 0); > if (TREE_CONSTANT (val)) > return val; > + if (predictor) > + *predictor = PRED_BUILTIN_EXPECT; > return gimple_call_arg (def, 1); > } > > @@ -1888,6 +1920,8 @@ expr_expected_value_1 (tree type, tree o > case BUILT_IN_ATOMIC_COMPARE_EXCHANGE_16: > /* Assume that any given atomic operation has low contention, > and thus the compare-and-swap operation succeeds. */ > + if (predictor) > + *predictor = PRED_COMPARE_AND_SWAP; > return boolean_true_node; > } > } > @@ -1898,10 +1932,13 @@ expr_expected_value_1 (tree type, tree o > if (get_gimple_rhs_class (code) == GIMPLE_BINARY_RHS) > { > tree res; > - op0 = expr_expected_value (op0, visited); > + enum br_predictor predictor2; > + op0 = expr_expected_value (op0, visited, predictor); > if (!op0) > return NULL; > - op1 = expr_expected_value (op1, visited); > + op1 = expr_expected_value (op1, visited, &predictor2); > + if (predictor && *predictor < predictor2) > + *predictor = predictor2; > if (!op1) > return NULL; > res = fold_build2 (code, type, op0, op1); > @@ -1912,7 +1949,7 @@ expr_expected_value_1 (tree type, tree o > if (get_gimple_rhs_class (code) == GIMPLE_UNARY_RHS) > { > tree res; > - op0 = expr_expected_value (op0, visited); > + op0 = expr_expected_value (op0, visited, predictor); > if (!op0) > return NULL; > res = fold_build1 (code, type, op0); > @@ -1932,17 +1969,22 @@ expr_expected_value_1 (tree type, tree o > implementation. */ > > static tree > -expr_expected_value (tree expr, bitmap visited) > +expr_expected_value (tree expr, bitmap visited, > + enum br_predictor *predictor) > { > enum tree_code code; > tree op0, op1; > > if (TREE_CONSTANT (expr)) > - return expr; > + { > + if (predictor) > + *predictor = PRED_UNCONDITIONAL; > + return expr; > + } > > extract_ops_from_tree (expr, &code, &op0, &op1); > return expr_expected_value_1 (TREE_TYPE (expr), > - op0, code, op1, visited); > + op0, code, op1, visited, predictor); > } > > > @@ -1967,14 +2009,16 @@ strip_predict_hints (void) > gsi_remove (&bi, true); > continue; > } > - else if (gimple_code (stmt) == GIMPLE_CALL) > + else if (is_gimple_call (stmt)) > { > tree fndecl = gimple_call_fndecl (stmt); > > - if (fndecl > - && DECL_BUILT_IN_CLASS (fndecl) == BUILT_IN_NORMAL > - && DECL_FUNCTION_CODE (fndecl) == BUILT_IN_EXPECT > - && gimple_call_num_args (stmt) == 2) > + if ((fndecl > + && DECL_BUILT_IN_CLASS (fndecl) == BUILT_IN_NORMAL > + && DECL_FUNCTION_CODE (fndecl) == BUILT_IN_EXPECT > + && gimple_call_num_args (stmt) == 2) > + || (gimple_call_internal_p (stmt) > + && gimple_call_internal_fn (stmt) == IFN_BUILTIN_EXPECT)) > { > var = gimple_call_lhs (stmt); > if (var) > @@ -2008,6 +2052,7 @@ tree_predict_by_opcode (basic_block bb) > enum tree_code cmp; > bitmap visited; > edge_iterator ei; > + enum br_predictor predictor; > > if (!stmt || gimple_code (stmt) != GIMPLE_COND) > return; > @@ -2019,16 +2064,23 @@ tree_predict_by_opcode (basic_block bb) > cmp = gimple_cond_code (stmt); > type = TREE_TYPE (op0); > visited = BITMAP_ALLOC (NULL); > - val = expr_expected_value_1 (boolean_type_node, op0, cmp, op1, visited); > + val = expr_expected_value_1 (boolean_type_node, op0, cmp, op1, visited, > + &predictor); > BITMAP_FREE (visited); > - if (val) > + if (val && TREE_CODE (val) == INTEGER_CST) > { > - int percent = PARAM_VALUE (BUILTIN_EXPECT_PROBABILITY); > + if (predictor == PRED_BUILTIN_EXPECT) > + { > + int percent = PARAM_VALUE (BUILTIN_EXPECT_PROBABILITY); > > - gcc_assert (percent >= 0 && percent <= 100); > - if (integer_zerop (val)) > - percent = 100 - percent; > - predict_edge (then_edge, PRED_BUILTIN_EXPECT, HITRATE (percent)); > + gcc_assert (percent >= 0 && percent <= 100); > + if (integer_zerop (val)) > + percent = 100 - percent; > + predict_edge (then_edge, PRED_BUILTIN_EXPECT, HITRATE (percent)); > + } > + else > + predict_edge (then_edge, predictor, > + integer_zerop (val) ? NOT_TAKEN : TAKEN); > } > /* Try "pointer heuristic." > A comparison ptr == 0 is predicted as false. > --- gcc/predict.def.jj 2014-01-03 11:40:57.220320576 +0100 > +++ gcc/predict.def 2014-03-14 10:12:26.255478253 +0100 > @@ -57,6 +57,11 @@ DEF_PREDICTOR (PRED_UNCONDITIONAL, "unco > DEF_PREDICTOR (PRED_LOOP_ITERATIONS, "loop iterations", PROB_ALWAYS, > PRED_FLAG_FIRST_MATCH) > > +/* Assume that any given atomic operation has low contention, > + and thus the compare-and-swap operation succeeds. */ > +DEF_PREDICTOR (PRED_COMPARE_AND_SWAP, "compare and swap", PROB_VERY_LIKELY, > + PRED_FLAG_FIRST_MATCH) > + > /* Hints dropped by user via __builtin_expect feature. Note: the > probability of PROB_VERY_LIKELY is now overwritten by param > builtin_expect_probability with a default value of HITRATE(90). > @@ -133,3 +138,41 @@ DEF_PREDICTOR (PRED_HOT_LABEL, "hot labe > /* Branches to cold labels are extremely unlikely. */ > DEF_PREDICTOR (PRED_COLD_LABEL, "cold label", PROB_VERY_LIKELY, > PRED_FLAG_FIRST_MATCH) > + > + > +/* The following predictors are used in Fortran. */ > + > +/* Branch leading to an integer overflow are extremely unlikely. */ > +DEF_PREDICTOR (PRED_FORTRAN_OVERFLOW, "overflow", PROB_ALWAYS, > + PRED_FLAG_FIRST_MATCH) > + > +/* Branch leading to a failure status are unlikely. This can occur for out > + of memory or when trying to allocate an already allocated allocated or > + deallocating an already deallocated allocatable. This predictor only > + occurs when the user explicitly asked for a return status. By default, > + the code aborts, which is handled via PRED_NORETURN. */ > +DEF_PREDICTOR (PRED_FORTRAN_FAIL_ALLOC, "fail alloc", PROB_VERY_LIKELY, 0) > + > +/* Branch leading to an I/O failure status are unlikely. This predictor is > + used for I/O failures such as for invalid unit numbers. This predictor > + only occurs when the user explicitly asked for a return status. By > default, > + the code aborts, which is handled via PRED_NORETURN. */ > +DEF_PREDICTOR (PRED_FORTRAN_FAIL_IO, "fail alloc", HITRATE(85), 0) > + > +/* Branch leading to a run-time warning message which is printed only once > + are unlikely. The print-warning branch itself can be likely or unlikely. > */ > +DEF_PREDICTOR (PRED_FORTRAN_WARN_ONCE, "warn once", HITRATE (75), 0) > + > +/* Branch belonging to a zero-sized array. */ > +DEF_PREDICTOR (PRED_FORTRAN_SIZE_ZERO, "zero-sized array", HITRATE(70), 0) > + > +/* Branch belonging to an invalid bound index, in a context where it is > + standard conform and well defined but rather pointless and, hence, rather > + unlikely to occur. */ > +DEF_PREDICTOR (PRED_FORTRAN_INVALID_BOUND, "zero-sized array", HITRATE(90), > 0) > + > +/* Branch belonging to the handling of absent optional arguments. This > + predictor is used when an optional dummy argument, associated with an > + absent argument, is passed on as actual argument to another procedure, > + which in turn has an optional argument. */ > +DEF_PREDICTOR (PRED_FORTRAN_ABSENT_DUMMY, "absent dummy", HITRATE(60), 0) > --- gcc/internal-fn.c.jj 2014-02-06 22:54:01.000000000 +0100 > +++ gcc/internal-fn.c 2014-03-14 13:05:28.276790384 +0100 > @@ -36,6 +36,7 @@ along with GCC; see the file COPYING3. > #include "predict.h" > #include "stringpool.h" > #include "tree-ssanames.h" > +#include "diagnostic-core.h" > > /* The names of each internal function, indexed by function number. */ > const char *const internal_fn_name_array[] = { > @@ -865,6 +866,23 @@ expand_ABNORMAL_DISPATCHER (gimple) > { > } > > +static void > +expand_BUILTIN_EXPECT (gimple stmt) > +{ > + /* When guessing was done, the hints should be already stripped away. */ > + gcc_assert (!flag_guess_branch_prob || optimize == 0 || seen_error ()); > + > + rtx target; > + tree lhs = gimple_call_lhs (stmt); > + if (lhs) > + target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); > + else > + target = const0_rtx; > + rtx val = expand_expr (gimple_call_arg (stmt, 0), target, VOIDmode, > EXPAND_NORMAL); > + if (lhs && val != target) > + emit_move_insn (target, val); > +} > + > /* Routines to expand each internal function, indexed by function number. > Each routine has the prototype: > > --- gcc/gimplify.c.jj 2014-03-06 13:05:17.000000000 +0100 > +++ gcc/gimplify.c 2014-03-14 13:03:22.017522169 +0100 > @@ -2215,7 +2215,7 @@ gimplify_call_expr (tree *expr_p, gimple > enum gimplify_status ret; > int i, nargs; > gimple call; > - bool builtin_va_start_p = FALSE; > + bool builtin_va_start_p = false; > location_t loc = EXPR_LOCATION (*expr_p); > > gcc_assert (TREE_CODE (*expr_p) == CALL_EXPR); > @@ -4566,8 +4566,20 @@ gimplify_modify_expr (tree *expr_p, gimp > tree fnptrtype = TREE_TYPE (CALL_EXPR_FN (*from_p)); > CALL_EXPR_FN (*from_p) = TREE_OPERAND (CALL_EXPR_FN (*from_p), 0); > STRIP_USELESS_TYPE_CONVERSION (CALL_EXPR_FN (*from_p)); > - assign = gimple_build_call_from_tree (*from_p); > - gimple_call_set_fntype (assign, TREE_TYPE (fnptrtype)); > + tree fndecl = get_callee_fndecl (*from_p); > + if (fndecl > + && DECL_BUILT_IN_CLASS (fndecl) == BUILT_IN_NORMAL > + && DECL_FUNCTION_CODE (fndecl) == BUILT_IN_EXPECT > + && call_expr_nargs (*from_p) == 3) > + assign = gimple_build_call_internal (IFN_BUILTIN_EXPECT, 3, > + CALL_EXPR_ARG (*from_p, 0), > + CALL_EXPR_ARG (*from_p, 1), > + CALL_EXPR_ARG (*from_p, 2)); > + else > + { > + assign = gimple_build_call_from_tree (*from_p); > + gimple_call_set_fntype (assign, TREE_TYPE (fnptrtype)); > + } > notice_special_calls (assign); > if (!gimple_call_noreturn_p (assign)) > gimple_call_set_lhs (assign, *to_p); > --- gcc/ipa-inline-analysis.c.jj 2014-02-06 11:50:58.000000000 +0100 > +++ gcc/ipa-inline-analysis.c 2014-03-14 13:15:54.824136137 +0100 > @@ -2306,7 +2306,10 @@ find_foldable_builtin_expect (basic_bloc > for (bsi = gsi_start_bb (bb); !gsi_end_p (bsi); gsi_next (&bsi)) > { > gimple stmt = gsi_stmt (bsi); > - if (gimple_call_builtin_p (stmt, BUILT_IN_EXPECT)) > + if (gimple_call_builtin_p (stmt, BUILT_IN_EXPECT) > + || (is_gimple_call (stmt) > + && gimple_call_internal_p (stmt) > + && gimple_call_internal_fn (stmt) == IFN_BUILTIN_EXPECT)) > { > tree var = gimple_call_lhs (stmt); > tree arg = gimple_call_arg (stmt, 0); > --- gcc/gimple-fold.c.jj 2014-02-12 17:46:47.000000000 +0100 > +++ gcc/gimple-fold.c 2014-03-14 14:02:40.322947112 +0100 > @@ -1181,6 +1181,20 @@ gimple_fold_call (gimple_stmt_iterator * > else if (gimple_call_builtin_p (stmt, BUILT_IN_MD)) > changed |= targetm.gimple_fold_builtin (gsi); > } > + else if (gimple_call_internal_p (stmt) > + && gimple_call_internal_fn (stmt) == IFN_BUILTIN_EXPECT) > + { > + tree result = fold_builtin_expect (gimple_location (stmt), > + gimple_call_arg (stmt, 0), > + gimple_call_arg (stmt, 1), > + gimple_call_arg (stmt, 2)); > + if (result) > + { > + if (!update_call_from_tree (gsi, result)) > + gimplify_and_update_call_from_tree (gsi, result); > + changed = true; > + } > + } > > return changed; > } > --- gcc/tree.h.jj 2014-02-06 11:52:50.000000000 +0100 > +++ gcc/tree.h 2014-03-14 13:54:45.941695181 +0100 > @@ -4548,6 +4548,7 @@ extern tree fold_builtin_stxcpy_chk (loc > enum built_in_function); > extern tree fold_builtin_stxncpy_chk (location_t, tree, tree, tree, tree, > tree, bool, > enum built_in_function); > +extern tree fold_builtin_expect (location_t, tree, tree, tree); > extern bool fold_builtin_next_arg (tree, bool); > extern enum built_in_function builtin_mathfn_code (const_tree); > extern tree fold_builtin_call_array (location_t, tree, tree, int, tree *); > --- gcc/builtins.c.jj 2014-03-12 10:13:41.000000000 +0100 > +++ gcc/builtins.c 2014-03-14 13:53:50.027019261 +0100 > @@ -140,7 +140,6 @@ static rtx expand_builtin_frame_address > static tree stabilize_va_list_loc (location_t, tree, int); > static rtx expand_builtin_expect (tree, rtx); > static tree fold_builtin_constant_p (tree); > -static tree fold_builtin_expect (location_t, tree, tree); > static tree fold_builtin_classify_type (tree); > static tree fold_builtin_strlen (location_t, tree, tree); > static tree fold_builtin_inf (location_t, tree, int); > @@ -6978,7 +6977,8 @@ fold_builtin_constant_p (tree arg) > return it as a truthvalue. */ > > static tree > -build_builtin_expect_predicate (location_t loc, tree pred, tree expected) > +build_builtin_expect_predicate (location_t loc, tree pred, tree expected, > + tree predictor) > { > tree fn, arg_types, pred_type, expected_type, call_expr, ret_type; > > @@ -6990,7 +6990,8 @@ build_builtin_expect_predicate (location > > pred = fold_convert_loc (loc, pred_type, pred); > expected = fold_convert_loc (loc, expected_type, expected); > - call_expr = build_call_expr_loc (loc, fn, 2, pred, expected); > + call_expr = build_call_expr_loc (loc, fn, predictor ? 3 : 2, pred, > expected, > + predictor); > > return build2 (NE_EXPR, TREE_TYPE (pred), call_expr, > build_int_cst (ret_type, 0)); > @@ -6999,8 +7000,8 @@ build_builtin_expect_predicate (location > /* Fold a call to builtin_expect with arguments ARG0 and ARG1. Return > NULL_TREE if no simplification is possible. */ > > -static tree > -fold_builtin_expect (location_t loc, tree arg0, tree arg1) > +tree > +fold_builtin_expect (location_t loc, tree arg0, tree arg1, tree arg2) > { > tree inner, fndecl, inner_arg0; > enum tree_code code; > @@ -7035,8 +7036,8 @@ fold_builtin_expect (location_t loc, tre > tree op0 = TREE_OPERAND (inner, 0); > tree op1 = TREE_OPERAND (inner, 1); > > - op0 = build_builtin_expect_predicate (loc, op0, arg1); > - op1 = build_builtin_expect_predicate (loc, op1, arg1); > + op0 = build_builtin_expect_predicate (loc, op0, arg1, arg2); > + op1 = build_builtin_expect_predicate (loc, op1, arg1, arg2); > inner = build2 (code, TREE_TYPE (inner), op0, op1); > > return fold_convert_loc (loc, TREE_TYPE (arg0), inner); > @@ -10852,7 +10853,7 @@ fold_builtin_2 (location_t loc, tree fnd > return fold_builtin_strpbrk (loc, arg0, arg1, type); > > case BUILT_IN_EXPECT: > - return fold_builtin_expect (loc, arg0, arg1); > + return fold_builtin_expect (loc, arg0, arg1, NULL_TREE); > > CASE_FLT_FN (BUILT_IN_POW): > return fold_builtin_pow (loc, fndecl, arg0, arg1, type); > @@ -11032,6 +11033,9 @@ fold_builtin_3 (location_t loc, tree fnd > return fold_builtin_fprintf (loc, fndecl, arg0, arg2, NULL_TREE, > ignore, fcode); > > + case BUILT_IN_EXPECT: > + return fold_builtin_expect (loc, arg0, arg1, arg2); > + > default: > break; > } > --- gcc/internal-fn.def.jj 2014-01-29 12:43:24.000000000 +0100 > +++ gcc/internal-fn.def 2014-03-14 13:04:07.333259773 +0100 > @@ -52,3 +52,4 @@ DEF_INTERNAL_FN (UBSAN_CHECK_ADD, ECF_CO > DEF_INTERNAL_FN (UBSAN_CHECK_SUB, ECF_CONST | ECF_LEAF | ECF_NOTHROW) > DEF_INTERNAL_FN (UBSAN_CHECK_MUL, ECF_CONST | ECF_LEAF | ECF_NOTHROW) > DEF_INTERNAL_FN (ABNORMAL_DISPATCHER, ECF_NORETURN) > +DEF_INTERNAL_FN (BUILTIN_EXPECT, ECF_CONST | ECF_LEAF | ECF_NOTHROW) > --- gcc/fortran/trans-array.c.jj 2014-03-07 13:57:22.936517443 +0100 > +++ gcc/fortran/trans-array.c 2014-03-14 10:12:26.148478839 +0100 > @@ -4993,12 +4993,14 @@ gfc_array_init_size (tree descriptor, in > TYPE_MAX_VALUE > (gfc_array_index_type)), > size); > cond = gfc_unlikely (fold_build2_loc (input_location, LT_EXPR, > - boolean_type_node, tmp, stride)); > + boolean_type_node, tmp, stride), > + PRED_FORTRAN_OVERFLOW); > tmp = fold_build3_loc (input_location, COND_EXPR, integer_type_node, > cond, > integer_one_node, integer_zero_node); > cond = gfc_unlikely (fold_build2_loc (input_location, EQ_EXPR, > boolean_type_node, size, > - gfc_index_zero_node)); > + gfc_index_zero_node), > + PRED_FORTRAN_SIZE_ZERO); > tmp = fold_build3_loc (input_location, COND_EXPR, integer_type_node, > cond, > integer_zero_node, tmp); > tmp = fold_build2_loc (input_location, PLUS_EXPR, integer_type_node, > @@ -5095,12 +5097,14 @@ gfc_array_init_size (tree descriptor, in > size_type_node, > TYPE_MAX_VALUE (size_type_node), element_size); > cond = gfc_unlikely (fold_build2_loc (input_location, LT_EXPR, > - boolean_type_node, tmp, stride)); > + boolean_type_node, tmp, stride), > + PRED_FORTRAN_OVERFLOW); > tmp = fold_build3_loc (input_location, COND_EXPR, integer_type_node, cond, > integer_one_node, integer_zero_node); > cond = gfc_unlikely (fold_build2_loc (input_location, EQ_EXPR, > boolean_type_node, element_size, > - build_int_cst (size_type_node, 0))); > + build_int_cst (size_type_node, 0)), > + PRED_FORTRAN_SIZE_ZERO); > tmp = fold_build3_loc (input_location, COND_EXPR, integer_type_node, cond, > integer_zero_node, tmp); > tmp = fold_build2_loc (input_location, PLUS_EXPR, integer_type_node, > @@ -5282,7 +5286,8 @@ gfc_array_allocate (gfc_se * se, gfc_exp > if (dimension) > { > cond = gfc_unlikely (fold_build2_loc (input_location, NE_EXPR, > - boolean_type_node, var_overflow, integer_zero_node)); > + boolean_type_node, var_overflow, integer_zero_node), > + PRED_FORTRAN_OVERFLOW); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, cond, > error, gfc_finish_block (&elseblock)); > } > @@ -5303,7 +5308,8 @@ gfc_array_allocate (gfc_se * se, gfc_exp > build_int_cst (TREE_TYPE (status), 0)); > gfc_add_expr_to_block (&se->pre, > fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_likely (cond), set_descriptor, > + gfc_likely (cond, PRED_FORTRAN_FAIL_ALLOC), > + set_descriptor, > build_empty_stmt (input_location))); > } > else > --- gcc/fortran/trans-expr.c.jj 2014-03-07 13:57:22.917517547 +0100 > +++ gcc/fortran/trans-expr.c 2014-03-14 10:12:26.173478725 +0100 > @@ -4099,7 +4099,7 @@ gfc_conv_procedure_call (gfc_se * se, gf > parmse.expr > = fold_build3_loc (input_location, COND_EXPR, > TREE_TYPE (parmse.expr), > - gfc_unlikely (tmp), > + gfc_unlikely (tmp, > PRED_FORTRAN_ABSENT_DUMMY), > fold_convert (TREE_TYPE (parmse.expr), > null_pointer_node), > parmse.expr); > --- gcc/fortran/trans.h.jj 2014-03-07 13:57:22.970517258 +0100 > +++ gcc/fortran/trans.h 2014-03-14 10:12:26.235478369 +0100 > @@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. > #ifndef GFC_TRANS_H > #define GFC_TRANS_H > > +#include "predict.h" /* For enum br_predictor and PRED_*. */ > + > /* Mangled symbols take the form __module__name. */ > #define GFC_MAX_MANGLED_SYMBOL_LEN (GFC_MAX_SYMBOL_LEN*2+4) > > @@ -580,8 +582,8 @@ void gfc_generate_constructors (void); > bool get_array_ctor_strlen (stmtblock_t *, gfc_constructor_base, tree *); > > /* Mark a condition as likely or unlikely. */ > -tree gfc_likely (tree); > -tree gfc_unlikely (tree); > +tree gfc_likely (tree, enum br_predictor); > +tree gfc_unlikely (tree, enum br_predictor); > > /* Return the string length of a deferred character length component. */ > bool gfc_deferred_strlen (gfc_component *, tree *); > @@ -630,7 +632,6 @@ tree gfc_trans_pointer_assignment (gfc_e > /* Initialize function decls for library functions. */ > void gfc_build_intrinsic_lib_fndecls (void); > /* Create function decls for IO library functions. */ > -void gfc_trans_io_runtime_check (tree, tree, int, const char *, stmtblock_t > *); > void gfc_build_io_library_fndecls (void); > /* Build a function decl for a library function. */ > tree gfc_build_library_function_decl (tree, tree, int, ...); > --- gcc/fortran/trans.c.jj 2014-03-07 13:57:22.968517269 +0100 > +++ gcc/fortran/trans.c 2014-03-14 14:12:04.909731639 +0100 > @@ -501,6 +501,11 @@ gfc_trans_runtime_check (bool error, boo > > gfc_start_block (&block); > > + /* For error, runtime_error_at already implies PRED_NORETURN. */ > + if (!error && once) > + gfc_add_expr_to_block (&block, build_predict_expr > (PRED_FORTRAN_WARN_ONCE, > + NOT_TAKEN)); > + > /* The code to generate the error. */ > va_start (ap, msgid); > gfc_add_expr_to_block (&block, > @@ -519,14 +524,12 @@ gfc_trans_runtime_check (bool error, boo > } > else > { > - /* Tell the compiler that this isn't likely. */ > if (once) > cond = fold_build2_loc (where->lb->location, TRUTH_AND_EXPR, > long_integer_type_node, tmpvar, cond); > else > cond = fold_convert (long_integer_type_node, cond); > > - cond = gfc_unlikely (cond); > tmp = fold_build3_loc (where->lb->location, COND_EXPR, void_type_node, > cond, body, > build_empty_stmt (where->lb->location)); > @@ -616,7 +619,8 @@ void > gfc_allocate_using_malloc (stmtblock_t * block, tree pointer, > tree size, tree status) > { > - tree tmp, on_error, error_cond; > + tree tmp, error_cond; > + stmtblock_t on_error; > tree status_type = status ? TREE_TYPE (status) : NULL_TREE; > > /* Evaluate size only once, and make sure it has the right type. */ > @@ -640,20 +644,31 @@ gfc_allocate_using_malloc (stmtblock_t * > build_int_cst (size_type_node, 1))))); > > /* What to do in case of error. */ > + gfc_start_block (&on_error); > if (status != NULL_TREE) > - on_error = fold_build2_loc (input_location, MODIFY_EXPR, status_type, > - status, build_int_cst (status_type, > LIBERROR_ALLOCATION)); > + { > + gfc_add_expr_to_block (&on_error, > + build_predict_expr (PRED_FORTRAN_FAIL_ALLOC, > + NOT_TAKEN)); > + tmp = fold_build2_loc (input_location, MODIFY_EXPR, status_type, > status, > + build_int_cst (status_type, LIBERROR_ALLOCATION)); > + gfc_add_expr_to_block (&on_error, tmp); > + } > else > - on_error = build_call_expr_loc (input_location, gfor_fndecl_os_error, 1, > + { > + /* Here, os_error already implies PRED_NORETURN. */ > + tmp = build_call_expr_loc (input_location, gfor_fndecl_os_error, 1, > gfc_build_addr_expr (pchar_type_node, > gfc_build_localized_cstring_const > - ("Allocation would exceed memory limit"))); > + ("Allocation would exceed memory limit"))); > + gfc_add_expr_to_block (&on_error, tmp); > + } > > error_cond = fold_build2_loc (input_location, EQ_EXPR, > boolean_type_node, pointer, > build_int_cst (prvoid_type_node, 0)); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (error_cond), on_error, > + error_cond, gfc_finish_block (&on_error), > build_empty_stmt (input_location)); > > gfc_add_expr_to_block (block, tmp); > @@ -750,7 +765,8 @@ gfc_allocate_allocatable (stmtblock_t * > > null_mem = gfc_unlikely (fold_build2_loc (input_location, NE_EXPR, > boolean_type_node, mem, > - build_int_cst (type, 0))); > + build_int_cst (type, 0)), > + PRED_FORTRAN_FAIL_ALLOC); > > /* If mem is NULL, we call gfc_allocate_using_malloc or > gfc_allocate_using_lib. */ > @@ -770,8 +786,8 @@ gfc_allocate_allocatable (stmtblock_t * > cond = fold_build2_loc (input_location, NE_EXPR, boolean_type_node, > status, build_zero_cst (TREE_TYPE (status))); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (cond), tmp, > - build_empty_stmt (input_location)); > + gfc_unlikely (cond, PRED_FORTRAN_FAIL_ALLOC), > + tmp, build_empty_stmt (input_location)); > gfc_add_expr_to_block (&alloc_block, tmp); > } > } > @@ -1268,8 +1284,8 @@ gfc_deallocate_with_status (tree pointer > status_type, status), > build_int_cst (status_type, 0)); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (cond2), tmp, > - build_empty_stmt (input_location)); > + gfc_unlikely (cond2, PRED_FORTRAN_FAIL_ALLOC), > + tmp, build_empty_stmt (input_location)); > gfc_add_expr_to_block (&non_null, tmp); > } > } > @@ -1327,8 +1343,8 @@ gfc_deallocate_with_status (tree pointer > cond2 = fold_build2_loc (input_location, NE_EXPR, boolean_type_node, > stat, build_zero_cst (TREE_TYPE (stat))); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (cond2), tmp, > - build_empty_stmt (input_location)); > + gfc_unlikely (cond2, PRED_FORTRAN_FAIL_ALLOC), > + tmp, build_empty_stmt (input_location)); > gfc_add_expr_to_block (&non_null, tmp); > } > } > @@ -2015,15 +2031,20 @@ gfc_finish_wrapped_block (gfc_wrapped_bl > /* Helper function for marking a boolean expression tree as unlikely. */ > > tree > -gfc_unlikely (tree cond) > +gfc_unlikely (tree cond, enum br_predictor predictor) > { > tree tmp; > > - cond = fold_convert (long_integer_type_node, cond); > - tmp = build_zero_cst (long_integer_type_node); > - cond = build_call_expr_loc (input_location, > - builtin_decl_explicit (BUILT_IN_EXPECT), > - 2, cond, tmp); > + if (optimize) > + { > + cond = fold_convert (long_integer_type_node, cond); > + tmp = build_zero_cst (long_integer_type_node); > + cond = build_call_expr_loc (input_location, > + builtin_decl_explicit (BUILT_IN_EXPECT), > + 3, cond, tmp, > + build_int_cst (integer_type_node, > + predictor)); > + } > cond = fold_convert (boolean_type_node, cond); > return cond; > } > @@ -2032,15 +2053,20 @@ gfc_unlikely (tree cond) > /* Helper function for marking a boolean expression tree as likely. */ > > tree > -gfc_likely (tree cond) > +gfc_likely (tree cond, enum br_predictor predictor) > { > tree tmp; > > - cond = fold_convert (long_integer_type_node, cond); > - tmp = build_one_cst (long_integer_type_node); > - cond = build_call_expr_loc (input_location, > - builtin_decl_explicit (BUILT_IN_EXPECT), > - 2, cond, tmp); > + if (optimize) > + { > + cond = fold_convert (long_integer_type_node, cond); > + tmp = build_one_cst (long_integer_type_node); > + cond = build_call_expr_loc (input_location, > + builtin_decl_explicit (BUILT_IN_EXPECT), > + 3, cond, tmp, > + build_int_cst (integer_type_node, > + predictor)); > + } > cond = fold_convert (boolean_type_node, cond); > return cond; > } > --- gcc/fortran/trans-io.c.jj 2014-01-09 21:07:24.238165839 +0100 > +++ gcc/fortran/trans-io.c 2014-03-14 10:12:26.212478499 +0100 > @@ -230,9 +230,10 @@ gfc_build_st_parameter (enum ioparam_typ > Therefore, the code to set these flags must be generated before > this function is used. */ > > -void > -gfc_trans_io_runtime_check (tree cond, tree var, int error_code, > - const char * msgid, stmtblock_t * pblock) > +static void > +gfc_trans_io_runtime_check (bool has_iostat, tree cond, tree var, > + int error_code, const char * msgid, > + stmtblock_t * pblock) > { > stmtblock_t block; > tree body; > @@ -246,6 +247,13 @@ gfc_trans_io_runtime_check (tree cond, t > /* The code to generate the error. */ > gfc_start_block (&block); > > + if (has_iostat) > + gfc_add_expr_to_block (&block, build_predict_expr (PRED_FORTRAN_FAIL_IO, > + NOT_TAKEN)); > + else > + gfc_add_expr_to_block (&block, build_predict_expr (PRED_NORETURN, > + NOT_TAKEN)); > + > arg1 = gfc_build_addr_expr (NULL_TREE, var); > > arg2 = build_int_cst (integer_type_node, error_code), > @@ -268,7 +276,6 @@ gfc_trans_io_runtime_check (tree cond, t > } > else > { > - cond = gfc_unlikely (cond); > tmp = build3_v (COND_EXPR, cond, body, build_empty_stmt > (input_location)); > gfc_add_expr_to_block (pblock, tmp); > } > @@ -494,8 +501,8 @@ set_parameter_const (stmtblock_t *block, > st_parameter_XXX structure. This is a pass by value. */ > > static unsigned int > -set_parameter_value (stmtblock_t *block, tree var, enum iofield type, > - gfc_expr *e) > +set_parameter_value (stmtblock_t *block, bool has_iostat, tree var, > + enum iofield type, gfc_expr *e) > { > gfc_se se; > tree tmp; > @@ -520,18 +527,18 @@ set_parameter_value (stmtblock_t *block, > cond = fold_build2_loc (input_location, LT_EXPR, boolean_type_node, > se.expr, > fold_convert (TREE_TYPE (se.expr), val)); > - gfc_trans_io_runtime_check (cond, var, LIBERROR_BAD_UNIT, > - "Unit number in I/O statement too small", > - &se.pre); > + gfc_trans_io_runtime_check (has_iostat, cond, var, LIBERROR_BAD_UNIT, > + "Unit number in I/O statement too small", > + &se.pre); > > /* UNIT numbers should be less than the max. */ > val = gfc_conv_mpz_to_tree (gfc_integer_kinds[i].huge, 4); > cond = fold_build2_loc (input_location, GT_EXPR, boolean_type_node, > se.expr, > fold_convert (TREE_TYPE (se.expr), val)); > - gfc_trans_io_runtime_check (cond, var, LIBERROR_BAD_UNIT, > - "Unit number in I/O statement too large", > - &se.pre); > + gfc_trans_io_runtime_check (has_iostat, cond, var, LIBERROR_BAD_UNIT, > + "Unit number in I/O statement too large", > + &se.pre); > > } > > @@ -960,7 +967,8 @@ gfc_trans_open (gfc_code * code) > mask |= set_string (&block, &post_block, var, IOPARM_open_form, p->form); > > if (p->recl) > - mask |= set_parameter_value (&block, var, IOPARM_open_recl_in, p->recl); > + mask |= set_parameter_value (&block, p->iostat, var, IOPARM_open_recl_in, > + p->recl); > > if (p->blank) > mask |= set_string (&block, &post_block, var, IOPARM_open_blank, > @@ -1010,7 +1018,7 @@ gfc_trans_open (gfc_code * code) > set_parameter_const (&block, var, IOPARM_common_flags, mask); > > if (p->unit) > - set_parameter_value (&block, var, IOPARM_common_unit, p->unit); > + set_parameter_value (&block, p->iostat, var, IOPARM_common_unit, > p->unit); > else > set_parameter_const (&block, var, IOPARM_common_unit, 0); > > @@ -1063,7 +1071,7 @@ gfc_trans_close (gfc_code * code) > set_parameter_const (&block, var, IOPARM_common_flags, mask); > > if (p->unit) > - set_parameter_value (&block, var, IOPARM_common_unit, p->unit); > + set_parameter_value (&block, p->iostat, var, IOPARM_common_unit, > p->unit); > else > set_parameter_const (&block, var, IOPARM_common_unit, 0); > > @@ -1114,7 +1122,7 @@ build_filepos (tree function, gfc_code * > set_parameter_const (&block, var, IOPARM_common_flags, mask); > > if (p->unit) > - set_parameter_value (&block, var, IOPARM_common_unit, p->unit); > + set_parameter_value (&block, p->iostat, var, IOPARM_common_unit, > p->unit); > else > set_parameter_const (&block, var, IOPARM_common_unit, 0); > > @@ -1375,7 +1383,7 @@ gfc_trans_inquire (gfc_code * code) > set_parameter_const (&block, var, IOPARM_common_flags, mask); > > if (p->unit) > - set_parameter_value (&block, var, IOPARM_common_unit, p->unit); > + set_parameter_value (&block, p->iostat, var, IOPARM_common_unit, > p->unit); > else > set_parameter_const (&block, var, IOPARM_common_unit, 0); > > @@ -1422,12 +1430,12 @@ gfc_trans_wait (gfc_code * code) > mask |= IOPARM_common_err; > > if (p->id) > - mask |= set_parameter_value (&block, var, IOPARM_wait_id, p->id); > + mask |= set_parameter_value (&block, p->iostat, var, IOPARM_wait_id, > p->id); > > set_parameter_const (&block, var, IOPARM_common_flags, mask); > > if (p->unit) > - set_parameter_value (&block, var, IOPARM_common_unit, p->unit); > + set_parameter_value (&block, p->iostat, var, IOPARM_common_unit, > p->unit); > > tmp = gfc_build_addr_expr (NULL_TREE, var); > tmp = build_call_expr_loc (input_location, > @@ -1718,7 +1726,8 @@ build_dt (tree function, gfc_code * code > IOPARM_dt_id, dt->id); > > if (dt->pos) > - mask |= set_parameter_value (&block, var, IOPARM_dt_pos, dt->pos); > + mask |= set_parameter_value (&block, dt->iostat, var, IOPARM_dt_pos, > + dt->pos); > > if (dt->asynchronous) > mask |= set_string (&block, &post_block, var, IOPARM_dt_asynchronous, > @@ -1749,7 +1758,8 @@ build_dt (tree function, gfc_code * code > dt->sign); > > if (dt->rec) > - mask |= set_parameter_value (&block, var, IOPARM_dt_rec, dt->rec); > + mask |= set_parameter_value (&block, dt->iostat, var, IOPARM_dt_rec, > + dt->rec); > > if (dt->advance) > mask |= set_string (&block, &post_block, var, IOPARM_dt_advance, > @@ -1801,7 +1811,8 @@ build_dt (tree function, gfc_code * code > set_parameter_const (&block, var, IOPARM_common_flags, mask); > > if (dt->io_unit && dt->io_unit->ts.type == BT_INTEGER) > - set_parameter_value (&block, var, IOPARM_common_unit, dt->io_unit); > + set_parameter_value (&block, dt->iostat, var, IOPARM_common_unit, > + dt->io_unit); > } > else > set_parameter_const (&block, var, IOPARM_common_flags, mask); > --- gcc/fortran/trans-stmt.c.jj 2014-03-07 13:57:22.968517269 +0100 > +++ gcc/fortran/trans-stmt.c 2014-03-14 10:12:26.221478449 +0100 > @@ -5107,8 +5107,8 @@ gfc_trans_allocate (gfc_code * code) > boolean_type_node, stat, > build_int_cst (TREE_TYPE (stat), 0)); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (parm), tmp, > - build_empty_stmt (input_location)); > + gfc_unlikely (parm, PRED_FORTRAN_FAIL_ALLOC), > + tmp, build_empty_stmt (input_location)); > gfc_add_expr_to_block (&block, tmp); > } > > @@ -5501,7 +5501,7 @@ gfc_trans_deallocate (gfc_code *code) > cond = fold_build2_loc (input_location, NE_EXPR, boolean_type_node, > stat, > build_int_cst (TREE_TYPE (stat), 0)); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (cond), > + gfc_unlikely (cond, PRED_FORTRAN_FAIL_ALLOC), > build1_v (GOTO_EXPR, label_errmsg), > build_empty_stmt (input_location)); > gfc_add_expr_to_block (&se.pre, tmp); > @@ -5541,7 +5541,7 @@ gfc_trans_deallocate (gfc_code *code) > cond = fold_build2_loc (input_location, NE_EXPR, boolean_type_node, > stat, > build_int_cst (TREE_TYPE (stat), 0)); > tmp = fold_build3_loc (input_location, COND_EXPR, void_type_node, > - gfc_unlikely (cond), tmp, > + gfc_unlikely (cond, PRED_FORTRAN_FAIL_ALLOC), tmp, > build_empty_stmt (input_location)); > > gfc_add_expr_to_block (&block, tmp); > --- gcc/fortran/trans-intrinsic.c.jj 2014-03-07 13:57:22.983517187 +0100 > +++ gcc/fortran/trans-intrinsic.c 2014-03-14 10:12:26.183478666 +0100 > @@ -1196,8 +1196,7 @@ trans_image_index (gfc_se * se, gfc_expr > boolean_type_node, invalid_bound, cond); > } > > - invalid_bound = gfc_unlikely (invalid_bound); > - > + invalid_bound = gfc_unlikely (invalid_bound, PRED_FORTRAN_INVALID_BOUND); > > /* See Fortran 2008, C.10 for the following algorithm. */ > > > Jakub