For uncounted loops, given how no information can be derived from the
max number of iterations, we wish to make no distinction between the
"main" exit and any additional exits.
That is, an epilogue is required across all exits to establish when
the exit condition was met within the final vectorized loop iteration.
This can be accomplished via a two-fold approach:
1. The need for all exits to go to the epilogue is shared with
counted loops with early-break exits after the IV-counting exit.
Such counted loops have the `LOOP_VINFO_EARLY_BREAKS_VECT_PEELED'
flag set. By modifying the flag's definition to encompass
uncounted loops, we can make considerable use of the code written
for this category of counted loops.
2. Currently, when populating the `loop_vinfo' struct for a given
loop, there is an assumption that the first exit condition in the
`vect_loop_form_info' struct is the IV condition and any
subsequent conditions are early break conditions. This
assumption breaks down for uncounted loops where _all_ exits
should be treated as being "early break" exits.
This is fixed by populating `LOOP_VINFO_LOOP_IV_COND (loop_vinfo)'
conditionally on the false evaluation of the
`LOOP_VINFO_NITERS_UNCOUNTED_P (loop_vinfo)' predicate, such that
if we do have an uncounted loop we leave this field unpopulated,
storing all conditions in `LOOP_VINFO_LOOP_CONDS (loop_vinfo)'.
This approach has a further benefit in that, give how
`LOOP_VINFO_EARLY_BREAKS' is defined by a non-empty list of early
break exit conditions (`LOOP_VINFO_LOOP_CONDS (loop_vinfo)'),
having LOOP_VINFO_LOOP_CONDS populated even for single-exit
uncounted loops means that `LOOP_VINFO_EARLY_BREAKS' evaluates to
true for all uncounted loops, irrespective of the number of exits
it has.
gcc/ChangeLog:
* tree-vectorizer.h (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED): OR
its current definition with `LOOP_VINFO_NITERS_UNCOUNTED_P(L)'
* tree-vect-loop.cc (vect_create_loop_vinfo): Don't populate
`LOOP_VINFO_LOOP_IV_COND' for uncounted loops.
---
gcc/tree-vect-loop.cc | 9 ++++++---
gcc/tree-vectorizer.h | 3 ++-
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
index b317386dc18..8d5fb64ca9c 100644
--- a/gcc/tree-vect-loop.cc
+++ b/gcc/tree-vect-loop.cc
@@ -1668,9 +1668,12 @@ vect_create_loop_vinfo (class loop *loop,
vec_info_shared *shared,
STMT_VINFO_DEF_TYPE (loop_cond_info) = vect_condition_def;
}
- for (unsigned i = 1; i < info->conds.length (); i ++)
- LOOP_VINFO_LOOP_CONDS (loop_vinfo).safe_push (info->conds[i]);
- LOOP_VINFO_LOOP_IV_COND (loop_vinfo) = info->conds[0];
+ unsigned cond_id = 0;
+ if (!LOOP_VINFO_NITERS_UNCOUNTED_P (loop_vinfo))
+ LOOP_VINFO_LOOP_IV_COND (loop_vinfo) = info->conds[cond_id++];
+
+ for (; cond_id < info->conds.length (); cond_id ++)
+ LOOP_VINFO_LOOP_CONDS (loop_vinfo).safe_push (info->conds[cond_id]);
LOOP_VINFO_IV_EXIT (loop_vinfo) = info->loop_exit;
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index bbc2e0df01d..0ae880b15bb 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -1308,7 +1308,8 @@ public:
#define LOOP_VINFO_EARLY_BREAKS(L) (L)->early_breaks
#define LOOP_VINFO_EARLY_BRK_STORES(L) (L)->early_break_stores
#define LOOP_VINFO_EARLY_BREAKS_VECT_PEELED(L) \
- (single_pred ((L)->loop->latch) != (L)->vec_loop_iv_exit->src)
+ ((single_pred ((L)->loop->latch) != (L)->vec_loop_iv_exit->src) \
+ || LOOP_VINFO_NITERS_UNCOUNTED_P (L))
#define LOOP_VINFO_EARLY_BREAKS_LIVE_IVS(L) \
(L)->early_break_live_ivs
#define LOOP_VINFO_EARLY_BRK_DEST_BB(L) (L)->early_break_dest_bb
--
2.43.0
