On Fri, Aug 1, 2025 at 8:30 AM Saurabh Jha <saurabh....@arm.com> wrote:
>
>
>
> On 7/29/2025 4:13 PM, Jennifer Schmitz wrote:
> > This patch adds dispatch scheduling for AArch64 by implementing the two
> > target
> > hooks TARGET_SCHED_DISPATCH and TARGET_SCHED_DISPATCH_DO.
> >
> > The motivation for this is that cores with out-of-order processing do
> > most of the reordering to avoid pipeline hazards on the hardware side
> > using large reorder buffers. For such cores, rather than scheduling
> > around instruction latencies and throughputs, the compiler should aim to
> > maximize the utilized dispatch bandwidth by inserting a certain
> > instruction mix into the frontend dispatch window.
> >
> > In the following, we will describe the overall implementation:
> > Recall that the Haifa scheduler makes the following 6 types of queries to
> > a dispatch scheduling model:
> > 1) targetm.sched.dispatch (NULL, IS_DISPATCH_ON)
> > 2) targetm.sched.dispatch_do (NULL, DISPATCH_INIT)
> > 3) targetm.sched.dispatch (insn, FITS_DISPATCH_WINDOW)
> > 4) targetm.sched.dispatch_do (insn, ADD_TO_DISPATCH_WINDOW)
> > 5) targetm.sched.dispatch (NULL, DISPATCH_VIOLATION)
> > 6) targetm.sched.dispatch (insn, IS_CMP)
> >
> > For 1), we created the new tune flag AARCH64_EXTRA_TUNE_DISPATCH_SCHED.
> >
> > For 2-5), we modeled dispatch scheduling using two classes:
> > dispatch_constraint
> > and dispatch_window.
> > A dispatch_constraint object represents one rule constraining the dispatch
> > process, consisting of a function describing the rule and (constant) number
> > of maximum slots. It also has a non-constant field to keep track of the
> > number
> > of free slots.
> > A dispatch_window object models the window of operations that is dispatched
> > per cycle. It contains an array of dispatch constraints and keeps track of
> > the available slots. It also exposes functions to ask whether a given
> > instruction would fit into the dispatch_window or to add an instruction to
> > the window.
> > The model operates using only one dispatch_window object that is constructed
> > when 2) is called. It copies the dispatch information provided in the tuning
> > model (more details on the changes to tune_params below). During scheduling,
> > instructions are added according to the dispatch constraints and when the
> > dispatch_window is full, the free slots in all dispatch_constraint objects
> > are
> > reset to the maximum.
> > A dispatch violation occurs when the number of free slots becomes negative
> > for any dispatch_constraint.
> >
> > For 6), there is a separate function using the "type"-attribute to check if
> > a given instruction is a compare instruction.
> >
> > Dispatch information for a core can be added in its tuning model. We added
> > two new fields: a *dispatch_constraint (reference to an array of dispatch
> > constraints) and an int (number of constraints).
> > All current tuning models were initialized with nullptr and 0.
> > (In the next patch, dispatch information will be added for Neoverse V2.)
> >
> > The patch was bootstrapped and tested on aarch64-linux-gnu, no regression.
> >
> > Signed-off-by: Jennifer Schmitz <jschm...@nvidia.com>
> >
> > gcc/ChangeLog:
> >
> > * config.gcc: Add aarch64-sched-dispatch.o to extra_objs.
> > * config/aarch64/aarch64-protos.h (struct tune_params): New
> > fields for dispatch scheduling.
> > * config/aarch64/aarch64-tuning-flags.def
> > (AARCH64_EXTRA_TUNING_OPTION): New flag to enable dispatch scheduling.
> > * config/aarch64/aarch64.cc (TARGET_SCHED_DISPATCH): Implement
> > target hook.
> > (TARGET_SCHED_DISPATCH_DO): Likewise.
> > (aarch64_override_options_internal): Add check for definition of
> > dispatch constraints if dispatch-scheduling tune flag is set.
> > * config/aarch64/t-aarch64: Add aarch64-sched-dispatch.o.
> > * config/aarch64/tuning_models/a64fx.h: Initialize fields for
> > dispatch scheduling in tune_params.
> > * config/aarch64/tuning_models/ampere1.h: Likewise.
> > * config/aarch64/tuning_models/ampere1a.h: Likewise.
> > * config/aarch64/tuning_models/ampere1b.h: Likewise.
> > * config/aarch64/tuning_models/cortexa35.h: Likewise.
> > * config/aarch64/tuning_models/cortexa53.h: Likewise.
> > * config/aarch64/tuning_models/cortexa57.h: Likewise.
> > * config/aarch64/tuning_models/cortexa72.h: Likewise.
> > * config/aarch64/tuning_models/cortexa73.h: Likewise.
> > * config/aarch64/tuning_models/cortexx925.h: Likewise.
> > * config/aarch64/tuning_models/emag.h: Likewise.
> > * config/aarch64/tuning_models/exynosm1.h: Likewise.
> > * config/aarch64/tuning_models/fujitsu_monaka.h: Likewise.
> > * config/aarch64/tuning_models/generic.h: Likewise.
> > * config/aarch64/tuning_models/generic_armv8_a.h: Likewise.
> > * config/aarch64/tuning_models/generic_armv9_a.h: Likewise.
> > * config/aarch64/tuning_models/neoverse512tvb.h: Likewise.
> > * config/aarch64/tuning_models/neoversen1.h: Likewise.
> > * config/aarch64/tuning_models/neoversen2.h: Likewise.
> > * config/aarch64/tuning_models/neoversen3.h: Likewise.
> > * config/aarch64/tuning_models/neoversev1.h: Likewise.
> > * config/aarch64/tuning_models/neoversev2.h: Likewise.
> > * config/aarch64/tuning_models/neoversev3.h: Likewise.
> > * config/aarch64/tuning_models/neoversev3ae.h: Likewise.
> > * config/aarch64/tuning_models/olympus.h: Likewise.
> > * config/aarch64/tuning_models/qdf24xx.h: Likewise.
> > * config/aarch64/tuning_models/saphira.h: Likewise.
> > * config/aarch64/tuning_models/thunderx.h: Likewise.
> > * config/aarch64/tuning_models/thunderx2t99.h: Likewise.
> > * config/aarch64/tuning_models/thunderx3t110.h: Likewise.
> > * config/aarch64/tuning_models/thunderxt88.h: Likewise.
> > * config/aarch64/tuning_models/tsv110.h: Likewise.
> > * config/aarch64/tuning_models/xgene1.h: Likewise.
> > * config/aarch64/aarch64-sched-dispatch.cc: New file for
> > dispatch scheduling for aarch64.
> > * config/aarch64/aarch64-sched-dispatch.h: New header file.
> > ---
> > gcc/config.gcc | 2 +-
> > gcc/config/aarch64/aarch64-protos.h | 5 +
> > gcc/config/aarch64/aarch64-sched-dispatch.cc | 272 ++++++++++++++++++
> > gcc/config/aarch64/aarch64-sched-dispatch.h | 66 +++++
> > gcc/config/aarch64/aarch64-tuning-flags.def | 3 +
> > gcc/config/aarch64/aarch64.cc | 14 +
> > gcc/config/aarch64/t-aarch64 | 8 +
> > gcc/config/aarch64/tuning_models/a64fx.h | 4 +-
> > gcc/config/aarch64/tuning_models/ampere1.h | 4 +-
> > gcc/config/aarch64/tuning_models/ampere1a.h | 4 +-
> > gcc/config/aarch64/tuning_models/ampere1b.h | 4 +-
> > gcc/config/aarch64/tuning_models/cortexa35.h | 4 +-
> > gcc/config/aarch64/tuning_models/cortexa53.h | 4 +-
> > gcc/config/aarch64/tuning_models/cortexa57.h | 4 +-
> > gcc/config/aarch64/tuning_models/cortexa72.h | 4 +-
> > gcc/config/aarch64/tuning_models/cortexa73.h | 4 +-
> > gcc/config/aarch64/tuning_models/cortexx925.h | 4 +-
> > gcc/config/aarch64/tuning_models/emag.h | 4 +-
> > gcc/config/aarch64/tuning_models/exynosm1.h | 4 +-
> > .../aarch64/tuning_models/fujitsu_monaka.h | 4 +-
> > gcc/config/aarch64/tuning_models/generic.h | 4 +-
> > .../aarch64/tuning_models/generic_armv8_a.h | 4 +-
> > .../aarch64/tuning_models/generic_armv9_a.h | 4 +-
> > .../aarch64/tuning_models/neoverse512tvb.h | 4 +-
> > gcc/config/aarch64/tuning_models/neoversen1.h | 4 +-
> > gcc/config/aarch64/tuning_models/neoversen2.h | 4 +-
> > gcc/config/aarch64/tuning_models/neoversen3.h | 4 +-
> > gcc/config/aarch64/tuning_models/neoversev1.h | 4 +-
> > gcc/config/aarch64/tuning_models/neoversev2.h | 4 +-
> > gcc/config/aarch64/tuning_models/neoversev3.h | 4 +-
> > .../aarch64/tuning_models/neoversev3ae.h | 4 +-
> > gcc/config/aarch64/tuning_models/olympus.h | 4 +-
> > gcc/config/aarch64/tuning_models/qdf24xx.h | 4 +-
> > gcc/config/aarch64/tuning_models/saphira.h | 4 +-
> > gcc/config/aarch64/tuning_models/thunderx.h | 4 +-
> > .../aarch64/tuning_models/thunderx2t99.h | 4 +-
> > .../aarch64/tuning_models/thunderx3t110.h | 4 +-
> > .../aarch64/tuning_models/thunderxt88.h | 4 +-
> > gcc/config/aarch64/tuning_models/tsv110.h | 4 +-
> > gcc/config/aarch64/tuning_models/xgene1.h | 4 +-
> > 40 files changed, 468 insertions(+), 34 deletions(-)
> > create mode 100644 gcc/config/aarch64/aarch64-sched-dispatch.cc
> > create mode 100644 gcc/config/aarch64/aarch64-sched-dispatch.h
> >
> > diff --git a/gcc/config.gcc b/gcc/config.gcc
> > index 0d8dbc4fb19..6ee11313740 100644
> > --- a/gcc/config.gcc
> > +++ b/gcc/config.gcc
> > @@ -351,7 +351,7 @@ aarch64*-*-*)
> > c_target_objs="aarch64-c.o"
> > cxx_target_objs="aarch64-c.o"
> > d_target_objs="aarch64-d.o"
> > - extra_objs="aarch64-builtins.o aarch-common.o aarch64-elf-metadata.o
> > aarch64-sve-builtins.o aarch64-sve-builtins-shapes.o
> > aarch64-sve-builtins-base.o aarch64-sve-builtins-sve2.o
> > aarch64-sve-builtins-sme.o cortex-a57-fma-steering.o aarch64-speculation.o
> > aarch-bti-insert.o aarch64-cc-fusion.o aarch64-early-ra.o
> > aarch64-ldp-fusion.o"
> > + extra_objs="aarch64-builtins.o aarch-common.o aarch64-elf-metadata.o
> > aarch64-sve-builtins.o aarch64-sve-builtins-shapes.o
> > aarch64-sve-builtins-base.o aarch64-sve-builtins-sve2.o
> > aarch64-sve-builtins-sme.o cortex-a57-fma-steering.o aarch64-speculation.o
> > aarch-bti-insert.o aarch64-cc-fusion.o aarch64-early-ra.o
> > aarch64-ldp-fusion.o aarch64-sched-dispatch.o"
> > target_gtfiles="\$(srcdir)/config/aarch64/aarch64-protos.h
> > \$(srcdir)/config/aarch64/aarch64-builtins.h
> > \$(srcdir)/config/aarch64/aarch64-builtins.cc
> > \$(srcdir)/config/aarch64/aarch64-sve-builtins.h
> > \$(srcdir)/config/aarch64/aarch64-sve-builtins.cc"
> > target_has_targetm_common=yes
> > ;;
> > diff --git a/gcc/config/aarch64/aarch64-protos.h
> > b/gcc/config/aarch64/aarch64-protos.h
> > index e946e8da11d..af14d2f6dda 100644
> > --- a/gcc/config/aarch64/aarch64-protos.h
> > +++ b/gcc/config/aarch64/aarch64-protos.h
> > @@ -24,6 +24,7 @@
> >
> > #include "input.h"
> > #include "config/arm/aarch-common.h"
> > +#include "aarch64-sched-dispatch.h"
> >
> > /* SYMBOL_SMALL_ABSOLUTE: Generate symbol accesses through
> > high and lo relocs that calculate the base address using a PC
> > @@ -581,6 +582,10 @@ struct tune_params
> >
> > /* Define models for the aarch64_ldp_stp_policy. */
> > enum aarch64_ldp_stp_policy ldp_policy_model, stp_policy_model;
> > +
> > + /* Dispatch constraints for instruction scheduling. */
> > + const dispatch_constraint *dispatch_constraints;
> > + int num_dispatch_constraints;
> > };
> >
> > /* Classifies an address.
> > diff --git a/gcc/config/aarch64/aarch64-sched-dispatch.cc
> > b/gcc/config/aarch64/aarch64-sched-dispatch.cc
> > new file mode 100644
> > index 00000000000..5fde528377c
> > --- /dev/null
> > +++ b/gcc/config/aarch64/aarch64-sched-dispatch.cc
> > @@ -0,0 +1,272 @@
> > +/* Dispatch scheduling hooks for AArch64.
> > + Copyright The GNU Toolchain Authors.
> > +
> > + This file is part of GCC.
> > +
> > + GCC is free software; you can redistribute it and/or modify it
> > + under the terms of the GNU General Public License as published by
> > + the Free Software Foundation; either version 3, or (at your option)
> > + any later version.
> > +
> > + GCC is distributed in the hope that it will be useful, but
> > + WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + General Public License for more details.
> > +
> > + You should have received a copy of the GNU General Public License
> > + along with GCC; see the file COPYING3. If not see
> > + <http://www.gnu.org/licenses/>. */
> > +
> > +#include "config.h"
> > +#include "system.h"
> > +#include "coretypes.h"
> > +#include "backend.h"
> > +#include "rtl.h"
> > +#include "insn-attr.h"
> > +#include "insn-attr-common.h"
> > +#include "aarch64-protos.h"
> > +#include "aarch64-sched-dispatch.h"
> > +#include "regset.h"
> > +#include "sched-int.h"
> > +#include "dumpfile.h"
> > +#include "print-rtl.h"
> > +
> > +/* This file implements the target hooks for dispatch scheduling for
> > AArch64.
> > + Instructions are scheduled into the current dispatch window according to
> > + dispatch constraints provided by the tuning model.
> > +
> > + To enable dispatch scheduling for a core (see Neoverse V2 for an
> > example):
> > + - in the tuning model, add the AARCH64_EXTRA_TUNE_DISPATCH_SCHED tune
> > flag
> > + - in the tuning model, add an array of dispatch constraints (more
> > details
> > + below) and add its reference and length to the tune_params struct
> > + - optionally, create a new instruction attribute to classify
> > instructions
> > + into dispatch groups (e.g. neoversev2_dispatch)
> > +
> > + Dispatch constraints are defined as an array of dispatch_constraint
> > objects.
> > + Each dispatch_constraint object is constructed with a name, a number of
> > + slots (max number of instructions of that type that can be dispatched
> > per
> > + cycle), and a callback function.
> > + The callback function takes an instruction as argument, determines
> > whether
> > + the instruction matches the constraint criteria, and returns the number
> > of
> > + slots occupied by the instruction.
> > + */
> > +
> > +static dispatch_window *current_dispatch_window;
> > +
> > +/* Constructor for class dispatch_constraint. */
> > +dispatch_constraint::dispatch_constraint (const char *name, const int
> > slots,
> > + int (*f) (rtx_insn *))
> > + : m_name (name), m_max_slots (slots), m_free_slots (slots)
> Elementary question: why couldn't you do `m_callback (f)` above instead
> of initialising it in the body? Is it because it is a function pointer?
More over why is this not a constexpr constructor in the header file?
I see you use it via initializing the global array
neoversev2_dispatch_constraints.
So without the constexpr part, the neoversev2_dispatch_constraints
array will be initialized at runtime which is definitely not something
we want.
I am not exactly a fan of using lambdas in the initialization of
neoversev2_dispatch_constraints either.
Thanks,
Andrew Pinski
> > +{
> > + this->m_callback = f;
> > +}
> > +
> > +/* Return TRUE if INSN matches the constraint criteria, and there are
> > + enough free slots. */
> > +bool
> > +dispatch_constraint::can_fit_insn (rtx_insn *insn) const
> > +{
> > + int slots = num_occupied_slots (insn);
> > + if (dump_file && (dump_flags & TDF_DETAILS) && slots > 0)
> > + {
> > + fprintf (dump_file, "Insn matches constraint \'%s\'. ", m_name);
> > + fprintf (dump_file, "Slots required: %d; slots free: %d.\n",
> > + slots, m_free_slots);
> > + }
> > + return slots <= m_free_slots;
> > +}
> > +
> > +/* Decrement the free slots by the number of slots occupied by INSN.
> > + Return whether the number of free slots is non-negative (no violation).
> > */
> > +bool
> > +dispatch_constraint::add_insn_without_violation (rtx_insn *insn)
> > +{
> > + m_free_slots -= num_occupied_slots (insn);
> > + return m_free_slots >= 0;
> > +}
> > +
> > +/* Reset the number of free slots to the maximum number of slots.
> > + This is called when the next dispatch window is started. */
> > +void
> > +dispatch_constraint::reset_slots ()
> > +{
> > + m_free_slots = m_max_slots;
> > +}
> > +
> > +/* Print the dispatch constraint to the given FILE. */
> > +void
> > +dispatch_constraint::print_dispatch_constraint (FILE *file) const
> > +{
> > + fprintf (file, "Dispatch constraint \'%s\': %d of %d slots free\n",
> > + m_name, m_free_slots, m_max_slots);
> > +}
> > +
> > +/* If INSN matches the dispatch constraint, return the number of slots
> > + occupied, else return 0. */
> > +int
> > +dispatch_constraint::num_occupied_slots (rtx_insn *insn) const
> > +{
> > + return m_callback (insn);
> > +}
> > +
> > +/* Constructor for class dispatch_window. */
> > +dispatch_window::dispatch_window (dispatch_constraint *constraints,
> > + int num_constraints)
> > + : m_constraints (constraints), m_num_constraints (num_constraints),
> > + m_violation (false)
> > +{
> > +}
> > +
> > +/* Return TRUE iff INSN fits into the dispatch window according to
> > + to all constraints. */
> > +bool
> > +dispatch_window::fits_dispatch_window (rtx_insn *insn) const
> > +{
> > + if (INSN_CODE (insn) >= 0)
> > + {
> > + if (dump_file)
> > + {
> > + fprintf (dump_file, "Checking if insn fits into dispatch
> > window:\n");
> > + print_rtl_single (dump_file, insn);
> > + }
> > + for (int i = 0; i < m_num_constraints; i++)
> > + {
> > + if (!m_constraints[i].can_fit_insn (insn))
> > + return false;
> > + }
> > + }
> > + return true;
> > +}
> > +
> > +/* If INSN does not fit into the dispatch window, reset the constraints and
> > + start a new dispatch window.
> > + Then, add INSN to the dispatch window and set the violation flag
> > + if there is a constraint violation. */
> > +void
> > +dispatch_window::add_insn_to_window (rtx_insn *insn)
> > +{
> > + if (INSN_CODE (insn) < 0)
> > + return;
> > +
> > + if (!fits_dispatch_window (insn))
> > + {
> > + if (dump_file)
> > + fprintf (dump_file, "Window full. Starting new dispatch window.\n");
> > + reset_constraints ();
> > + }
> > +
> > + for (int i = 0; i < m_num_constraints; i++)
> > + {
> > + if (!m_constraints[i].add_insn_without_violation (insn))
> > + m_violation = true;
> Could you double check whether the indentation is okay for the line above?
> > + }
> > + if (dump_file)
> > + {
> > + fprintf (dump_file, "Insn added to dispatch window.\n");
> > + if (dump_flags & TDF_DETAILS)
> > + print_dispatch_window (dump_file);
> Please also double check the indentation of this line. It could be just
> that it is not formatted correctly by the mail text.
> > + }
> > +}
> > +
> > +/* Return TRUE iff there is a dispatch violation, i.e. one of the dispatch
> > + constraints has a negative number of free slots. */
> > +bool
> > +dispatch_window::has_violation () const
> > +{
> > + return m_violation;
> > +}
> > +
> > +/* Print information about the dispatch window to the given FILE. */
> > +void
> > +dispatch_window::print_dispatch_window (FILE *file) const
> > +{
> > + fprintf (file, "==== Current dispatch window ====\n");
> > + fprintf (file, "Violation: %s\n", m_violation ? "true" : "false");
> > + for (int i = 0; i < m_num_constraints; i++)
> > + m_constraints[i].print_dispatch_constraint (file);
> > + fprintf (file, "\n");
> > +}
> > +
> > +/* For all dispatch constraints, reset the number of free slots to the
> > + maximum number of slots.
> > + This is called when the next dispatch window is started. */
> > +void
> > +dispatch_window::reset_constraints ()
> > +{
> > + for (int i = 0; i < m_num_constraints; i++)
> > + m_constraints[i].reset_slots ();
> > + m_violation = false;
> > +}
> > +
> > +
> > +/* Initialize the dispatch window using the constraints from the tuning
> > model.
> > + This is called once at the beginning of scheduling. */
> > +void
> > +init_dispatch_window (void)
> > +{
> > + const dispatch_constraint *constraints =
> > + aarch64_tune_params.dispatch_constraints;
> > + int num_constraints = aarch64_tune_params.num_dispatch_constraints;
> > + current_dispatch_window =
> > + new dispatch_window (const_cast<dispatch_constraint *>(constraints),
> > + num_constraints);
> > +
> > + if (dump_file)
> > + {
> > + fprintf (dump_file, "DISPATCH WINDOW INITIALIZED\n");
> > + if (dump_flags & TDF_DETAILS)
> > + current_dispatch_window->print_dispatch_window (dump_file);
> > + }
> > +}
> > +
> > +/* Return TRUE if INSN is a comparison instruction. */
> > +bool
> > +is_cmp_insn (rtx_insn *insn)
> > +{
> > + enum attr_type type;
> > + type = get_attr_type (insn);
> > + return (type == TYPE_FCMPS
> > + || type == TYPE_FCMPD
> > + || type == TYPE_FCCMPS
> > + || type == TYPE_FCCMPD
> > + || GET_CODE (PATTERN (insn)) == COMPARE);
> > +}
> > +
> > +/* The next two functions implement the dispatch-scheduling target hooks
> > + for aarch64 and are the drivers of the dispatch scheduler. */
> > +void
> > +aarch64_sched_dispatch_do (rtx_insn *insn, int mode)
> > +{
> > + if (mode == DISPATCH_INIT)
> > + init_dispatch_window ();
> > + else if (mode == ADD_TO_DISPATCH_WINDOW && current_dispatch_window)
> > + current_dispatch_window->add_insn_to_window (insn);
> > +}
> > +
> > +bool
> > +aarch64_sched_dispatch (rtx_insn *insn, int action)
> > +{
> > + /* We only want dispatch scheduling to be enabled during the last
> > + scheduling pass, i.e. after reload and sched_fusion. */
> > + if ((aarch64_tune_params.extra_tuning_flags
> > + & AARCH64_EXTRA_TUNE_DISPATCH_SCHED)
> > + && reload_completed &&!sched_fusion)
> > + switch (action)
> > + {
> > + default:
> > + return false;
> Is there a particular reason `default:` is the first case clause?
> > +
> > + case IS_DISPATCH_ON:
> > + return true;
> > +
> > + case IS_CMP:
> > + return is_cmp_insn (insn);
> > +
> > + case DISPATCH_VIOLATION:
> > + return current_dispatch_window->has_violation ();
> > +
> > + case FITS_DISPATCH_WINDOW:
> > + return current_dispatch_window->fits_dispatch_window (insn);
> > + }
> > + return false;
> > +}
> > \ No newline at end of file
> > diff --git a/gcc/config/aarch64/aarch64-sched-dispatch.h
> > b/gcc/config/aarch64/aarch64-sched-dispatch.h
> > new file mode 100644
> > index 00000000000..1e0dc420139
> > --- /dev/null
> > +++ b/gcc/config/aarch64/aarch64-sched-dispatch.h
> > @@ -0,0 +1,66 @@
> > +/* Dispatch scheduling hooks for AArch64.
> > + Copyright The GNU Toolchain Authors.
> > +
> > + This file is part of GCC.
> > +
> > + GCC is free software; you can redistribute it and/or modify it
> > + under the terms of the GNU General Public License as published by
> > + the Free Software Foundation; either version 3, or (at your option)
> > + any later version.
> > +
> > + GCC is distributed in the hope that it will be useful, but
> > + WITHOUT ANY WARRANTY; without even the implied warranty of
> > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + General Public License for more details.
> > +
> > + You should have received a copy of the GNU General Public License
> > + along with GCC; see the file COPYING3. If not see
> > + <http://www.gnu.org/licenses/>. */
> > +
> > +#ifndef GCC_AARCH64_SCHED_DISPATCH_H
> > +#define GCC_AARCH64_SCHED_DISPATCH_H
> > +
> > +void aarch64_sched_dispatch_do (rtx_insn *, int);
> > +bool aarch64_sched_dispatch (rtx_insn *, int);
> > +bool is_cmp_insn (rtx_insn *);
> > +
> > +/* Describes one dispatch constraint in terms of a callback function
> > + and a number of slots per dispatch window. */
> > +class dispatch_constraint
> > +{
> > +public:
> > + dispatch_constraint (const char *, const int, int (*) (rtx_insn *));
> > + bool can_fit_insn (rtx_insn *) const;
> > + bool add_insn_without_violation (rtx_insn *);
> > + void reset_slots ();
> > + void print_dispatch_constraint (FILE *) const;
> > +
> > +private:
> > + int num_occupied_slots (rtx_insn *) const;
> > +
> > + const char *m_name;
> > + int (*m_callback) (rtx_insn *);
> > + const int m_max_slots;
> > + int m_free_slots;
> > +};
> > +
> > +/* Describes a dispatch window and keeps track of the dispatch
> > constraints. */
> > +class dispatch_window
> > +{
> > +public:
> > + dispatch_window (dispatch_constraint *constraints, int num_constraints);
> > +
> > + bool fits_dispatch_window (rtx_insn *) const;
> > + void add_insn_to_window (rtx_insn *);
> > + bool has_violation () const;
> > + void print_dispatch_window (FILE *) const;
> > +
> > +private:
> > + void reset_constraints ();
> > +
> > + dispatch_constraint *m_constraints;
> > + const int m_num_constraints;
> > + bool m_violation;
> > +};
> > +
> > +#endif /* GCC_AARCH64_SCHED_DISPATCH_H */
> > diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> > b/gcc/config/aarch64/aarch64-tuning-flags.def
> > index dd91324e9c8..9c9af451b8e 100644
> > --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> > +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> > @@ -69,4 +69,7 @@ AARCH64_EXTRA_TUNING_OPTION ("cheap_fpmr_write",
> > CHEAP_FPMR_WRITE)
> > #define AARCH64_EXTRA_TUNE_BASE (AARCH64_EXTRA_TUNE_CHEAP_SHIFT_EXTEND
> > \
> > | AARCH64_EXTRA_TUNE_FULLY_PIPELINED_FMA)
> >
> > +/* Enables dispatch scheduling. */
> > +AARCH64_EXTRA_TUNING_OPTION ("dispatch_sched", DISPATCH_SCHED)
> > +
> > #undef AARCH64_EXTRA_TUNING_OPTION
> > diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
> > index 2c5b6173d83..6cd95d41136 100644
> > --- a/gcc/config/aarch64/aarch64.cc
> > +++ b/gcc/config/aarch64/aarch64.cc
> > @@ -98,6 +98,7 @@
> > #include "ipa-prop.h"
> > #include "ipa-fnsummary.h"
> > #include "hash-map.h"
> > +#include "aarch64-sched-dispatch.h"
> >
> > /* This file should be included last. */
> > #include "target-def.h"
> > @@ -19078,6 +19079,13 @@ aarch64_override_options_internal (struct
> > gcc_options *opts)
> > SET_OPTION_IF_UNSET (opts, &global_options_set,
> > param_fully_pipelined_fma,
> > 1);
> >
> > + /* If dispatch scheduling is enabled, dispatch_constraints and
> > + num_dispatch_constraints in the tune_params struct must be defined.
> > */
> > + if (aarch64_tune_params.extra_tuning_flags
> > + & AARCH64_EXTRA_TUNE_DISPATCH_SCHED)
> > + gcc_assert (aarch64_tune_params.dispatch_constraints != NULL
> > + && aarch64_tune_params.num_dispatch_constraints > 0);
> > +
> > /* TODO: SME codegen without SVE2 is not supported, once this support
> > is added
> > remove this 'sorry' and the implicit enablement of SVE2 in the
> > checks for
> > streaming mode above in this function. */
> > @@ -32244,6 +32252,12 @@ aarch64_libgcc_floating_mode_supported_p
> > #undef TARGET_SCHED_REASSOCIATION_WIDTH
> > #define TARGET_SCHED_REASSOCIATION_WIDTH aarch64_reassociation_width
> >
> > +#undef TARGET_SCHED_DISPATCH
> > +#define TARGET_SCHED_DISPATCH aarch64_sched_dispatch
> > +
> > +#undef TARGET_SCHED_DISPATCH_DO
> > +#define TARGET_SCHED_DISPATCH_DO aarch64_sched_dispatch_do
> > +
> > #undef TARGET_DWARF_FRAME_REG_MODE
> > #define TARGET_DWARF_FRAME_REG_MODE aarch64_dwarf_frame_reg_mode
> >
> > diff --git a/gcc/config/aarch64/t-aarch64 b/gcc/config/aarch64/t-aarch64
> > index 38a8c063725..40c70a90d13 100644
> > --- a/gcc/config/aarch64/t-aarch64
> > +++ b/gcc/config/aarch64/t-aarch64
> > @@ -208,6 +208,14 @@ aarch64-ldp-fusion.o:
> > $(srcdir)/config/aarch64/aarch64-ldp-fusion.cc \
> > $(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
> > $(srcdir)/config/aarch64/aarch64-ldp-fusion.cc
> >
> > +aarch64-sched-dispatch.o:
> > $(srcdir)/config/aarch64/aarch64-sched-dispatch.cc \
> > + $(CONFIG_H) $(SYSTEM_H) $(CORETYPES_H) $(BACKEND_H) $(RTL_H) \
> > + $(INSN_ATTR_H) $(REGSET_H) sched-int.h $(DUMPFILE_H) \
> > + $(srcdir)/config/aarch64/aarch64-protos.h \
> > + $(srcdir)/config/aarch64/aarch64-sched-dispatch.h
> > + $(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
> > + $(srcdir)/config/aarch64/aarch64-sched-dispatch.cc
> > +
> > comma=,
> > MULTILIB_OPTIONS = $(subst $(comma),/, $(patsubst %, mabi=%, $(subst
> > $(comma),$(comma)mabi=,$(TM_MULTILIB_CONFIG))))
> > MULTILIB_DIRNAMES = $(subst $(comma), ,$(TM_MULTILIB_CONFIG))
> > diff --git a/gcc/config/aarch64/tuning_models/a64fx.h
> > b/gcc/config/aarch64/tuning_models/a64fx.h
> > index 2622cd80346..29ef4a1d33c 100644
> > --- a/gcc/config/aarch64/tuning_models/a64fx.h
> > +++ b/gcc/config/aarch64/tuning_models/a64fx.h
> > @@ -165,7 +165,9 @@ static const struct tune_params a64fx_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &a64fx_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_A64FX. */
> > diff --git a/gcc/config/aarch64/tuning_models/ampere1.h
> > b/gcc/config/aarch64/tuning_models/ampere1.h
> > index f033016d010..ed4f2068d12 100644
> > --- a/gcc/config/aarch64/tuning_models/ampere1.h
> > +++ b/gcc/config/aarch64/tuning_models/ampere1.h
> > @@ -105,7 +105,9 @@ static const struct tune_params ampere1_tunings =
> > | AARCH64_EXTRA_TUNE_FULLY_PIPELINED_FMA), /* tune_flags. */
> > &ere1_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_AMPERE1. */
> > diff --git a/gcc/config/aarch64/tuning_models/ampere1a.h
> > b/gcc/config/aarch64/tuning_models/ampere1a.h
> > index 41481a7f077..6aada99913e 100644
> > --- a/gcc/config/aarch64/tuning_models/ampere1a.h
> > +++ b/gcc/config/aarch64/tuning_models/ampere1a.h
> > @@ -57,7 +57,9 @@ static const struct tune_params ampere1a_tunings =
> > | AARCH64_EXTRA_TUNE_FULLY_PIPELINED_FMA), /* tune_flags. */
> > &ere1_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_AMPERE1A. */
> > diff --git a/gcc/config/aarch64/tuning_models/ampere1b.h
> > b/gcc/config/aarch64/tuning_models/ampere1b.h
> > index 2ad6003d65c..b0e49a32d88 100644
> > --- a/gcc/config/aarch64/tuning_models/ampere1b.h
> > +++ b/gcc/config/aarch64/tuning_models/ampere1b.h
> > @@ -106,7 +106,9 @@ static const struct tune_params ampere1b_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_CROSS_LOOP_FMA), /* tune_flags. */
> > &ere1b_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_AMPERE1B */
> > diff --git a/gcc/config/aarch64/tuning_models/cortexa35.h
> > b/gcc/config/aarch64/tuning_models/cortexa35.h
> > index 56168c8ebc6..a3294634bb4 100644
> > --- a/gcc/config/aarch64/tuning_models/cortexa35.h
> > +++ b/gcc/config/aarch64/tuning_models/cortexa35.h
> > @@ -56,7 +56,9 @@ static const struct tune_params cortexa35_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_CORTEXA35. */
> > diff --git a/gcc/config/aarch64/tuning_models/cortexa53.h
> > b/gcc/config/aarch64/tuning_models/cortexa53.h
> > index 83daae4d38c..ed62ccf0a2a 100644
> > --- a/gcc/config/aarch64/tuning_models/cortexa53.h
> > +++ b/gcc/config/aarch64/tuning_models/cortexa53.h
> > @@ -65,7 +65,9 @@ static const struct tune_params cortexa53_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_CORTEXA53. */
> > diff --git a/gcc/config/aarch64/tuning_models/cortexa57.h
> > b/gcc/config/aarch64/tuning_models/cortexa57.h
> > index 8da7fa9d80d..dfca057c71c 100644
> > --- a/gcc/config/aarch64/tuning_models/cortexa57.h
> > +++ b/gcc/config/aarch64/tuning_models/cortexa57.h
> > @@ -102,7 +102,9 @@ static const struct tune_params cortexa57_tunings =
> > (AARCH64_EXTRA_TUNE_RENAME_FMA_REGS), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_CORTEXA57. */
> > diff --git a/gcc/config/aarch64/tuning_models/cortexa72.h
> > b/gcc/config/aarch64/tuning_models/cortexa72.h
> > index f9a330f4a18..953104cade3 100644
> > --- a/gcc/config/aarch64/tuning_models/cortexa72.h
> > +++ b/gcc/config/aarch64/tuning_models/cortexa72.h
> > @@ -54,7 +54,9 @@ static const struct tune_params cortexa72_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_CORTEXA72. */
> > diff --git a/gcc/config/aarch64/tuning_models/cortexa73.h
> > b/gcc/config/aarch64/tuning_models/cortexa73.h
> > index 038fd0896b9..dcd2a0ac286 100644
> > --- a/gcc/config/aarch64/tuning_models/cortexa73.h
> > +++ b/gcc/config/aarch64/tuning_models/cortexa73.h
> > @@ -55,7 +55,9 @@ static const struct tune_params cortexa73_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> >
> > diff --git a/gcc/config/aarch64/tuning_models/cortexx925.h
> > b/gcc/config/aarch64/tuning_models/cortexx925.h
> > index f448493b1bc..cbd572cd424 100644
> > --- a/gcc/config/aarch64/tuning_models/cortexx925.h
> > +++ b/gcc/config/aarch64/tuning_models/cortexx925.h
> > @@ -226,7 +226,9 @@ static const struct tune_params cortexx925_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_CORTEXX925. */
> > diff --git a/gcc/config/aarch64/tuning_models/emag.h
> > b/gcc/config/aarch64/tuning_models/emag.h
> > index 264a2810299..ccb68e471aa 100644
> > --- a/gcc/config/aarch64/tuning_models/emag.h
> > +++ b/gcc/config/aarch64/tuning_models/emag.h
> > @@ -54,7 +54,9 @@ static const struct tune_params emag_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &xgene1_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_EMAG. */
> > diff --git a/gcc/config/aarch64/tuning_models/exynosm1.h
> > b/gcc/config/aarch64/tuning_models/exynosm1.h
> > index 71876df7785..6ae27f65866 100644
> > --- a/gcc/config/aarch64/tuning_models/exynosm1.h
> > +++ b/gcc/config/aarch64/tuning_models/exynosm1.h
> > @@ -138,7 +138,9 @@ static const struct tune_params exynosm1_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &exynosm1_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_EXYNOSM1. */
> > diff --git a/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
> > b/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
> > index 5dc40243fe3..041e907d38f 100644
> > --- a/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
> > +++ b/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
> > @@ -58,7 +58,9 @@ static const struct tune_params fujitsu_monaka_tunings =
> > | AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_FUJITSU_MONAKA. */
> > diff --git a/gcc/config/aarch64/tuning_models/generic.h
> > b/gcc/config/aarch64/tuning_models/generic.h
> > index a822c055fe9..a962f8f72ac 100644
> > --- a/gcc/config/aarch64/tuning_models/generic.h
> > +++ b/gcc/config/aarch64/tuning_models/generic.h
> > @@ -186,7 +186,9 @@ static const struct tune_params generic_tunings =
> > (AARCH64_EXTRA_TUNE_CSE_SVE_VL_CONSTANTS), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_GENERIC. */
> > diff --git a/gcc/config/aarch64/tuning_models/generic_armv8_a.h
> > b/gcc/config/aarch64/tuning_models/generic_armv8_a.h
> > index 01080cade46..e7d512a4423 100644
> > --- a/gcc/config/aarch64/tuning_models/generic_armv8_a.h
> > +++ b/gcc/config/aarch64/tuning_models/generic_armv8_a.h
> > @@ -186,7 +186,9 @@ static const struct tune_params generic_armv8_a_tunings
> > =
> > | AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
> > &generic_armv8_a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_GENERIC_ARMV8_A. */
> > diff --git a/gcc/config/aarch64/tuning_models/generic_armv9_a.h
> > b/gcc/config/aarch64/tuning_models/generic_armv9_a.h
> > index f76a2506f38..cff40882cfd 100644
> > --- a/gcc/config/aarch64/tuning_models/generic_armv9_a.h
> > +++ b/gcc/config/aarch64/tuning_models/generic_armv9_a.h
> > @@ -254,7 +254,9 @@ static const struct tune_params generic_armv9_a_tunings
> > =
> > | AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_GENERIC_ARMV9_A. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoverse512tvb.h
> > b/gcc/config/aarch64/tuning_models/neoverse512tvb.h
> > index 964b4ac284a..8c4ec3ac733 100644
> > --- a/gcc/config/aarch64/tuning_models/neoverse512tvb.h
> > +++ b/gcc/config/aarch64/tuning_models/neoverse512tvb.h
> > @@ -161,7 +161,9 @@ static const struct tune_params neoverse512tvb_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSE512TVB. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoversen1.h
> > b/gcc/config/aarch64/tuning_models/neoversen1.h
> > index 9dc37bd7fd6..d5425d5bf99 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversen1.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversen1.h
> > @@ -54,7 +54,9 @@ static const struct tune_params neoversen1_tunings =
> > (AARCH64_EXTRA_TUNE_BASE), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSEN1. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoversen2.h
> > b/gcc/config/aarch64/tuning_models/neoversen2.h
> > index 9fbc059ea12..d52b0404ec5 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversen2.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversen2.h
> > @@ -223,7 +223,9 @@ static const struct tune_params neoversen2_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSEN2. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoversen3.h
> > b/gcc/config/aarch64/tuning_models/neoversen3.h
> > index 78177e78e07..ce5cef6fb24 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversen3.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversen3.h
> > @@ -222,7 +222,9 @@ static const struct tune_params neoversen3_tunings =
> > | AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSEN3. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoversev1.h
> > b/gcc/config/aarch64/tuning_models/neoversev1.h
> > index f1ec7dcdda7..2f224bfb2ca 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversev1.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversev1.h
> > @@ -232,7 +232,9 @@ static const struct tune_params neoversev1_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> >
> > diff --git a/gcc/config/aarch64/tuning_models/neoversev2.h
> > b/gcc/config/aarch64/tuning_models/neoversev2.h
> > index 266d8f190a2..faf06d8e7ed 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversev2.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversev2.h
> > @@ -224,7 +224,9 @@ static const struct tune_params neoversev2_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSEV2. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoversev3.h
> > b/gcc/config/aarch64/tuning_models/neoversev3.h
> > index f5566d270da..19281096006 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversev3.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversev3.h
> > @@ -224,7 +224,9 @@ static const struct tune_params neoversev3_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSEV3. */
> > diff --git a/gcc/config/aarch64/tuning_models/neoversev3ae.h
> > b/gcc/config/aarch64/tuning_models/neoversev3ae.h
> > index 5796e52a266..329e004eab8 100644
> > --- a/gcc/config/aarch64/tuning_models/neoversev3ae.h
> > +++ b/gcc/config/aarch64/tuning_models/neoversev3ae.h
> > @@ -224,7 +224,9 @@ static const struct tune_params neoversev3ae_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
> > &generic_armv9a_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_NEOVERSEV3AE. */
> > diff --git a/gcc/config/aarch64/tuning_models/olympus.h
> > b/gcc/config/aarch64/tuning_models/olympus.h
> > index 268789db019..6f471a1c5b7 100644
> > --- a/gcc/config/aarch64/tuning_models/olympus.h
> > +++ b/gcc/config/aarch64/tuning_models/olympus.h
> > @@ -204,7 +204,9 @@ static struct tune_params olympus_tunings =
> > | AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
> > &olympus_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_OLYMPUS. */
> > diff --git a/gcc/config/aarch64/tuning_models/qdf24xx.h
> > b/gcc/config/aarch64/tuning_models/qdf24xx.h
> > index 583d30a4d17..6ef115a0882 100644
> > --- a/gcc/config/aarch64/tuning_models/qdf24xx.h
> > +++ b/gcc/config/aarch64/tuning_models/qdf24xx.h
> > @@ -130,7 +130,9 @@ static const struct tune_params qdf24xx_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &qdf24xx_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_QDF24XX. */
> > diff --git a/gcc/config/aarch64/tuning_models/saphira.h
> > b/gcc/config/aarch64/tuning_models/saphira.h
> > index 684f3951a60..c37a731649d 100644
> > --- a/gcc/config/aarch64/tuning_models/saphira.h
> > +++ b/gcc/config/aarch64/tuning_models/saphira.h
> > @@ -56,7 +56,9 @@ static const struct tune_params saphira_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &generic_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_SAPHIRA. */
> > diff --git a/gcc/config/aarch64/tuning_models/thunderx.h
> > b/gcc/config/aarch64/tuning_models/thunderx.h
> > index d79c1391b8c..f94ef397bdc 100644
> > --- a/gcc/config/aarch64/tuning_models/thunderx.h
> > +++ b/gcc/config/aarch64/tuning_models/thunderx.h
> > @@ -111,7 +111,9 @@ static const struct tune_params thunderx_tunings =
> > (AARCH64_EXTRA_TUNE_CHEAP_SHIFT_EXTEND), /* tune_flags. */
> > &thunderx_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_THUNDERX. */
> > diff --git a/gcc/config/aarch64/tuning_models/thunderx2t99.h
> > b/gcc/config/aarch64/tuning_models/thunderx2t99.h
> > index 513c6158084..ea5cc39f0e4 100644
> > --- a/gcc/config/aarch64/tuning_models/thunderx2t99.h
> > +++ b/gcc/config/aarch64/tuning_models/thunderx2t99.h
> > @@ -131,7 +131,9 @@ static const struct tune_params thunderx2t99_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &thunderx2t99_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_THUNDERX2T99. */
> > diff --git a/gcc/config/aarch64/tuning_models/thunderx3t110.h
> > b/gcc/config/aarch64/tuning_models/thunderx3t110.h
> > index a2547b8e54d..a3a3d72ce43 100644
> > --- a/gcc/config/aarch64/tuning_models/thunderx3t110.h
> > +++ b/gcc/config/aarch64/tuning_models/thunderx3t110.h
> > @@ -130,7 +130,9 @@ static const struct tune_params thunderx3t110_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &thunderx3t110_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_THUNDERX3T110. */
> > diff --git a/gcc/config/aarch64/tuning_models/thunderxt88.h
> > b/gcc/config/aarch64/tuning_models/thunderxt88.h
> > index 6be5c526fc1..85b06e96233 100644
> > --- a/gcc/config/aarch64/tuning_models/thunderxt88.h
> > +++ b/gcc/config/aarch64/tuning_models/thunderxt88.h
> > @@ -66,7 +66,9 @@ static const struct tune_params thunderxt88_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &thunderxt88_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_THUNDERXT88. */
> > diff --git a/gcc/config/aarch64/tuning_models/tsv110.h
> > b/gcc/config/aarch64/tuning_models/tsv110.h
> > index 458286e0e80..450f13390d0 100644
> > --- a/gcc/config/aarch64/tuning_models/tsv110.h
> > +++ b/gcc/config/aarch64/tuning_models/tsv110.h
> > @@ -131,7 +131,9 @@ static const struct tune_params tsv110_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &tsv110_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_TSV110. */
> > diff --git a/gcc/config/aarch64/tuning_models/xgene1.h
> > b/gcc/config/aarch64/tuning_models/xgene1.h
> > index b4f01ee92f5..f69a5de6fe2 100644
> > --- a/gcc/config/aarch64/tuning_models/xgene1.h
> > +++ b/gcc/config/aarch64/tuning_models/xgene1.h
> > @@ -139,7 +139,9 @@ static const struct tune_params xgene1_tunings =
> > (AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
> > &xgene1_prefetch_tune,
> > AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
> > - AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
> > + AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
> > + nullptr, /* dispatch_constraints. */
> > + 0 /* num_dispatch_constraints. */
> > };
> >
> > #endif /* GCC_AARCH64_H_XGENE1. */
>
