[PATCH v4 06/54] tcg/i386: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment, and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns HostAddress and TCGLabelQemuLdst structures. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 344 +

[PATCH v4 10/54] tcg/aarch64: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment, and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns HostAddress and TCGLabelQemuLdst structures. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 313 ++

[PATCH v4 30/54] tcg: Move TCGLabelQemuLdst to tcg.c

Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/tcg.c | 13 + tcg/tcg-ldst.c.inc | 14 -- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index cfd3262a4a..6f5daaee5f 100644 --- a/tcg/tcg.c +++ b

[PATCH v4 15/54] tcg/loongarch64: Introduce HostAddress

Collect the 2 parts of the host address into a struct. Reorg tcg_out_qemu_{ld,st}_direct to use it. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 55 +--- 1 file changed, 30 insertions(+), 25 deletions(-)

[PATCH v4 36/54] tcg/i386: Convert tcg_out_qemu_st_slow_path

Use tcg_out_st_helper_args. This eliminates the use of a tail call to the store helper. This may or may not be an improvement, depending on the call/return branch prediction of the host microarchitecture. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 57 +++--

[PATCH v4 37/54] tcg/aarch64: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 40 +++- 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/t

[PATCH v4 39/54] tcg/loongarch64: Convert tcg_out_qemu_{ld, st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 37 ++-- 1 file changed, 11 insertions(+), 26 deletions(-) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loong

[PATCH v4 12/54] tcg/arm: Introduce HostAddress

Collect the parts of the host address, and condition, into a struct. Merge tcg_out_qemu_*_{index,direct} and use it. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 248 ++- 1 file changed, 115 insertions(+), 133 deletions(-) diff --git a/tcg/

[PATCH v4 08/54] tcg/aarch64: Rationalize args to tcg_out_qemu_{ld, st}

Rename the 'ext' parameter 'data_type' to make the use clearer; pass it to tcg_out_qemu_st as well to even out the interfaces. Rename the 'otype' local 'addr_type' to make the use clearer. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 36

[PATCH v4 28/54] tcg/sparc64: Drop is_64 test from tcg_out_qemu_ld data return

In tcg_canonicalize_memop, we remove MO_SIGN from MO_32 operations with TCG_TYPE_I32. Thus this is never set. We already have an identical test just above which does not include is_64 Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 2 +-

[PATCH v4 33/54] tcg: Widen helper_*_st[bw]_mmu val arguments

While the old type was correct in the ideal sense, some ABIs require the argument to be zero-extended. Using uint32_t for all such values is a decent compromise. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 10 +++--- accel/tcg/cputlb.c

[PATCH v4 54/54] tcg/s390x: Simplify constraints on qemu_ld/st

Adjust the softmmu tlb to use R0+R1, not any of the normally available registers. Since we handle overlap betwen inputs and helper arguments, we can allow any allocatable reg. Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target-con-set.h | 2 -- tcg/s390x/tcg-target-con-str.h | 1 - tcg

[PATCH v4 13/54] tcg/arm: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns HostAddress and TCGLabelQemuLdst structures. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 351 ++

[PATCH v4 49/54] tcg/ppc: Adjust constraints on qemu_ld/st

The softmmu tlb uses TCG_REG_{TMP1,TMP2,R0}, not any of the normally available registers. Now that we handle overlap betwen inputs and helper arguments, we can allow any allocatable reg. Reviewed-by: Daniel Henrique Barboza Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target-con-set.h | 11

[PATCH v4 35/54] tcg/i386: Convert tcg_out_qemu_ld_slow_path

Use tcg_out_ld_helper_args and tcg_out_ld_helper_ret. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 71 +++ 1 file changed, 28 insertions(+), 43 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 8752968af

[PATCH v4 44/54] tcg/loongarch64: Simplify constraints on qemu_ld/st

The softmmu tlb uses TCG_REG_TMP[0-2], not any of the normally available registers. Now that we handle overlap betwen inputs and helper arguments, we can allow any allocatable reg. Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target-con-set.h | 2 -- tcg/loongarch64/tcg-target-con-

[PATCH v4 43/54] tcg/s390x: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target.c.inc | 35 ++- 1 file changed, 10 insertions(+), 25 deletions(-) diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-targ

[PATCH v4 04/54] tcg/i386: Drop r0+r1 local variables from tcg_out_tlb_load

Use TCG_REG_L[01] constants directly. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 32 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc i

[PATCH v4 23/54] tcg/riscv: Rationalize args to tcg_out_qemu_{ld,st}

Interpret the variable argument placement in the caller. Pass data_type instead of is64 -- there are several places where we already convert back from bool to type. Clean things up by using type throughout. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Daniel Henrique Barboza Signed-off-by:

[PATCH v4 01/54] tcg/i386: Rationalize args to tcg_out_qemu_{ld,st}

Interpret the variable argument placement in the caller. Pass data_type instead of is64 -- there are several places where we already convert back from bool to type. Clean things up by using type throughout. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/i386/tcg-

[PATCH v4 15/57] accel/tcg: Use have_atomic16 in ldst_atomicity.c.inc

Hosts using Intel and AMD AVX cpus are quite common. Add fast paths through ldst_atomicity using this. Only enable with CONFIG_INT128; some older clang versions do not support __int128_t, and the inline assembly won't work on structures. Signed-off-by: Richard Henderson --- accel/tcg/ldst_atomi

[PATCH v4 20/57] tcg: Introduce TCG_OPF_TYPE_MASK

Reorg TCG_OPF_64BIT and TCG_OPF_VECTOR into a two-bit field so that we can add TCG_OPF_128BIT without requiring another bit. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- include/tcg/tcg.h| 22 -- tcg/optimize.c | 15 +

[PATCH v4 09/54] tcg/aarch64: Introduce HostAddress

Collect the 3 potential parts of the host address into a struct. Reorg tcg_out_qemu_{ld,st}_direct to use it. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 86 +--- 1 file changed, 59 insertions(+), 27 del

[PATCH v4 10/57] accel/tcg: Implement helper_{ld, st}*_mmu for user-only

TCG backends may need to defer to a helper to implement the atomicity required by a given operation. Mirror the interface used in system mode. Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 6 +- accel/tcg/user-exec.c | 393 - tcg/tcg.c

[PATCH v4 51/54] tcg/ppc: Remove unused constraint J

Never used since its introduction. Fixes: 3d582c6179c ("tcg-ppc64: Rearrange integer constant constraints") Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target-con-str.h | 1 - tcg/ppc/tcg-target.c.inc | 3 --- 2 files changed, 4 deletions(-) diff --git a/tcg/ppc/tcg-target-con-str.h b/

[PATCH v4 38/54] tcg/arm: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. This allows our local tcg_out_arg_* infrastructure to be removed. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 140 +-- 1 file changed, 18 insertions(+), 122 del

[PATCH v4 18/54] tcg/mips: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment, and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns HostAddress and TCGLabelQemuLdst structures. Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.c.inc | 404 +

[PATCH v4 46/54] tcg/mips: Reorg tlb load within prepare_host_addr

Compare the address vs the tlb entry with sign-extended values. This simplifies the page+alignment mask constant, and the generation of the last byte address for the misaligned test. Move the tlb addend load up, and the zero-extension down. This frees up a register, which allows us use TMP3 as th

[PATCH v4 00/54] tcg: Simplify calls to load/store helpers

v1: https://lore.kernel.org/qemu-devel/20230408024314.3357414-1-richard.hender...@linaro.org/ v2: https://lore.kernel.org/qemu-devel/20230411010512.5375-1-richard.hender...@linaro.org/ v3: https://lore.kernel.org/qemu-devel/20230424054105.1579315-1-richard.hender...@linaro.org/ There are severa

[PATCH v4 19/54] tcg/ppc: Rationalize args to tcg_out_qemu_{ld,st}

Interpret the variable argument placement in the caller. Pass data_type instead of is64 -- there are several places where we already convert back from bool to type. Clean things up by using type throughout. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Daniel Henrique Barboza Signed-off-by:

[PATCH v4 14/57] tcg/i386: Add have_atomic16

Notice when Intel or AMD have guaranteed that vmovdqa is atomic. The new variable will also be used in generated code. Signed-off-by: Richard Henderson --- include/qemu/cpuid.h | 18 ++ tcg/i386/tcg-target.h | 1 + tcg/i386/tcg-target.c.inc | 27

[PATCH v4 50/54] tcg/ppc: Remove unused constraints A, B, C, D

These constraints have not been used for quite some time. Fixes: 77b73de67632 ("Use rem/div[u]_i32 drop div[u]2_i32") Reviewed-by: Daniel Henrique Barboza Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target-con-str.h | 4 1 file changed, 4 deletions

[PATCH v4 19/57] accel/tcg: Add have_lse2 support in ldst_atomicity

Add fast paths for FEAT_LSE2, using the detection in tcg. Signed-off-by: Richard Henderson --- accel/tcg/ldst_atomicity.c.inc | 37 ++ 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc in

[PATCH v4 11/54] tcg/arm: Rationalize args to tcg_out_qemu_{ld,st}

Interpret the variable argument placement in the caller. Pass data_type instead of is_64. We need to set this in TCGLabelQemuLdst, so plumb this all the way through from tcg_out_op. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 113

[PATCH v4 48/54] tcg/ppc: Reorg tcg_out_tlb_read

Allocate TCG_REG_TMP2. Use R0, TMP1, TMP2 instead of any of the normally allocated registers for the tlb load. Reviewed-by: Daniel Henrique Barboza Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 84 1 file changed, 51 insertions(+), 33

[PATCH v4 05/57] accel/tcg: Reorg system mode store helpers

Instead of trying to unify all operations on uint64_t, use mmu_lookup() to perform the basic tlb hit and resolution. Create individual functions to handle access by size. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 408 + 1 file changed,

[PATCH v4 17/54] tcg/mips: Rationalize args to tcg_out_qemu_{ld,st}

Interpret the variable argument placement in the caller. There are several places where we already convert back from bool to type. Clean things up by using type throughout. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.c.inc | 186

[PATCH v4 04/57] accel/tcg: Reorg system mode load helpers

Instead of trying to unify all operations on uint64_t, pull out mmu_lookup() to perform the basic tlb hit and resolution. Create individual functions to handle access by size. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 644 +---

[PATCH v4 29/54] tcg/sparc64: Pass TCGType to tcg_out_qemu_{ld,st}

We need to set this in TCGLabelQemuLdst, so plumb this all the way through from tcg_out_op. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.in

[PATCH v4 08/57] target/loongarch: Do not include tcg-ldst.h

This header is supposed to be private to tcg and in fact does not need to be included here at all. Reviewed-by: Song Gao Signed-off-by: Richard Henderson --- target/loongarch/csr_helper.c | 1 - target/loongarch/iocsr_helper.c | 1 - 2 files changed, 2 deletions(-) diff --git a/target/loonga

[PATCH v4 47/54] tcg/mips: Simplify constraints on qemu_ld/st

The softmmu tlb uses TCG_REG_TMP[0-3], not any of the normally available registers. Now that we handle overlap betwen inputs and helper arguments, and have eliminated use of A0, we can allow any allocatable reg. Signed-off-by: Richard Henderson --- tcg/mips/tcg-target-con-set.h | 13 +--

[PATCH v4 22/54] tcg/riscv: Require TCG_TARGET_REG_BITS == 64

The port currently does not support "oversize" guests, which means riscv32 can only target 32-bit guests. We will soon be building TCG once for all guests. This implies that we can only support riscv64. Since all Linux distributions target riscv64 not riscv32, this is not much of a restriction a

[PATCH v4 03/57] accel/tcg: Introduce tlb_read_idx

Instead of playing with offsetof in various places, use MMUAccessType to index an array. This is easily defined instead of the previous dummy padding array in the union. Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- include/exec/cpu-defs.h |

[PATCH v4 21/54] tcg/ppc: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment, and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns HostAddress and TCGLabelQemuLdst structures. Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 377 ++

[PATCH v4 43/57] tcg/i386: Use atom_and_align_for_opc

No change to the ultimate load/store routines yet, so some atomicity conditions not yet honored, but plumbs the change to alignment through the relevant functions. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 34 ++ 1 file changed, 22 insertion

[PATCH v4 45/57] tcg/arm: Use atom_and_align_for_opc

No change to the ultimate load/store routines yet, so some atomicity conditions not yet honored, but plumbs the change to alignment through the relevant functions. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-)

[PATCH v4 24/57] tcg/loongarch64: Use full load/store helpers in user-only mode

Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 30

[PATCH v4 27/57] tcg/arm: Use full load/store helpers in user-only mode

Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 45

[PATCH v4 38/57] tcg/riscv: Support softmmu unaligned accesses

The system is required to emulate unaligned accesses, even if the hardware does not support it. The resulting trap may or may not be more efficient than the qemu slow path. There are linux kernel patches in flight to allow userspace to query hardware support; we can re-evaluate whether to enable

[PATCH v4 56/57] tcg/ppc: Support 128-bit load/store

Use LQ/STQ with ISA v2.07, and 16-byte atomicity is required. Note that these instructions do not require 16-byte alignment. Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target-con-set.h | 2 + tcg/ppc/tcg-target-con-str.h | 1 + tcg/ppc/tcg-target.h | 3 +- tcg/ppc/tcg-target.

[PATCH v4 48/57] tcg/ppc: Use atom_and_align_for_opc

Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index f0a4118bbb..60375804cd 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc

[PATCH v4 32/54] tcg: Introduce arg_slot_stk_ofs

Unify all computation of argument stack offset in one function. This requires that we adjust ref_slot to be in the same units, by adding max_reg_slots during init_call_layout. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/tcg.c | 29 +

[PATCH v4 35/57] accel/tcg: Remove helper_unaligned_{ld,st}

These functions are now unused. Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 6 -- accel/tcg/user-exec.c | 10 -- 2 files changed, 16 deletions(-) diff --git a/include/tcg/tcg-ldst.h b/include/tcg/tcg-ldst.h index 64f48e6990..7dd57013e9 100644 --- a/include/tcg/tc

[PATCH v4 34/54] tcg: Add routines for calling slow-path helpers

Add tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. These and their subroutines use the existing knowledge of the host function call abi to load the function call arguments and return results. These will be used to simplify the backends in turn. Signed-off-by: Richard

[PATCH v4 34/57] tcg/sparc64: Use standard slow path for softmmu

Drop the target-specific trampolines for the standard slow path. This lets us use tcg_out_helper_{ld,st}_args, and handles the new atomicity bits within MemOp. At the same time, use the full load/store helpers for user-only mode. Drop inline unaligned access support for user-only mode, as it does

[PATCH v4 52/57] tcg/i386: Honor 64-bit atomicity in 32-bit mode

Use the fpu to perform 64-bit loads and stores. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 44 +-- 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 3e21f067d6..5c6c

[PATCH v4 33/57] tcg/sparc64: Split out tcg_out_movi_s32

Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index e244209890..4375a06377 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg

[PATCH v4 42/54] tcg/riscv: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. Reviewed-by: Daniel Henrique Barboza Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.c.inc | 37 ++--- 1 file changed, 10 insertions(+), 27 deletions(-) diff --git a/tcg/ris

[PATCH v4 22/57] tcg/aarch64: Use full load/store helpers in user-only mode

Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 35

[PATCH v4 03/54] tcg/i386: Introduce HostAddress

Collect the 4 potential parts of the host address into a struct. Reorg tcg_out_qemu_{ld,st}_direct to use it. Reorg guest_base handling to use it. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 165 +- 1 f

[PATCH v4 57/57] tcg/s390x: Support 128-bit load/store

Use LPQ/STPQ when 16-byte atomicity is required. Note that these instructions require 16-byte alignment. Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target-con-set.h | 2 + tcg/s390x/tcg-target.h | 2 +- tcg/s390x/tcg-target.c.inc | 100 - 3

[PATCH v4 07/54] tcg/i386: Use indexed addressing for softmmu fast path

Since tcg_out_{ld,st}_helper_args, the slow path no longer requires the address argument to be set up by the tlb load sequence. Use a plain load for the addend and indexed addressing with the original input address register. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 25 ++

[PATCH v4 23/57] tcg/ppc: Use full load/store helpers in user-only mode

Instead of using helper_unaligned_{ld,st}, use the full load/store helpers. This will allow the fast path to increase alignment to implement atomicity while not immediately raising an alignment exception. Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 44

[PATCH v4 12/57] tcg: Add 128-bit guest memory primitives

Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h| 3 + include/tcg/tcg-ldst.h | 4 + accel/tcg/cputlb.c | 392 + accel/tcg/user-exec.c | 94 ++-- tcg/tcg-op.c | 184 +++- accel/

[PATCH v4 11/57] tcg/tci: Use helper_{ld,st}*_mmu for user-only

We can now fold these two pieces of code. Signed-off-by: Richard Henderson --- tcg/tci.c | 89 --- 1 file changed, 89 deletions(-) diff --git a/tcg/tci.c b/tcg/tci.c index 5bde2e1f2e..15f2f8c463 100644 --- a/tcg/tci.c +++ b/tcg/tci.c @@ -292,7

[PATCH v4 25/54] tcg/s390x: Pass TCGType to tcg_out_qemu_{ld,st}

We need to set this in TCGLabelQemuLdst, so plumb this all the way through from tcg_out_op. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target.c.inc | 22 ++ 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/tcg/s390x/t

[PATCH v4 53/57] tcg/i386: Support 128-bit load/store with have_atomic16

Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 3 +- tcg/i386/tcg-target.c.inc | 184 +- 2 files changed, 182 insertions(+), 5 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 943af6775e..7f69997e30 100644 --- a

[PATCH v4 05/54] tcg/i386: Introduce tcg_out_testi

Split out a helper for choosing testb vs testl. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 30 ++ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target

[PATCH v4 54/57] tcg/aarch64: Rename temporaries

We will need to allocate a second general-purpose temporary. Rename the existing temps to add a distinguishing number. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 50 ++-- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/tcg/

[PATCH v4 50/57] tcg/s390x: Use atom_and_align_for_opc

Signed-off-by: Richard Henderson --- tcg/s390x/tcg-target.c.inc | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 22f0206b5a..ddd9860a6a 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-targ

[PATCH v4 46/57] tcg/loongarch64: Use atom_and_align_for_opc

Signed-off-by: Richard Henderson --- tcg/loongarch64/tcg-target.c.inc | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 62bf823084..43341524f2 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg

[PATCH v4 55/57] tcg/aarch64: Support 128-bit load/store

Use LDXP+STXP when LSE2 is not present and 16-byte atomicity is required, and LDP/STP otherwise. This requires allocating a second general-purpose temporary, as Rs cannot overlap Rn in STXP. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target-con-set.h | 2 + tcg/aarch64/tcg-target.h

[PATCH v4 31/57] tcg/sparc64: Rename tcg_out_movi_imm13 to tcg_out_movi_s13

Emphasize that the constant is signed. Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 30 +++--- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 64464ab363..2e6127d506 1

[PATCH v4 16/57] accel/tcg: Add aarch64 specific support in ldst_atomicity

We have code in atomic128.h noting that through GCC 8, there was no support for atomic operations on __uint128. This has been fixed in GCC 10. But we can still improve over any basic compare-and-swap loop using the ldxp/stxp instructions. Signed-off-by: Richard Henderson --- accel/tcg/ldst_ato

[PATCH v4 41/54] tcg/ppc: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. Reviewed-by: Daniel Henrique Barboza Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 88 1 file changed, 26 insertions(+), 62 deletions(-) diff --git a/tcg/pp

[PATCH v4 01/57] include/exec/memop: Add bits describing atomicity

These bits may be used to describe the precise atomicity requirements of the guest, which may then be used to constrain the methods by which it may be emulated by the host. For instance, the AArch64 LDP (32-bit) instruction changes semantics with ARMv8.4 LSE2, from MO_64 | MO_ATMAX_4 | MO_ATOM_

[PATCH v4 44/57] tcg/aarch64: Use atom_and_align_for_opc

Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 38 +++- 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 8e5f3d3688..1d6d382edd 100644 --- a/tcg/aarch64/tcg-target

[PATCH v4 42/57] tcg: Introduce atom_and_align_for_opc

Examine MemOp for atomicity and alignment, adjusting alignment as required to implement atomicity on the host. Signed-off-by: Richard Henderson --- tcg/tcg.c | 69 +++ 1 file changed, 69 insertions(+) diff --git a/tcg/tcg.c b/tcg/tcg.c index 3

[PATCH 09/84] tcg: Reduce copies for plugin_gen_mem_callbacks

We only need to make copies for loads, when the destination overlaps the address. For now, only eliminate the copy for stores and 128-bit loads. Rename plugin_prep_mem_callbacks to plugin_maybe_preserve_addr, returning NULL if no copy is made. Signed-off-by: Richard Henderson --- tcg/tcg-op-ld

[PATCH v4 39/57] tcg: Introduce tcg_target_has_memory_bswap

Replace the unparameterized TCG_TARGET_HAS_MEMORY_BSWAP macro with a function with a memop argument. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 1 - tcg/arm/tcg-target.h | 1 - tcg/i386/tcg-target.h| 3 --- tcg/loongarch64/tcg-target.h

[PATCH v4 40/57] tcg: Add INDEX_op_qemu_{ld,st}_i128

Add opcodes for backend support for 128-bit memory operations. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- include/tcg/tcg-opc.h| 8 + tcg/aarch64/tcg-target.h | 2 ++ tcg/arm/tcg-target.h | 2 ++ tcg/i386/tcg-target.h| 2 ++ tcg/

[PATCH 09/84] tcg: Reduce copies for plugin_gen_mem_callbacks

We only need to make copies for loads, when the destination overlaps the address. For now, only eliminate the copy for stores and 128-bit loads. Rename plugin_prep_mem_callbacks to plugin_maybe_preserve_addr, returning NULL if no copy is made. Signed-off-by: Richard Henderson --- tcg/tcg-op-ld

[PATCH 03/84] accel/tcg: Widen tcg-ldst.h addresses to uint64_t

Always pass the target address as uint64_t. Adjust tcg_out_{ld,st}_helper_args to match. Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 26 +- accel/tcg/cputlb.c | 26 +- accel/tcg/user-exec.c | 26 +- tcg/tcg.c | 6

[PATCH v4 31/54] tcg: Replace REG_P with arg_loc_reg_p

An inline function is safer than a macro, and REG_P was rather too generic. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/tcg-internal.h | 4 tcg/tcg.c | 16 +--- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/tcg/tcg-in

[PATCH 02/84] tcg: Widen gen_insn_data to uint64_t

We already pass uint64_t to restore_state_to_opc; this changes all of the other uses from insn_start through the encoding to decoding. Signed-off-by: Richard Henderson --- include/tcg/tcg-op.h | 39 +-- include/tcg/tcg-opc.h | 2 +- include/tcg/tcg.h

[PATCH v4 17/57] tcg/aarch64: Detect have_lse, have_lse2 for linux

Notice when the host has additional atomic instructions. The new variables will also be used in generated code. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 3 +++ tcg/aarch64/tcg-target.c.inc | 12 2 files changed, 15 ins

[PATCH v4 18/57] tcg/aarch64: Detect have_lse, have_lse2 for darwin

These features are present for Apple M1. Tested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.c.inc | 28 1 file changed, 28 insertions(+) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/

[PATCH v4 02/57] accel/tcg: Add cpu_in_serial_context

Like cpu_in_exclusive_context, but also true if there is no other cpu against which we could race. Use it in tb_flush as a direct replacement. Use it in cpu_loop_exit_atomic to ensure that there is no loop against cpu_exec_step_atomic. Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé

[PULL v2 02/12] accel/tcg: Uncache the host address for instruction fetch when tlb size < 1

From: Weiwei Li When PMP entry overlap part of the page, we'll set the tlb_size to 1, which will make the address in tlb entry set with TLB_INVALID_MASK, and the next access will again go through tlb_fill.However, this way will not work in tb_gen_code() => get_page_addr_code_hostp(): the TLB host

[PATCH 04/84] tcg: Widen helper_{ld,st}_i128 addresses to uint64_t

Always pass the target address as uint64_t. Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h | 4 ++-- accel/tcg/cputlb.c | 5 ++--- accel/tcg/user-exec.c | 5 ++--- tcg/tcg-op-ldst.c | 26 -- 4 files changed, 30 insertions(+), 10 deletions(-)

[PATCH v4 00/57] tcg: Improve atomicity support

v1: https://lore.kernel.org/qemu-devel/20221118094754.242910-1-richard.hender...@linaro.org/ v2: https://lore.kernel.org/qemu-devel/20230216025739.1211680-1-richard.hender...@linaro.org/ v3: https://lore.kernel.org/qemu-devel/20230425193146.2106111-1-richard.hender...@linaro.org/ Based-on: 2023

[PATCH 15/84] tcg/tci: Elimnate TARGET_LONG_BITS, target_ulong

We now have the address size as part of the opcode, so we no longer need to test TARGET_LONG_BITS. We can use uint64_t for target_ulong, as passed into load/store helpers. Signed-off-by: Richard Henderson --- tcg/tci.c| 61 +--- tcg/tci/tcg-ta

[PULL v2 07/12] qemu/int128: Re-shuffle Int128Alias members

Clang 14, with --enable-tcg-interpreter errors with include/qemu/int128.h:487:16: error: alignment of field 'i' (128 bits) does not match the alignment of the first field in transparent union; transparent_union attribute ignored [-Werror,-Wignored-attributes] __int128_t i; ^

[PATCH v4 13/57] meson: Detect atomic128 support with optimization

There is an edge condition prior to gcc13 for which optimization is required to generate 16-byte atomic sequences. Detect this. Signed-off-by: Richard Henderson --- accel/tcg/ldst_atomicity.c.inc | 38 ++--- meson.build| 52 ++-

[PATCH v4 24/54] tcg/riscv: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment, and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns TCGReg and TCGLabelQemuLdst. Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.c.inc | 253 +---

[PULL v2 04/12] qemu/host-utils.h: Add clz and ctz functions for lower-bit integers

From: Kiran Ostrolenk This is for use in the RISC-V vclz and vctz instructions (implemented in proceeding commit). Signed-off-by: Kiran Ostrolenk Reviewed-by: Richard Henderson Message-Id: <20230428144757.57530-11-lawrence.hun...@codethink.co.uk> Signed-off-by: Richard Henderson --- include/

[PATCH 68/84] target/arm: Tidy helpers for translation

Move most includes from *translate*.c to translate.h, ensuring that we get the ordering correct. Ensure cpu.h is first. Use disas/disas.h instead of exec/log.h. Drop otherwise unused includes. Signed-off-by: Richard Henderson --- target/arm/tcg/translate.h| 3 +++ target/arm/tcg/transl

[PULL v2 05/12] tcg: Add tcg_gen_gvec_andcs

From: Nazar Kazakov Add tcg expander and helper functions for and-compliment vector with scalar operand. Signed-off-by: Nazar Kazakov Message-Id: <20230428144757.57530-10-lawrence.hun...@codethink.co.uk> [rth: Split out of larger patch.] Signed-off-by: Richard Henderson --- accel/tcg/tcg-runt

[PATCH v4 40/54] tcg/mips: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret, and tcg_out_st_helper_args. This allows our local tcg_out_arg_* infrastructure to be removed. We are no longer filling the call or return branch delay slots, nor are we tail-calling for the store, but this seems a small price to pay. Signed-off-

[PATCH 42/84] tcg: Split out tcg/oversized-guest.h

Move a use of TARGET_LONG_BITS out of tcg/tcg.h. Include the new file only where required. Signed-off-by: Richard Henderson --- include/exec/cpu_ldst.h | 3 +-- include/tcg/oversized-guest.h | 23 +++ include/tcg/tcg.h | 9 - accel/tcg/cputlb.c

  1   2   3   4   5   6   >