[PATCH v6 17/19] migration/multifd: Make MultiFDMethods const

2024-08-27 Thread Fabiano Rosas
The methods are defined at module_init time and don't ever change. Make them const. Suggested-by: Philippe Mathieu-Daudé Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-nocomp.c | 2 +- migration/multifd-qpl.c| 2 +- migration/multifd-uadk.c | 2 +- migration/mul

[PATCH v6 12/19] migration/multifd: Replace multifd_send_state->pages with client data

2024-08-27 Thread Fabiano Rosas
Multifd currently has a simple scheduling mechanism that distributes work to the various channels by keeping storage space within each channel and an extra space that is given to the client. Each time the client fills the space with data and calls into multifd, that space is given to the next idle

[PATCH v6 15/19] migration/multifd: Register nocomp ops dynamically

2024-08-27 Thread Fabiano Rosas
Prior to moving the ram code into multifd-nocomp.c, change the code to register the nocomp ops dynamically so we don't need to have the ops structure defined in multifd.c. While here, move the ops struct initialization to the end of the file to make the next diff cleaner. Reviewed-by: Prasad Pand

[PATCH v6 14/19] migration/multifd: Standardize on multifd ops names

2024-08-27 Thread Fabiano Rosas
Add the multifd_ prefix to all functions and remove the useless docstrings. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 57 migration/multifd-uadk.c | 55 --- migration/multifd-zlib.c | 81 ++-

[PATCH v6 13/19] migration/multifd: Allow multifd sync without flush

2024-08-27 Thread Fabiano Rosas
Separate the multifd sync from flushing the client data to the channels. These two operations are closely related but not strictly necessary to be executed together. The multifd sync is intrinsic to how multifd works. The multiple channels operate independently and may finish IO out of order in re

[PATCH v6 05/19] migration/multifd: Introduce MultiFDSendData

2024-08-27 Thread Fabiano Rosas
Add a new data structure to replace p->pages in the multifd channel. This new structure will hide the multifd payload type behind an union, so we don't need to add a new field to the channel each time we want to handle a different data type. This also allow us to keep multifd_send_pages() as is, w

[PATCH v6 16/19] migration/multifd: Move nocomp code into multifd-nocomp.c

2024-08-27 Thread Fabiano Rosas
In preparation for adding new payload types to multifd, move most of the no-compression code into multifd-nocomp.c. Let's try to keep a semblance of layering by not mixing general multifd control flow with the details of transmitting pages of ram. There are still some pieces leftover, namely the p

[PATCH v6 09/19] migration/multifd: Remove total pages tracing

2024-08-27 Thread Fabiano Rosas
The total_normal_pages and total_zero_pages elements are used only for the end tracepoints of the multifd threads. These are not super useful since they record per-channel numbers and are just the sum of all the pages that are transmitted per-packet, for which we already have tracepoints. Remove th

[PATCH v6 00/19] migration/multifd: Remove multifd_send_state->pages

2024-08-27 Thread Fabiano Rosas
Hi, Added some docs for the multifd methods since I removed the docstrings from the various implementations. For the recv packet cleanup, I decided to go with the const packet suggestion. I generally prefer to not have that sort of side effect, even if it's for sanitization. CI run: https://gitl

[PATCH v6 11/19] migration/multifd: Don't send ram data during SYNC

2024-08-27 Thread Fabiano Rosas
Skip saving and loading any ram data in the packet in the case of a SYNC. This fixes a shortcoming of the current code which requires a reset of the MultiFDPages_t fields right after the previous pending_job finishes, otherwise the very next job might be a SYNC and multifd_send_fill_packet() will p

[PATCH v6 04/19] migration/multifd: Pass in MultiFDPages_t to file_write_ramblock_iov

2024-08-27 Thread Fabiano Rosas
We want to stop dereferencing 'pages' so it can be replaced by an opaque pointer in the next patches. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/file.c| 3 ++- migration/file.h| 2 +- migration/multifd.c | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) dif

[PATCH v6 07/19] migration/multifd: Replace p->pages with an union pointer

2024-08-27 Thread Fabiano Rosas
We want multifd to be able to handle more types of data than just ram pages. To start decoupling multifd from pages, replace p->pages (MultiFDPages_t) with the new type MultiFDSendData that hides the client payload inside an union. The general idea here is to isolate functions that *need* to handl

[PATCH v6 18/19] migration/multifd: Stop changing the packet on recv side

2024-08-27 Thread Fabiano Rosas
As observed by Philippe, the multifd_ram_unfill_packet() function currently leaves the MultiFDPacket structure with mixed endianness. This is harmless, but ultimately not very clean. Aside from that, the packet is also written to on the recv side to ensure the ramblock name is null-terminated. Sto

[PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Fabiano Rosas
Add documentation clarifying the usage of the multifd methods. The general idea is that the client code calls into multifd to trigger send/recv of data and multifd then calls these hooks back from the worker threads at opportune moments so the client can process a portion of the data. Suggested-by

[PATCH v6 08/19] migration/multifd: Move pages accounting into multifd_send_zero_page_detect()

2024-08-27 Thread Fabiano Rosas
All references to pages are being removed from the multifd worker threads in order to allow multifd to deal with different payload types. multifd_send_zero_page_detect() is called by all multifd migration paths that deal with pages and is the last spot where zero pages and normal page amounts are

[PATCH v6 03/19] migration/multifd: Remove pages->allocated

2024-08-27 Thread Fabiano Rosas
This value never changes and is always the same as page_count. We don't need a copy of it per-channel plus one in the extra slot. Remove it. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd.c | 6 ++ migration/multifd.h | 2 -- 2 files changed, 2 insertions(+), 6 del

[PATCH v6 01/19] migration/multifd: Reduce access to p->pages

2024-08-27 Thread Fabiano Rosas
I'm about to replace the p->pages pointer with an opaque pointer, so do a cleanup now to reduce direct accesses to p->page, which makes the next diffs cleaner. Reviewed-by: Peter Xu Signed-off-by: Fabiano Rosas --- migration/multifd-qpl.c | 8 +--- migration/multifd-uadk.c | 9 +

Re: [PATCH v1] softmmu/physmem: fix memory leak in dirty_memory_extend()

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 01:28:02PM -0400, Stefan Hajnoczi wrote: > On Tue, 27 Aug 2024 at 13:24, David Hildenbrand wrote: > > > > On 27.08.24 18:52, Stefan Hajnoczi wrote: > > > On Tue, 27 Aug 2024 at 04:38, David Hildenbrand wrote: > > >> > > >> As reported by Peter, we might be leaking memory w

Re: [PATCH] mark with for-crc32 in a consistent manner

2024-08-27 Thread Michael Tokarev
27.08.2024 15:09, Mark Cave-Ayland wrote: On 27/08/2024 11:02, Michael Tokarev wrote: in many cases, is only included for crc32 function, and in some of them, there's a comment saying that, but in a different way.  In one place (hw/net/rtl8139.c), there was another #include added between the c

[PATCH v2 04/17] thread-pool: Add a DestroyNotify parameter to thread_pool_submit{, _aio)()

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" Automatic memory management is less prone to mistakes or confusion who is responsible for freeing the memory backing the "arg" parameter or dropping a strong reference to object pointed by this parameter. Signed-off-by: Maciej S. Szmigiero --- backends/tpm/tpm_backe

[PATCH v2 09/17] migration/multifd: Device state transfer support - receive side

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" Add a basic support for receiving device state via multifd channels - channels that are shared with RAM transfers. To differentiate between a device state and a RAM packet the packet header is read first. Depending whether MULTIFD_FLAG_DEVICE_STATE flag is present or

[PATCH v2 10/17] migration/multifd: Convert multifd_send()::next_channel to atomic

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This is necessary for multifd_send() to be able to be called from multiple threads. Signed-off-by: Maciej S. Szmigiero --- migration/multifd.c | 24 ++-- 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/migration/multifd.c b/migrati

[PATCH v2 06/17] migration: Add save_live_complete_precopy_{begin, end} handlers

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" These SaveVMHandlers help device provide its own asynchronous transmission of the remaining data at the end of a precopy phase. In this use case the save_live_complete_precopy_begin handler might be used to mark the stream boundary before proceeding with asynchronous

[PATCH v2 14/17] migration: Add save_live_complete_precopy_thread handler

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This SaveVMHandler helps device provide its own asynchronous transmission of the remaining data at the end of a precopy phase via multifd channels, in parallel with the transfer done by save_live_complete_precopy handlers. These threads are launched only when multifd

[PATCH v2 13/17] migration/multifd: Add migration_has_device_state_support()

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" Since device state transfer via multifd channels requires multifd channels with packets and is currently not compatible with multifd compression add an appropriate query function so device can learn whether it can actually make use of it. Signed-off-by: Maciej S. Szmi

[PATCH v2 01/17] vfio/migration: Add save_{iterate, complete_precopy}_started trace events

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This way both the start and end points of migrating a particular VFIO device are known. Add also a vfio_save_iterate_empty_hit trace event so it is known when there's no more data to send for that device. Signed-off-by: Maciej S. Szmigiero --- hw/vfio/migration.c

[PATCH v2 15/17] vfio/migration: Multifd device state transfer support - receive side

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" The multifd received data needs to be reassembled since device state packets sent via different multifd channels can arrive out-of-order. Therefore, each VFIO device state packet carries a header indicating its position in the stream. The last such VFIO device state

[PATCH v2 07/17] migration: Add qemu_loadvm_load_state_buffer() and its handler

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" qemu_loadvm_load_state_buffer() and its load_state_buffer SaveVMHandler allow providing device state buffer to explicitly specified device via its idstr and instance id. Signed-off-by: Maciej S. Szmigiero --- include/migration/register.h | 15 +++ migrat

[PATCH v2 11/17] migration/multifd: Add an explicit MultiFDSendData destructor

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This way if there are fields there that needs explicit disposal (like, for example, some attached buffers) they will be handled appropriately. Add a related assert to multifd_set_payload_type() in order to make sure that this function is only used to fill a previously

[PATCH v2 17/17] vfio/migration: Multifd device state transfer support - send side

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" Implement the multifd device state transfer via additional per-device thread inside save_live_complete_precopy_thread handler. Switch between doing the data transfer in the new handler and doing it in the old save_state handler depending on the x-migration-multifd-tra

[PATCH v2 02/17] migration/ram: Add load start trace event

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" There's a RAM load complete trace event but there wasn't its start equivalent. Signed-off-by: Maciej S. Szmigiero --- migration/ram.c| 1 + migration/trace-events | 1 + 2 files changed, 2 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 6

Re: [PATCH v1] softmmu/physmem: fix memory leak in dirty_memory_extend()

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 10:37:15AM +0200, David Hildenbrand wrote: > /* Called with ram_list.mutex held */ > -static void dirty_memory_extend(ram_addr_t old_ram_size, > -ram_addr_t new_ram_size) > +static void dirty_memory_extend(ram_addr_t new_ram_size) > { > -

[PATCH v2 00/17] Multifd 🔀 device state transfer support with VFIO consumer

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This is an updated v2 patch series of the v1 series located here: https://lore.kernel.org/qemu-devel/cover.1718717584.git.maciej.szmigi...@oracle.com/ Changes from v1: * Extended the QEMU thread-pool with non-AIO (generic) pool support, implemented automatic memory ma

[PATCH v2 08/17] migration: Add load_finish handler and associated functions

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" load_finish SaveVMHandler allows migration code to poll whether a device-specific asynchronous device state loading operation had finished. In order to avoid calling this handler needlessly the device is supposed to notify the migration code of its possible readiness

[PATCH v2 03/17] migration/multifd: Zero p->flags before starting filling a packet

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This way there aren't stale flags there. p->flags can't contain SYNC to be sent at the next RAM packet since syncs are now handled separately in multifd_send_thread. Signed-off-by: Maciej S. Szmigiero --- migration/multifd.c | 2 +- 1 file changed, 1 insertion(+),

[PATCH v2 16/17] vfio/migration: Add x-migration-multifd-transfer VFIO property

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" This property allows configuring at runtime whether to send the particular device state via multifd channels when live migrating that device. It is ignored on the receive side and defaults to "false" for bit stream compatibility with older QEMU versions. Signed-off-b

[PATCH v2 12/17] migration/multifd: Device state transfer support - send side

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" A new function multifd_queue_device_state() is provided for device to queue its state for transmission via a multifd channel. Signed-off-by: Maciej S. Szmigiero --- include/migration/misc.h | 4 ++ migration/meson.build| 1 + migration/multifd-

Re: [PATCH v1] softmmu/physmem: fix memory leak in dirty_memory_extend()

2024-08-27 Thread David Hildenbrand
On 27.08.24 19:50, Peter Xu wrote: On Tue, Aug 27, 2024 at 01:28:02PM -0400, Stefan Hajnoczi wrote: On Tue, 27 Aug 2024 at 13:24, David Hildenbrand wrote: On 27.08.24 18:52, Stefan Hajnoczi wrote: On Tue, 27 Aug 2024 at 04:38, David Hildenbrand wrote: As reported by Peter, we might be lea

[PATCH v2 05/17] thread-pool: Implement non-AIO (generic) pool support

2024-08-27 Thread Maciej S. Szmigiero
From: "Maciej S. Szmigiero" Migration code wants to manage device data sending threads in one place. QEMU has an existing thread pool implementation, however it was limited to queuing AIO operations only and essentially had a 1:1 mapping between the current AioContext and the ThreadPool in use.

Re: [PATCH v1] softmmu/physmem: fix memory leak in dirty_memory_extend()

2024-08-27 Thread David Hildenbrand
On 27.08.24 19:57, Peter Xu wrote: On Tue, Aug 27, 2024 at 10:37:15AM +0200, David Hildenbrand wrote: /* Called with ram_list.mutex held */ -static void dirty_memory_extend(ram_addr_t old_ram_size, -ram_addr_t new_ram_size) +static void dirty_memory_extend(ram_a

Re: [PATCH v3 4/4] tests/tcg/aarch64: Extend MTE gdbstub tests to system mode

2024-08-27 Thread Gustavo Romero
Hi Phil, On 8/27/24 9:42 AM, Gustavo Romero wrote: Hi Phil! On 8/26/24 3:10 AM, Philippe Mathieu-Daudé wrote: Hi Gustavo, On 25/8/24 16:52, Gustavo Romero wrote: Extend MTE gdbstub tests to also run in system mode (share tests between user mode and system mode). The tests will only run if a

Re: [PATCH v6 18/19] migration/multifd: Stop changing the packet on recv side

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 02:46:05PM -0300, Fabiano Rosas wrote: > @@ -254,12 +250,10 @@ int multifd_ram_unfill_packet(MultiFDRecvParams *p, > Error **errp) > return 0; > } > > -/* make sure that ramblock is 0 terminated */ > -packet->ramblock[255] = 0; > -p->block = qemu

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 02:46:06PM -0300, Fabiano Rosas wrote: > Add documentation clarifying the usage of the multifd methods. The > general idea is that the client code calls into multifd to trigger > send/recv of data and multifd then calls these hooks back from the > worker threads at opportune

Re: [PATCH v1] softmmu/physmem: fix memory leak in dirty_memory_extend()

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 08:00:07PM +0200, David Hildenbrand wrote: > On 27.08.24 19:57, Peter Xu wrote: > > On Tue, Aug 27, 2024 at 10:37:15AM +0200, David Hildenbrand wrote: > > > /* Called with ram_list.mutex held */ > > > -static void dirty_memory_extend(ram_addr_t old_ram_size, > > > -

Re: [PATCH v6 18/19] migration/multifd: Stop changing the packet on recv side

2024-08-27 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Aug 27, 2024 at 02:46:05PM -0300, Fabiano Rosas wrote: >> @@ -254,12 +250,10 @@ int multifd_ram_unfill_packet(MultiFDRecvParams *p, >> Error **errp) >> return 0; >> } >> >> -/* make sure that ramblock is 0 terminated */ >> -packet->ramblock[255]

Re: [PATCH RESEND] hw/openrisc/openrisc_sim: keep serial@90000000 as default

2024-08-27 Thread Stafford Horne
On Sun, Aug 25, 2024 at 03:09:20PM +0100, Peter Maydell wrote: > On Sun, 25 Aug 2024 at 12:35, Jason A. Donenfeld wrote: > > > > On Fri, Aug 23, 2024 at 07:28:43AM +0100, Stafford Horne wrote: > > > Also, I will wait to see if Jason has anything to say. > > > > So long as this doesn't change the a

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Aug 27, 2024 at 02:46:06PM -0300, Fabiano Rosas wrote: >> Add documentation clarifying the usage of the multifd methods. The >> general idea is that the client code calls into multifd to trigger >> send/recv of data and multifd then calls these hooks back from the >> wo

Re: [PATCH v6 18/19] migration/multifd: Stop changing the packet on recv side

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 03:45:11PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Tue, Aug 27, 2024 at 02:46:05PM -0300, Fabiano Rosas wrote: > >> @@ -254,12 +250,10 @@ int multifd_ram_unfill_packet(MultiFDRecvParams *p, > >> Error **errp) > >> return 0; > >> } > >> > >>

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 03:54:51PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Tue, Aug 27, 2024 at 02:46:06PM -0300, Fabiano Rosas wrote: > >> Add documentation clarifying the usage of the multifd methods. The > >> general idea is that the client code calls into multifd to trigger >

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Aug 27, 2024 at 03:54:51PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Tue, Aug 27, 2024 at 02:46:06PM -0300, Fabiano Rosas wrote: >> >> Add documentation clarifying the usage of the multifd methods. The >> >> general idea is that the client code calls

Re: [PATCH v2 0/2] qdev-monitor: avoid QemuOpts in QMP device_add

2024-08-27 Thread Stefan Hajnoczi
On Tue, Aug 13, 2024 at 09:18:46AM +0100, Paul Durrant wrote: > On 12/08/2024 19:15, Stefan Hajnoczi wrote: > > On Fri, Aug 02, 2024 at 10:10:43AM +0200, Markus Armbruster wrote: > > > Can we additionally cut out the QemuOpts middleman in > > > usbback_portid_add()? > > > > > > qdict = qdict_

Re: [ANNOUNCE] QEMU 8.2.6 Stable released

2024-08-27 Thread Sergio Durigan Junior
On Wednesday, July 17 2024, Michael Tokarev wrote: > Hi everyone, Hey Michael, > The QEMU v8.2.6 stable release is now available. > > You can grab the tarball from our download page here: > > https://www.qemu.org/download/#source > > https://download.qemu.org/qemu-8.2.6.tar.xz > https://do

Re: [PATCH v6 18/19] migration/multifd: Stop changing the packet on recv side

2024-08-27 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Aug 27, 2024 at 03:45:11PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Tue, Aug 27, 2024 at 02:46:05PM -0300, Fabiano Rosas wrote: >> >> @@ -254,12 +250,10 @@ int multifd_ram_unfill_packet(MultiFDRecvParams *p, >> >> Error **errp) >> >> return

[PATCH v3 0/2] qdev-monitor: avoid QemuOpts in QMP device_add()

2024-08-27 Thread Stefan Hajnoczi
v3: - Duplicate drain_call_rcu() into hmp_device_add() because moving it into qdev_device_add_from_qdict turned out to be unsafe. v2: - Rename Patch 1 to indicate that we're avoiding QemuOpts rather than doing a full conversion to QAPI. Also mention that 'gen': false is still being used. [Mar

[PATCH v3 2/2] vl: use qmp_device_add() in qemu_create_cli_devices()

2024-08-27 Thread Stefan Hajnoczi
qemu_create_cli_devices() should use qmp_device_add() to match the behavior of the QMP monitor. A comment explained that libvirt changes implementing strict CLI syntax were needed. Peter Krempa has confirmed that modern libvirt uses the same JSON for -device (CLI) and device_add (QMP). Go ahead a

[PATCH v3 1/2] qdev-monitor: avoid QemuOpts in QMP device_add

2024-08-27 Thread Stefan Hajnoczi
The QMP device_add monitor command converts the QDict arguments to QemuOpts and then back again to QDict. This process only supports scalar types. Device properties like virtio-blk-pci's iothread-vq-mapping (an array of objects) are silently dropped by qemu_opts_from_qdict() during the QemuOpts con

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 04:17:59PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Tue, Aug 27, 2024 at 03:54:51PM -0300, Fabiano Rosas wrote: > >> Peter Xu writes: > >> > >> > On Tue, Aug 27, 2024 at 02:46:06PM -0300, Fabiano Rosas wrote: > >> >> Add documentation clarifying the usage

Re: [PATCH v6 18/19] migration/multifd: Stop changing the packet on recv side

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 04:27:42PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Tue, Aug 27, 2024 at 03:45:11PM -0300, Fabiano Rosas wrote: > >> Peter Xu writes: > >> > >> > On Tue, Aug 27, 2024 at 02:46:05PM -0300, Fabiano Rosas wrote: > >> >> @@ -254,12 +250,10 @@ int multifd_ram_u

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-08-27 Thread Peter Xu
On Tue, Jun 04, 2024 at 08:14:06PM +0800, Gonglei wrote: > From: Jialin Wang > > Hi, > > This patch series attempts to refactor RDMA live migration by > introducing a new QIOChannelRDMA class based on the rsocket API. > > The /usr/include/rdma/rsocket.h provides a higher level rsocket API > tha

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Fabiano Rosas
Peter Xu writes: > On Tue, Aug 27, 2024 at 04:17:59PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Tue, Aug 27, 2024 at 03:54:51PM -0300, Fabiano Rosas wrote: >> >> Peter Xu writes: >> >> >> >> > On Tue, Aug 27, 2024 at 02:46:06PM -0300, Fabiano Rosas wrote: >> >> >> Add documen

Re: [PULL 02/96] spapr: Migrate ail-mode-3 spapr cap

2024-08-27 Thread Michael Tokarev
26.07.2024 02:52, Nicholas Piggin пишет: This cap did not add the migration code when it was introduced. This results in migration failure when changing the default using the command line. Cc: qemu-sta...@nongnu.org Fixes: ccc5a4c5e10 ("spapr: Add SPAPR_CAP_AIL_MODE_3 for AIL mode 3 support for

Re: [PATCH v3] vhost-user: Do not wait for reply for not sent VHOST_USER_SET_LOG_BASE

2024-08-27 Thread Michael S. Tsirkin
On Tue, Aug 27, 2024 at 09:00:35PM +0800, BillXiang wrote: > > > From: "Prasad Pandit" > > Date:  Tue, Aug 27, 2024, 20:37 > > Subject:  Re: [PATCH v3] vhost-user: Do not wait for reply for not sent > > VHOST_USER_SET_LOG_BASE > > To: "BillXiang" > > Cc: "Michael S. Tsirkin", > > On Tue, 27 Aug

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-08-27 Thread Michael S. Tsirkin
On Tue, Aug 27, 2024 at 04:15:42PM -0400, Peter Xu wrote: > On Tue, Jun 04, 2024 at 08:14:06PM +0800, Gonglei wrote: > > From: Jialin Wang > > > > Hi, > > > > This patch series attempts to refactor RDMA live migration by > > introducing a new QIOChannelRDMA class based on the rsocket API. > > >

Re: [PULL for-9.1-rc4 0/1] NBD patches for 2024-08-26

2024-08-27 Thread Richard Henderson
On 8/26/24 23:49, Eric Blake wrote: The following changes since commit f259e4cb8a8b4ef5463326fc214a7d8d7703d5de: Merge tag 'pull-trivial-patches' of https://gitlab.com/mjt0k/qemu into staging (2024-08-24 08:09:27 +1000) are available in the Git repository at: https://repo.or.cz/qemu/eri

Re: [PATCH v6 19/19] migration/multifd: Add documentation for multifd methods

2024-08-27 Thread Peter Xu
On Tue, Aug 27, 2024 at 05:22:32PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Tue, Aug 27, 2024 at 04:17:59PM -0300, Fabiano Rosas wrote: > >> Peter Xu writes: > >> > >> > On Tue, Aug 27, 2024 at 03:54:51PM -0300, Fabiano Rosas wrote: > >> >> Peter Xu writes: > >> >> > >> >> > On

[PATCH v3 1/2] plugins: add plugin API to read guest memory

2024-08-27 Thread Rowan Hart
Signed-off-by: Rowan Hart Reviewed-by: Pierrick Bouvier --- include/qemu/qemu-plugin.h | 32 +++- plugins/api.c| 20 plugins/qemu-plugins.symbols | 1 + 3 files changed, 52 insertions(+), 1 deletion(-) diff --git a/include/qemu

[PATCH v3 0/2] plugins: add plugin API to read guest memory

2024-08-27 Thread Rowan Hart
This patch adds one API function to the QEMU plugin API bool qemu_plugin_read_memory_vaddr(vaddr, GByteArray *, size_t); The API allows reading memory from an arbitrary guest virtual address, which is useful for many things but the motivating examples are: * Virtual Machine Introspection (VMI) *

[PATCH v3 2/2] plugins: add option to dump write argument to syscall plugin

2024-08-27 Thread Rowan Hart
Signed-off-by: Rowan Hart Reviewed-by: Pierrick Bouvier Tested-by: Pierrick Bouvier --- docs/about/emulation.rst| 14 - tests/tcg/plugins/syscall.c | 117 2 files changed, 130 insertions(+), 1 deletion(-) diff --git a/docs/about/emulation.rst b/doc

Re: [PATCH v9 11/17] target/riscv: mmu changes for zicfiss shadow stack protection

2024-08-27 Thread Deepak Gupta
On Tue, Aug 27, 2024 at 03:59:08PM +1000, Alistair Francis wrote: On Tue, Aug 27, 2024 at 1:31 AM Deepak Gupta wrote: zicfiss protects shadow stack using new page table encodings PTE.W=0, PTE.R=0 and PTE.X=0. This encoding is reserved if zicfiss is not Shouldn't this be R=0, W=1, and X=0 ?

[PATCH v10 21/21] target/riscv: Expose zicfiss extension as a cpu property

2024-08-27 Thread Deepak Gupta
Signed-off-by: Deepak Gupta --- target/riscv/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index d6cdd81fd8..66dc5af20c 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1481,6 +1481,7 @@ const RISCVCPUMultiExtConfig riscv_cpu_extensi

[PATCH v10 02/21] linux-user/riscv: set priv for qemu-user and defaults for *envcfg

2024-08-27 Thread Deepak Gupta
set priv to be PRV_U for qemu-user on riscv. And set default value for *envcfg CSR. Signed-off-by: Deepak Gupta --- linux-user/riscv/cpu_loop.c | 4 1 file changed, 4 insertions(+) diff --git a/linux-user/riscv/cpu_loop.c b/linux-user/riscv/cpu_loop.c index 52c49c2e42..7a68e8717b 100644 --

[PATCH v10 03/21] target/riscv: Add zicfilp extension

2024-08-27 Thread Deepak Gupta
zicfilp [1] riscv cpu extension enables forward control flow integrity. If enabled, all indirect calls must land on a landing pad instruction. This patch sets up space for zicfilp extension in cpuconfig. zicfilp is dependend on zicsr. [1] - https://github.com/riscv/riscv-cfi Signed-off-by: Deepa

[PATCH v10 11/21] target/riscv: Add zicfiss extension

2024-08-27 Thread Deepak Gupta
zicfiss [1] riscv cpu extension enables backward control flow integrity. This patch sets up space for zicfiss extension in cpuconfig. And imple- ments dependency on A, zicsr, zimop and zcmop extensions. [1] - https://github.com/riscv/riscv-cfi Signed-off-by: Deepak Gupta Co-developed-by: Jim Sh

[PATCH v10 07/21] target/riscv: tracking indirect branches (fcfi) for zicfilp

2024-08-27 Thread Deepak Gupta
zicfilp protects forward control flow (if enabled) by enforcing all indirect call and jmp must land on a landing pad instruction `lpad`. If target of an indirect call or jmp is not `lpad` then cpu/hart must raise a sw check exception with tval = 2. This patch implements the mechanism using TCG. Ta

[PATCH v10 17/21] target/riscv: implement zicfiss instructions

2024-08-27 Thread Deepak Gupta
zicfiss has following instructions - sspopchk: pops a value from shadow stack and compares with x1/x5. If they dont match, reports a sw check exception with tval = 3. - sspush: pushes value in x1/x5 on shadow stack - ssrdp: reads current shadow stack - ssamoswap: swaps contents of shadow sta

[PATCH v10 20/21] disas/riscv: enable disassembly for compressed sspush/sspopchk

2024-08-27 Thread Deepak Gupta
sspush and sspopchk have equivalent compressed encoding taken from zcmop. cmop.1 is sspush x1 while cmop.5 is sspopchk x5. Due to unusual encoding for both rs1 and rs2 from space bitfield, this required a new codec. Signed-off-by: Deepak Gupta --- disas/riscv.c | 19 ++- disas/ri

[PATCH v10 13/21] target/riscv: tb flag for shadow stack instructions

2024-08-27 Thread Deepak Gupta
Shadow stack instructions can be decoded as zimop / zcmop or shadow stack instructions depending on whether shadow stack are enabled at current privilege. This requires a TB flag so that correct TB generation and correct TB lookup happens. `DisasContext` gets a field indicating whether bcfi is enab

[PATCH v10 01/21] target/riscv: expose *envcfg csr and priv to qemu-user as well

2024-08-27 Thread Deepak Gupta
Execution environment config CSR controlling user env and current privilege state shouldn't be limited to qemu-system only. *envcfg CSRs control enabling of features in next lesser mode. In some cases bits *envcfg CSR can be lit up by kernel as part of kernel policy or software (user app) can choos

[PATCH v10 00/21] riscv support for control flow integrity extensions

2024-08-27 Thread Deepak Gupta
v10 for riscv zicfilp and zicfiss extensions support in qemu. Links for previous versions [1] - v1 https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06017.html [2] - v2 https://lore.kernel.org/all/ed23bcbc-fdc4-4492-803c-daa958803...@linaro.org/T/ [3] - v3 https://lists.nongnu.org/archi

[PATCH v10 09/21] disas/riscv: enable `lpad` disassembly

2024-08-27 Thread Deepak Gupta
Signed-off-by: Deepak Gupta Co-developed-by: Jim Shu Co-developed-by: Andy Chiu Reviewed-by: Richard Henderson Reviewed-by: Alistair Francis --- disas/riscv.c | 18 +- disas/riscv.h | 2 ++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/disas/riscv.c b/disas/r

[PATCH v10 06/21] target/riscv: additional code information for sw check

2024-08-27 Thread Deepak Gupta
sw check exception support was recently added. This patch further augments sw check exception by providing support for additional code which is provided in *tval. Adds `sw_check_code` field in cpuarchstate. Whenever sw check exception is raised *tval gets the value deposited in `sw_check_code`. Si

[PATCH v10 05/21] target/riscv: save and restore elp state on priv transitions

2024-08-27 Thread Deepak Gupta
elp state is recorded in *status on trap entry (less privilege to higher privilege) and restored in elp from *status on trap exit (higher to less privilege). Additionally this patch introduces a forward cfi helper function to determine if current privilege has forward cfi is enabled or not based o

Re: [PATCH v10 00/21] riscv support for control flow integrity extensions

2024-08-27 Thread Deepak Gupta
On Tue, Aug 27, 2024 at 04:18:44PM -0700, Deepak Gupta wrote: v10 for riscv zicfilp and zicfiss extensions support in qemu. Links for previous versions [1] - v1 https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06017.html [2] - v2 https://lore.kernel.org/all/ed23bcbc-fdc4-4492-803c-da

[PATCH v10 04/21] target/riscv: Introduce elp state and enabling controls for zicfilp

2024-08-27 Thread Deepak Gupta
zicfilp introduces a new state elp ("expected landing pad") in cpu. During normal execution, elp is idle (NO_LP_EXPECTED) i.e not expecting landing pad. On an indirect call, elp moves LP_EXPECTED. When elp is LP_EXPECTED, only a subsquent landing pad instruction can set state back to NO_LP_EXPECTED

[PATCH v10 08/21] target/riscv: zicfilp `lpad` impl and branch tracking

2024-08-27 Thread Deepak Gupta
Implements setting lp expected when `jalr` is encountered and implements `lpad` instruction of zicfilp. `lpad` instruction is taken out of auipc x0, . This is an existing HINTNOP space. If `lpad` is target of an indirect branch, cpu checks for 20 bit value in x7 upper with 20 bit value embedded in

[PATCH v10 10/21] target/riscv: Expose zicfilp extension as a cpu property

2024-08-27 Thread Deepak Gupta
Signed-off-by: Deepak Gupta --- target/riscv/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 2d031e3e74..8e1f05e5b1 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1476,6 +1476,7 @@ const RISCVCPUMultiExtConfig riscv_cpu_extensi

[PATCH v10 19/21] disas/riscv: enable disassembly for zicfiss instructions

2024-08-27 Thread Deepak Gupta
Enable disassembly for sspush, sspopchk, ssrdp & ssamoswap. Disasembly is only enabled if zimop and zicfiss ext is set to true. Signed-off-by: Deepak Gupta --- disas/riscv.c | 40 +++- disas/riscv.h | 1 + 2 files changed, 40 insertions(+), 1 deletion(-) dif

[PATCH v10 15/21] target/riscv: AMO operations always raise store/AMO fault

2024-08-27 Thread Deepak Gupta
This patch adds one more word for tcg compile which can be obtained during unwind time to determine fault type for original operation (example AMO). Depending on that, fault can be promoted to store/AMO fault. Signed-off-by: Deepak Gupta Suggested-by: Richard Henderson Reviewed-by: Richard Hende

[PATCH v10 14/21] target/riscv: mmu changes for zicfiss shadow stack protection

2024-08-27 Thread Deepak Gupta
zicfiss protects shadow stack using new page table encodings PTE.W=1, PTE.R=0 and PTE.X=0. This encoding is reserved if zicfiss is not implemented or if shadow stack are not enabled. Loads on shadow stack memory are allowed while stores to shadow stack memory leads to access faults. Shadow stack ac

[PATCH v10 12/21] target/riscv: introduce ssp and enabling controls for zicfiss

2024-08-27 Thread Deepak Gupta
zicfiss introduces a new state ssp ("shadow stack register") in cpu. ssp is expressed as a new unprivileged csr (CSR_SSP=0x11) and holds virtual address for shadow stack as programmed by software. Shadow stack (for each mode) is enabled via bit3 in *envcfg CSRs. Shadow stack can be enabled for a m

[PATCH v10 16/21] target/riscv: update `decode_save_opc` to store extra word2

2024-08-27 Thread Deepak Gupta
Extra word 2 is stored during tcg compile and `decode_save_opc` needs additional argument in order to pass the value. This will be used during unwind to get extra information about instruction like how to massage exceptions. Updated all callsites as well. Signed-off-by: Deepak Gupta Reviewed-by:

[PATCH v10 18/21] target/riscv: compressed encodings for sspush and sspopchk

2024-08-27 Thread Deepak Gupta
sspush/sspopchk have compressed encodings carved out of zcmops. compressed sspush is designated as c.mop.1 while compressed sspopchk is designated as c.mop.5. Note that c.sspush x1 exists while c.sspush x5 doesn't. Similarly c.sspopchk x5 exists while c.sspopchk x1 doesn't. Signed-off-by: Deepak

Re: [PATCH v10 03/21] target/riscv: Add zicfilp extension

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 9:20 AM Deepak Gupta wrote: > > zicfilp [1] riscv cpu extension enables forward control flow integrity. > If enabled, all indirect calls must land on a landing pad instruction. > > This patch sets up space for zicfilp extension in cpuconfig. zicfilp > is dependend on zicsr.

Re: [PATCH v10 11/21] target/riscv: Add zicfiss extension

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 9:20 AM Deepak Gupta wrote: > > zicfiss [1] riscv cpu extension enables backward control flow integrity. > > This patch sets up space for zicfiss extension in cpuconfig. And imple- > ments dependency on A, zicsr, zimop and zcmop extensions. > > [1] - https://github.com/risc

Re: [PATCH v10 10/21] target/riscv: Expose zicfilp extension as a cpu property

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 9:23 AM Deepak Gupta wrote: > > Signed-off-by: Deepak Gupta Reviewed-by: Alistair Francis Alistair > --- > target/riscv/cpu.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c > index 2d031e3e74..8e1f05e5b1 100644 > --- a

Re: [PATCH v10 00/21] riscv support for control flow integrity extensions

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 9:21 AM Deepak Gupta wrote: > > v10 for riscv zicfilp and zicfiss extensions support in qemu. Please specify the exact version of the spec you used Alistair > > Links for previous versions > [1] - v1 > https://lists.nongnu.org/archive/html/qemu-devel/2024-07/msg06017.ht

Re: [PATCH v10 00/21] riscv support for control flow integrity extensions

2024-08-27 Thread Deepak Gupta
On Wed, Aug 28, 2024 at 10:02:42AM +1000, Alistair Francis wrote: On Wed, Aug 28, 2024 at 9:21 AM Deepak Gupta wrote: v10 for riscv zicfilp and zicfiss extensions support in qemu. Please specify the exact version of the spec you used https://github.com/riscv/riscv-cfi/releases/download/v1.

Re: [PATCH v10 01/21] target/riscv: expose *envcfg csr and priv to qemu-user as well

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 9:21 AM Deepak Gupta wrote: > > Execution environment config CSR controlling user env and current > privilege state shouldn't be limited to qemu-system only. *envcfg > CSRs control enabling of features in next lesser mode. In some cases > bits *envcfg CSR can be lit up by k

Re: [PATCH v10 02/21] linux-user/riscv: set priv for qemu-user and defaults for *envcfg

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 9:20 AM Deepak Gupta wrote: > > set priv to be PRV_U for qemu-user on riscv. And set default value for > *envcfg CSR. > > Signed-off-by: Deepak Gupta You can probably just squash this with the previous patch > --- > linux-user/riscv/cpu_loop.c | 4 > 1 file changed

Re: [PATCH v10 00/21] riscv support for control flow integrity extensions

2024-08-27 Thread Alistair Francis
On Wed, Aug 28, 2024 at 10:04 AM Deepak Gupta wrote: > > On Wed, Aug 28, 2024 at 10:02:42AM +1000, Alistair Francis wrote: > >On Wed, Aug 28, 2024 at 9:21 AM Deepak Gupta wrote: > >> > >> v10 for riscv zicfilp and zicfiss extensions support in qemu. > > > >Please specify the exact version of the

<    1   2   3   >