Revisiting parallel save/restore

2024-04-17 Thread Jim Fehlig via Devel
Hi All, While Fabiano has been working on improving save/restore performance in qemu, I've been tinkering with the same in libvirt. The end goal is to introduce a new VIR_DOMAIN_SAVE_PARALLEL flag for save/restore, along with a VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS parameter to specify th

Re: Revisiting parallel save/restore

2024-04-25 Thread Jim Fehlig via Devel
On 4/17/24 5:12 PM, Jim Fehlig wrote: Hi All, While Fabiano has been working on improving save/restore performance in qemu, I've been tinkering with the same in libvirt. The end goal is to introduce a new VIR_DOMAIN_SAVE_PARALLEL flag for save/restore, along with a VIR_DOMAIN_SAVE_PARAM_PARAL

Re: [PATCH v1 15/20] node_device_udev: Pass the driver state as parameter in preparation for the next commit

2024-04-25 Thread Jim Fehlig via Devel
On 4/23/24 3:41 AM, Marc Hartmayer wrote: On Tue, Apr 23, 2024 at 10:06 AM +0100, Daniel P. Berrangé wrote: On Tue, Apr 23, 2024 at 10:46:14AM +0200, Marc Hartmayer wrote: On Tue, Apr 23, 2024 at 09:10 AM +0100, Daniel P. Berrangé wrote: On Tue, Apr 23, 2024 at 10:03:35AM +0200, Marc Hartm

Re: Revisiting parallel save/restore

2024-04-26 Thread Jim Fehlig via Devel
On 4/26/24 4:04 AM, Daniel P. Berrangé wrote: On Wed, Apr 17, 2024 at 05:12:27PM -0600, Jim Fehlig via Devel wrote: A good starting point on this journey is supporting the new mapped-ram capability in qemu 9.0 [2]. Since mapped-ram is a new on-disk format, I assume we'll need

Re: Revisiting parallel save/restore

2024-04-26 Thread Jim Fehlig via Devel
On 4/26/24 4:07 AM, Daniel P. Berrangé wrote: On Thu, Apr 25, 2024 at 04:41:02PM -0600, Jim Fehlig via Devel wrote: On 4/17/24 5:12 PM, Jim Fehlig wrote: Hi All, While Fabiano has been working on improving save/restore performance in qemu, I've been tinkering with the same in libvirt. Th

[PATCH] libxl: Fix domxml-to-native conversion

2024-04-29 Thread Jim Fehlig via Devel
Similar to commit 57d084febe, another case of the libxl driver not adapting to modular daemons. When converting configuration that contains a type='network' interface, the converter calls virNetworkLookupByName, passing the hypervisor connection object instead of a connection to virtnetworkd. E.g.

Re: Revisiting parallel save/restore

2024-05-01 Thread Jim Fehlig via Devel
On 4/26/24 4:04 AM, Daniel P. Berrangé wrote: On Wed, Apr 17, 2024 at 05:12:27PM -0600, Jim Fehlig via Devel wrote: A good starting point on this journey is supporting the new mapped-ram capability in qemu 9.0 [2]. Since mapped-ram is a new on-disk format, I assume we'll need

Re: [PATCH] libxl: Fix domxml-to-native conversion

2024-05-02 Thread Jim Fehlig via Devel
On 5/2/24 5:24 AM, Ján Tomko wrote: On a Monday in 2024, Jim Fehlig via Devel wrote: Similar to commit 57d084febe, another case of the libxl driver not adapting to modular daemons. When converting configuration that contains a type='network' interface, the converter calls virNetworkLo

Add iommu device when VM configured with > 255 vcpus

2024-05-28 Thread Jim Fehlig via Devel
Hi All, I vaguely recall a discussion about $subject, but can't find it now. Perhaps buried in another thread. The topic has been raised internally again, and I'd like to gauge the community's interest in automatically adding the necessary devices/config when user has specified vcpus > 255.

Re: Add iommu device when VM configured with > 255 vcpus

2024-05-29 Thread Jim Fehlig via Devel
On 5/29/24 1:41 AM, Peter Krempa wrote: On Tue, May 28, 2024 at 16:26:18 -0600, Jim Fehlig via Devel wrote: Hi All, I vaguely recall a discussion about $subject, but can't find it now. Perhaps buried in another thread. The topic has been raised internally again, and I'd like to

Re: Add iommu device when VM configured with > 255 vcpus

2024-05-30 Thread Jim Fehlig via Devel
On 5/30/24 6:45 AM, Igor Mammedov wrote: On Wed, 29 May 2024 14:44:52 -0400 Sergio Durigan Junior wrote: On Tuesday, May 28 2024, Jim Fehlig via Devel wrote: Hi All, I vaguely recall a discussion about $subject, but can't find it now. Perhaps buried in another thread. The topic has

[PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-06-13 Thread Jim Fehlig via Devel
This series is a RFC for support of QEMU's mapped-ram migration capability [1] for saving and restoring VMs. It implements the first part of the design approach we discussed for supporting parallel save/restore [2]. In summary, the approach is 1. Add mapped-ram migration capability 2. Steal an ele

[PATCH RFC 1/9] qemu: Enable mapped-ram migration capability

2024-06-13 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 1 + src/qemu/qemu_migration_params.h | 1 + 2 files changed, 2 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index 48f8657f71..201286e58c 100644 --- a/src/qemu/qemu_migration_params

[PATCH RFC 2/9] qemu_fd: Add function to retrieve fdset ID

2024-06-13 Thread Jim Fehlig via Devel
Add new function qemuFDPassGetId(), to be used when adding support for mapped-ram save format. Signed-off-by: Jim Fehlig --- src/qemu/qemu_fd.c | 18 ++ src/qemu/qemu_fd.h | 3 +++ 2 files changed, 21 insertions(+) diff --git a/src/qemu/qemu_fd.c b/src/qemu/qemu_fd.c index e847

[PATCH RFC 3/9] qemu: Add function to get migration params for save

2024-06-13 Thread Jim Fehlig via Devel
Introduce qemuMigrationParamsForMappedSave() to create a qemuMigrationParams object initialized with appropriate migration capabilities and parameters for a save operation using mapped-ram. Note that mapped-ram also requires the multifd capability. For now, the number of multifd channels is set to

[PATCH RFC 4/9] qemu: Add a 'features' element to save image header and bump version

2024-06-13 Thread Jim Fehlig via Devel
QEMU's new mapped-ram stream format [1] is incompatible with the existing sequential format. In order to support the new format in libvirt, a new 'features' element is added to the saved image header. This element can be used now indicate the use of mapped-ram feature, and provides a mechanism to s

[PATCH RFC 5/9] qemu: conf: Add setting for save image version

2024-06-13 Thread Jim Fehlig via Devel
Add a 'save_image_version' setting to qemu.conf to control the image version when saving a VM with 'virsh save' or 'virsh managedsave'. Default to the new version 3. Signed-off-by: Jim Fehlig --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 6 ++ src/qemu/

[PATCH RFC 6/9] qemu: Add support for mapped-ram on save

2024-06-13 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 2 +- src/qemu/qemu_migration.c | 79 src/qemu/qemu_migration.h | 7 +++ src/qemu/qemu_monitor.c | 32 src/qemu/qemu_monitor.h | 4 ++ src/qemu/qemu_saveimage.c | 105 ++

[PATCH RFC 7/9] qemu: Enable mapped-ram on restore

2024-06-13 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration.c | 19 +++ src/qemu/qemu_migration.h | 3 ++- src/qemu/qemu_process.c | 49 --- src/qemu/qemu_process.h | 13 +++ src/qemu/qemu_saveimage.c | 26 ++--- 5 files chang

[PATCH RFC 8/9] qemu: Support O_DIRECT with mapped-ram on save

2024-06-13 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- I'm not happy with this and the subsequent patch, which pass another FD around for QEMU to use for reading/writing unaligned state when BYPASS_CACHE has been specified. One idea is to pass the qemuFdPass object around the various functions, but qemu_fd.h already incl

[PATCH RFC 9/9] qemu: Support O_DIRECT with mapped-ram on restore

2024-06-13 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 22 +- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_process.c | 20 src/qemu/qemu_process.h | 3 +++ src/qemu/qemu_saveimage.c | 29 + src/qemu/qemu_saveimage.h

Re: [PATCH] virt-aa-helper: Drop needless comments

2024-07-09 Thread Jim Fehlig via Devel
On 7/9/24 04:04, Michal Privoznik wrote: When generating paths for a domain specific AppArmor profile each path undergoes a validation where it's matched against an array of well known prefixes (among other things). Now, for OVMF/AAVMF/... images we have a list and some entries have comments to w

Re: [PATCH] apparmor: Allow more paths for qemu-bridge-helper

2024-08-06 Thread Jim Fehlig via Devel
On 8/5/24 08:25, Andrea Bolognani wrote: The QEMU package in Debian has recently moved the qemu-bridge-helper binary under /usr/libexec/qemu. Update the AppArmor profile accordingly. https://bugs.debian.org/1077915 Signed-off-by: Andrea Bolognani --- src/security/apparmor/usr.sbin.libvirtd.i

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-07 Thread Jim Fehlig via Devel
Hi Martin, On 8/7/24 06:32, Martin Kletzander wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: This series is a RFC for support of QEMU's mapped-ram migration capability [1] for saving and restoring VMs. It implements the first part of the design approa

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-07 Thread Jim Fehlig via Devel
On 8/7/24 09:45, Daniel P. Berrangé wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: This series is a RFC for support of QEMU's mapped-ram migration capability [1] for saving and restoring VMs. It implements the first part of the design approach we discusse

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-07 Thread Jim Fehlig via Devel
On 8/7/24 09:49, Daniel P. Berrangé wrote: On Wed, Aug 07, 2024 at 02:32:57PM +0200, Martin Kletzander wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: This series is a RFC for support of QEMU's mapped-ram migration capability [1] for saving and restoring VM

[PATCH 00/20] qemu: support mapped-ram+directio+mulitfd

2024-08-08 Thread Jim Fehlig via Devel
This series is essentially V1 of a prior RFC [1] to support QEMU's mapped-ram stream format [2] and migration capability. Along with supporting mapped-ram, it implements a design approach we discussed for supporting parallel save/restore [3]. In summary, the approach is 1. Add mapped-ram migration

[PATCH 02/20] qemu_fd: Add function to retrieve fdset ID

2024-08-08 Thread Jim Fehlig via Devel
Add new function qemuFDPassGetId() for retrieving the fdset ID of provided qemuFDPass object. Signed-off-by: Jim Fehlig --- src/qemu/qemu_fd.c | 18 ++ src/qemu/qemu_fd.h | 3 +++ 2 files changed, 21 insertions(+) diff --git a/src/qemu/qemu_fd.c b/src/qemu/qemu_fd.c index e8470

[PATCH 01/20] lib: virDomainSaveParams: Ensure absolute save path

2024-08-08 Thread Jim Fehlig via Devel
When invoking virDomainSaveParams with a relative path, the image is saved to the daemon's CWD, which in most cases is '/'. Ensure a relative path is converted to absolute before invoking the driver 'domainSaveParams' function. Signed-off-by: Jim Fehlig --- src/libvirt-domain.c | 46

[PATCH 03/20] qemu: Add function to check capability in migration params

2024-08-08 Thread Jim Fehlig via Devel
Add new function qemuMigrationParamsCapEnabled() to check if a capability is set in the caller-provided migration parameters. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 16 src/qemu/qemu_migration_params.h | 4 2 files changed, 20 insertions(+) diff

[PATCH 04/20] qemu: Add function to get bool value from migration params

2024-08-08 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 18 ++ src/qemu/qemu_migration_params.h | 5 + 2 files changed, 23 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index c3c9120c22..daa52269f4 100644 --- a/src/qe

[PATCH 05/20] qemu: Add mapped-ram migration capability

2024-08-08 Thread Jim Fehlig via Devel
Add the mapped-ram migration capability introduced in QEMU 9.0. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 1 + src/qemu/qemu_migration_params.h | 1 + 2 files changed, 2 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qemu/qemu_migration_params.c index d

[PATCH 06/20] qemu: Add function to get migration params for save

2024-08-08 Thread Jim Fehlig via Devel
Introduce qemuMigrationParamsForSave() to create a qemuMigrationParams object initialized with appropriate migration capabilities and parameters for a save operation. Note that mapped-ram capability also requires the multifd capability. For now, the number of multifd channels is set to 1. Future w

[PATCH 07/20] qemu: QEMU_SAVE_VERSION: Bump to version 3

2024-08-08 Thread Jim Fehlig via Devel
QEMU's new mapped-ram stream format [1] is incompatible with the existing sequential stream format. An older libvirt+QEMU that does not support mapped-ram must not attempt to restore a mapped-ram saved image. Currently the only way to achieve this is to bump QEMU_SAVE_VERSION. To avoid future vers

[PATCH 08/20] qemu: conf: Add setting for save image version

2024-08-08 Thread Jim Fehlig via Devel
Add a 'save_image_version' setting to qemu.conf to control the image version when saving a VM with 'virsh save' or 'virsh managedsave'. Default to the new version 3. Signed-off-by: Jim Fehlig --- src/qemu/libvirtd_qemu.aug | 1 + src/qemu/qemu.conf.in | 6 ++ src/qemu/

[PATCH 09/20] qemu: Add helper function for creating save image fd

2024-08-08 Thread Jim Fehlig via Devel
Move the code in qemuSaveImageCreate that opens, labels, and wraps the save image fd to a helper function, providing more flexibility for upcoming mapped-ram support. Signed-off-by: Jim Fehlig --- src/qemu/qemu_saveimage.c | 65 +++ 1 file changed, 45 insertio

[PATCH 10/20] qemu: Add support for mapped-ram on save

2024-08-08 Thread Jim Fehlig via Devel
Introduce support for QEMU's new mapped-ram stream format [1]. mapped-ram is enabled by default if the underlying QEMU advertises the mapped-ram migration capability. It can be disabled by changing the 'save_image_version' setting in qemu.conf to version '2'. To use mapped-ram with QEMU: - The 'ma

[PATCH 11/20] qemu: Decompose qemuSaveImageOpen

2024-08-08 Thread Jim Fehlig via Devel
Split the reading of libvirt's save image metadata from the opening of the fd that will be passed to QEMU. This provides flexibility for an upcoming patch adding mapped-ram support for restore. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 37 src/qemu/qemu_saveimage

[PATCH 13/20] qemu: Apply migration parameters in qemuMigrationDstRun

2024-08-08 Thread Jim Fehlig via Devel
Similar to qemuMigrationSrcRun, apply migration parameters in qemuMigrationDstRun. This allows callers to create customized migration parameters, but delegates their application to the function performing the migration. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration.c | 15 +--

[PATCH 12/20] qemu: Move creation of qemuProcessIncomingDef struct

2024-08-08 Thread Jim Fehlig via Devel
qemuProcessStartWithMemoryState() is the only caller of qemuProcessStart() that uses the qemuProcessIncomingDef struct. Move creation of the struct to qemuProcessStartWithMemoryState(). Signed-off-by: Jim Fehlig --- src/qemu/qemu_process.c | 44 - src/qemu

[PATCH 14/20] qemu: Add support for mapped-ram on restore

2024-08-08 Thread Jim Fehlig via Devel
Add support for the mapped-ram migration capability on restore. Using mapped-ram with QEMU to restore an image requires the same steps as saving: - The 'mapped-ram' migration capability must be set to true - The 'multifd' migration capability must be set to true and the 'multifd-channels' migrat

[PATCH 15/20] qemu: Support O_DIRECT with mapped-ram on save

2024-08-08 Thread Jim Fehlig via Devel
When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 9 + src/qemu/qemu_migration.c| 32

[PATCH 16/20] qemu: Support O_DIRECT with mapped-ram on restore

2024-08-08 Thread Jim Fehlig via Devel
When using the mapped-ram migration capability, direct IO is enabled by setting the "direct-io" migration parameter to "true" and passing QEMU an additional fd with O_DIRECT set. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration.c | 11 ++- src/qemu/qemu_process.c | 30

[PATCH 17/20] include: Define constants for parallel save/restore

2024-08-08 Thread Jim Fehlig via Devel
From: Claudio Fontana Add a new VIR_DOMAIN_SAVE_PARALLEL flag to the save and restore APIs, which can be used to specify the use of multiple, parallel channels for saving a domain. The number of parallel channels can be set using the VIR_DOMAIN_SAVE_PARAM_PARALLEL_CONNECTIONS typed parameter. Si

[PATCH 18/20] qemu: Add support for parallel save and restore

2024-08-08 Thread Jim Fehlig via Devel
Add support for parallel save and restore by mapping libvirt's "parallel-connections" parameter to QEMU's "multifd-channels" migration parameter. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 31 --- src/qemu/qemu_migration_params.c | 25 +++

[PATCH 19/20] tools: add parallel parameter to virsh save command

2024-08-08 Thread Jim Fehlig via Devel
From: Li Zhang Signed-off-by: Claudio Fontana Signed-off-by: Jim Fehlig --- tools/virsh-domain.c | 41 + 1 file changed, 37 insertions(+), 4 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 50e80689a2..ec0e43ae7b 100644 --- a/

[PATCH 20/20] tools: add parallel parameter to virsh restore command

2024-08-08 Thread Jim Fehlig via Devel
From: Claudio Fontana Signed-off-by: Claudio Fontana Signed-off-by: Jim Fehlig --- docs/manpages/virsh.rst | 9 +++-- tools/virsh-domain.c| 38 ++ 2 files changed, 41 insertions(+), 6 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpage

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-08 Thread Jim Fehlig via Devel
On 8/7/24 12:39, Daniel P. Berrangé wrote: On Wed, Aug 07, 2024 at 12:04:18PM -0600, Jim Fehlig wrote: On 8/7/24 09:45, Daniel P. Berrangé wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: The QEMU mapped-ram capability currently does not support directio. Fabino is

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-08 Thread Jim Fehlig via Devel
On 8/8/24 17:46, Jim Fehlig wrote: On 8/7/24 12:39, Daniel P. Berrangé wrote: On Wed, Aug 07, 2024 at 12:04:18PM -0600, Jim Fehlig wrote: On 8/7/24 09:45, Daniel P. Berrangé wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: The QEMU mapped-ram capability currently

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-12 Thread Jim Fehlig via Devel
On 8/7/24 09:45, Daniel P. Berrangé wrote: On Thu, Jun 13, 2024 at 04:43:14PM -0600, Jim Fehlig via Devel wrote: This series is a RFC for support of QEMU's mapped-ram migration capability [1] for saving and restoring VMs. It implements the first part of the design approach we discusse

[PATCH 0/2] qemu: Clarify purpose of image format settings

2024-08-16 Thread Jim Fehlig via Devel
The current documentation of the various foo_image_format settings in qemu.conf subtly implies they are only used for specifying compression. Patch1 of this small series attempts to clarify and improve the description of the settings. It defines image format as a way to specify the desired layout o

[PATCH 1/2] qemu: conf: Improve the foo_image_format setting descriptions

2024-08-16 Thread Jim Fehlig via Devel
The current description of the various foo_image_format settings can be construded to imply the setting is only used to control compression of the image. Improve the documentation to clarify that format describes the representation of guest memory blocks on disk, which includes compression among ot

[PATCH 2/2] qemu: Rename compressed field of save image header

2024-08-16 Thread Jim Fehlig via Devel
The corresponding qemu.conf setting is named save_image_format and the enum of supported format types is declared with name virQEMUSaveFormat. Let's be consistent and use 'format' instead of 'compressed' as a field name in the virQEMUSaveHeader struct. Signed-off-by: Jim Fehlig --- src/qemu/qemu

Re: [PATCH RFC 0/9] qemu: Support mapped-ram migration capability

2024-08-16 Thread Jim Fehlig via Devel
On 8/12/24 17:16, Jim Fehlig wrote: On 8/7/24 09:45, Daniel P. Berrangé wrote: Annoyingly we already have a 'save_image_formt' in qemu.conf though taking  'raw', 'zstd', 'lzop', etc to choose the compression type. So we have a terminology clash. Thinking about this more, and your previous idea

[PATCH V2 0/2] qemu: Clarify purpose of image format settings

2024-08-16 Thread Jim Fehlig via Devel
The current documentation of the various foo_image_format settings in qemu.conf subtly implies they are only used for specifying compression. Patch1 of this small series attempts to clarify and improve the description of the settings. It defines image format as a way to specify the desired layout o

[PATCH V2 1/2] qemu: conf: Improve the foo_image_format setting descriptions

2024-08-16 Thread Jim Fehlig via Devel
The current description of the various foo_image_format settings can be construded to imply the setting is only used to control compression of the image. Improve the documentation to clarify that format describes the representation of guest memory blocks on disk, which includes compression among ot

[PATCH V2 2/2] qemu: Use consistent naming for save image format

2024-08-16 Thread Jim Fehlig via Devel
The image format setting in qemu.conf is named 'save_image_format'. The enum of supported format types is declared with name 'virQEMUSaveFormat'. Let's be consistent and use 'format' instead of 'compressed' when referring to the save image format. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driv

Re: [PATCH 0/2] qemu: Clarify purpose of image format settings

2024-08-16 Thread Jim Fehlig via Devel
On 8/16/24 11:20, Jim Fehlig wrote: The current documentation of the various foo_image_format settings in qemu.conf subtly implies they are only used for specifying compression. Patch1 of this small series attempts to clarify and improve the description of the settings. It defines image format as

Discouraging use of virInterface* APIs

2024-08-21 Thread Jim Fehlig via Devel
Laine's attempt long ago [1] to deprecate/obsolete the virInterface* APIs did not receive a standing ovation. However he raised many good points which are still valid today. If anything, netcf, the libvirt netcf backend, and the whole interface driver have become more stale. Personally, I wish w

Re: [PATCH V2 0/2] qemu: Clarify purpose of image format settings

2024-08-30 Thread Jim Fehlig via Devel
Hi All, Any comments on this idea? Recall the motivation for this change is to subsequently use the image format settings to request mapped-ram. Regards, Jim On 8/16/24 16:25, Jim Fehlig wrote: The current documentation of the various foo_image_format settings in qemu.conf subtly implies the

[PATCH 0/2] Reject Xen VM config containing nwfilter references

2024-09-11 Thread Jim Fehlig via Devel
This is essentially V2 of a small series inspired by a report on the security list about nwfilters not working with Xen VMs. V1 was posted to the security list, so no public reference. The libxl driver simply does not support nwfilters, so the report is really a RFE vs a security issue. I'm now mo

[PATCH 1/2] docs: Clarify hypervisor support for nwfilter profiles

2024-09-11 Thread Jim Fehlig via Devel
Enhance the 'since' annotation of documentation to note it's only supported by the QEMU, LXC, and ch hypervisor drivers. Signed-off-by: Jim Fehlig --- docs/formatdomain.rst | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst

[PATCH 2/2] libxl: Reject VM config referencing nwfilters

2024-09-11 Thread Jim Fehlig via Devel
The Xen libxl driver does not support nwfilter. Add a check for nwfilters to the devicesPostParseCallback, returning VIR_ERR_CONFIG_UNSUPPORTED if any are found. It's generally preferred for drivers to ignore unsupported XML features, but ignoring a user's request to filter VM network traffic can

Re: [PATCH 0/2] Reject Xen VM config containing nwfilter references

2024-09-11 Thread Jim Fehlig via Devel
On 9/11/24 15:49, Demi Marie Obenour wrote: On Wed, Sep 11, 2024 at 03:02:40PM -0600, Jim Fehlig wrote: This is essentially V2 of a small series inspired by a report on the security list about nwfilters not working with Xen VMs. V1 was posted to the security list, so no public reference. The lib

Re: [PATCH 1/2] docs: Clarify hypervisor support for nwfilter profiles

2024-09-11 Thread Jim Fehlig via Devel
On 9/11/24 15:54, Demi Marie Obenour wrote: On Wed, Sep 11, 2024 at 03:02:41PM -0600, Jim Fehlig wrote: Enhance the 'since' annotation of documentation to note it's only supported by the QEMU, LXC, and ch hypervisor drivers. Signed-off-by: Jim Fehlig --- docs/formatdomain.rst | 8

Re: [PATCH 1/2] docs: Clarify hypervisor support for nwfilter profiles

2024-09-11 Thread Jim Fehlig via Devel
On 9/11/24 16:08, Laine Stump wrote: On 9/11/24 5:02 PM, Jim Fehlig via Devel wrote: Enhance the 'since' annotation of documentation to note it's only supported by the QEMU, LXC, and ch hypervisor drivers. Signed-off-by: Jim Fehlig ---   docs/formatdomain.rst | 8   1

Re: [PATCH 2/2] libxl: Reject VM config referencing nwfilters

2024-09-11 Thread Jim Fehlig via Devel
On 9/11/24 16:24, Laine Stump wrote: On 9/11/24 5:02 PM, Jim Fehlig via Devel wrote: The Xen libxl driver does not support nwfilter. Add a check for nwfilters to the devicesPostParseCallback, returning VIR_ERR_CONFIG_UNSUPPORTED if any are found. It's generally preferred for drivers to i

Re: [PATCH 1/2] docs: Clarify hypervisor support for nwfilter profiles

2024-09-12 Thread Jim Fehlig via Devel
On 9/11/24 16:47, Jim Fehlig wrote: On 9/11/24 16:08, Laine Stump wrote: On 9/11/24 5:02 PM, Jim Fehlig via Devel wrote: Enhance the 'since' annotation of documentation to note it's only supported by the QEMU, LXC, and ch hypervisor drivers. Signed-off-by: Jim Feh

Re: [PATCH 2/2] libxl: Reject VM config referencing nwfilters

2024-09-12 Thread Jim Fehlig via Devel
On 9/12/24 01:37, Peter Krempa wrote: On Wed, Sep 11, 2024 at 18:24:07 -0400, Laine Stump wrote: On 9/11/24 5:02 PM, Jim Fehlig via Devel wrote: The Xen libxl driver does not support nwfilter. Add a check for nwfilters to the devicesPostParseCallback, returning VIR_ERR_CONFIG_UNSUPPORTED if

[PATCH V2] libxl: Reject VM config referencing nwfilters

2024-09-12 Thread Jim Fehlig via Devel
The Xen libxl driver does not support nwfilter. Introduce a deviceValidateCallback function with a check for nwfilters, returning VIR_ERR_CONFIG_UNSUPPORTED if any are found. Also fail to start any existing VMs referencing nwfilters. Drivers generally ignore unrecognized XML configuration, but ign

[PATCH V3] libxl: Reject VM config referencing nwfilters

2024-10-08 Thread Jim Fehlig via Devel
The Xen libxl driver does not support nwfilter. Introduce a deviceValidateCallback function with a check for nwfilters, returning VIR_ERR_CONFIG_UNSUPPORTED if any are found. Also fail to start any existing VMs referencing nwfilters. Drivers generally ignore unrecognized XML configuration, but ign

Re: [PATCH V2 1/2] qemu: conf: Improve the foo_image_format setting descriptions

2024-10-08 Thread Jim Fehlig via Devel
On 10/2/24 02:37, Martin Kletzander wrote: On Fri, Aug 16, 2024 at 04:25:25PM -0600, Jim Fehlig via Devel wrote: The current description of the various foo_image_format settings can be construded to imply the setting is only used to control compression of the image. Improve the documentation to

Re: [PATCH 17/20] include: Define constants for parallel save/restore

2024-10-14 Thread Jim Fehlig via Devel
On 10/14/24 11:42, Daniel P. Berrangé wrote: On Mon, Oct 14, 2024 at 06:00:53PM +0200, Claudio Fontana wrote: On 10/10/24 15:29, Daniel P. Berrangé wrote: On Thu, Aug 08, 2024 at 05:38:10PM -0600, Jim Fehlig via Devel wrote: From: Claudio Fontana Add a new VIR_DOMAIN_SAVE_PARALLEL flag to

[PATCH] NEWS: Mention documention improvements of image format settings

2024-10-09 Thread Jim Fehlig via Devel
Signed-off-by: Jim Fehlig --- Although small, the change seems NEWS noteworthy. NEWS.rst | 8 1 file changed, 8 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index 7f4f33c8f8..dd9f261933 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -35,6 +35,14 @@ v10.9.0 (unreleased) Users are encou

[PATCH] spec: Drop nwfilter dependency in libvirt-daemon-xen

2024-10-21 Thread Jim Fehlig via Devel
The libvirt xen driver does not support nwfilters. In fact, since commit d721b6840f, the driver rejects VM configuration referencing nwfilters. Drop the needless nwfilter dependency from libvirt-daemon-xen. Signed-off-by: Jim Fehlig --- libvirt.spec.in | 1 - 1 file changed, 1 deletion(-) diff

Re: [PATCH 07/20] qemu: QEMU_SAVE_VERSION: Bump to version 3

2024-10-16 Thread Jim Fehlig via Devel
, Daniel P. Berrangé wrote: On Thu, Aug 08, 2024 at 05:38:00PM -0600, Jim Fehlig via Devel wrote: QEMU's new mapped-ram stream format [1] is incompatible with the existing sequential stream format. An older libvirt+QEMU that does not support mapped-ram must not attempt to restore a mapped-

Re: [PATCH 18/20] qemu: Add support for parallel save and restore

2024-10-16 Thread Jim Fehlig via Devel
On 10/10/24 07:43, Daniel P. Berrangé wrote: On Thu, Aug 08, 2024 at 05:38:11PM -0600, Jim Fehlig via Devel wrote: Add support for parallel save and restore by mapping libvirt's "parallel-connections" parameter to QEMU's "multifd-channels" migration paramet

Re: [PATCH 07/20] qemu: QEMU_SAVE_VERSION: Bump to version 3

2024-10-10 Thread Jim Fehlig via Devel
On 10/10/24 07:06, Daniel P. Berrangé wrote: On Thu, Aug 08, 2024 at 05:38:00PM -0600, Jim Fehlig via Devel wrote: QEMU's new mapped-ram stream format [1] is incompatible with the existing sequential stream format. An older libvirt+QEMU that does not support mapped-ram must not attem

Re: [PATCH] apparmor: Allow running i686 VMs on Debian 12

2024-10-15 Thread Jim Fehlig via Devel
On 10/15/24 03:57, Andrea Bolognani wrote: In Debian 12, the qemu-system-i386 binary in /usr/bin is a wrapper script, with the actual executable living in /usr/libexec instead. This makes it impossible to run i686 VMs when AppArmor is enabled. Allow running the actual binary. https://bugs.debia

Re: [PATCH v2 1/4] security_apparmor: fix memleaks in AppArmorSetFDLabel

2025-01-06 Thread Jim Fehlig via Devel
On 11/13/24 07:28, Georgia Garcia wrote: proc and fd_path are allocated but never freed. Fix by using g_autofree instead. Fixes: b9757fea30785a92aa95ea675b9bc371e4fb2e8c Signed-off-by: Georgia Garcia --- src/security/security_apparmor.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(

Re: [PATCH] qemu: Add audit entries for suspend and resume

2025-01-06 Thread Jim Fehlig via Devel
Happy new year everyone! Any comments on this small patch? Regards, Jim On 12/16/24 16:56, Jim Fehlig wrote: We recently received a request from certification auditors to provide audit entries for suspend and resume. This small patch uses the existing virtDomainAudit{Start,Stop} functions with

[PATCH] security: apparmor: Remove hardcoded "libvirtd" profile name

2025-01-06 Thread Jim Fehlig via Devel
The apparmor driver probe function checks for an active profile matching the full path of the running daemon binary. If not found, it checks for a profile named "libvirtd". This works fine when the running daemon is the old monolithic libvirtd, but fails with modular daemons. Remove the check for

Re: [PATCH v2 3/4] apparmor: fix UUID specification

2025-01-06 Thread Jim Fehlig via Devel
On 11/13/24 07:28, Georgia Garcia wrote: There is a common misconception when writing AppArmor policy that [0-9]* applies * to the [0-9] class, but that's not the case. For this example, [0-9]* matches a single digit followed by any number of characters except for / Create a UUID variable that u

Re: [PATCH v2 2/4] security: replace uses of label and VIR_FREE by g_autofree

2025-01-06 Thread Jim Fehlig via Devel
On 11/13/24 07:28, Georgia Garcia wrote: Moving towards full adoption of GLib APIs in the AppArmor code. Signed-off-by: Georgia Garcia --- src/security/security_apparmor.c | 41 - src/security/virt-aa-helper.c| 100 ++- 2 files changed, 45 insert

Re: [PATCH] qemu: Add audit entries for suspend and resume

2025-01-07 Thread Jim Fehlig via Devel
On 1/7/25 04:22, Daniel P. Berrangé wrote: On Tue, Jan 07, 2025 at 12:06:59PM +0100, Michal Prívozník wrote: On 12/17/24 00:56, Jim Fehlig via Devel wrote: We recently received a request from certification auditors to provide audit entries for suspend and resume. This small patch uses the

Re: [PATCH v3 3/4] apparmor: fix UUID specification

2025-01-07 Thread Jim Fehlig via Devel
On 1/7/25 08:23, Georgia Garcia wrote: There is a common misconception when writing AppArmor policy that [0-9]* applies * to the [0-9] class, but that's not the case. For this example, [0-9]* matches a single digit followed by any number of characters except for / Create a UUID variable that use

[PATCH V2 1/3] qemu: Move unlinking corrupt save image file to caller

2025-01-30 Thread Jim Fehlig via Devel
qemuDomainObjRestore is the only caller of qemuSaveImageOpen that requests an unlink of a corrupted save image. Provide a function to check for a corrupt image and move unlinking it to qemuDomainObjRestore. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 23 ++- src/qemu/qe

[PATCH V2 0/3] qemu: Improve opening and verifying save images

2025-01-30 Thread Jim Fehlig via Devel
V2 of https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/DATZFZY6ETRYOQ6ORQ2JIVHBTFGKUBJM/ Changes in V2: * Move unlinking corrupt save images from qemuSaveImageOpen to the only caller using that functionality * Add a function to read save image header * Correctly position

[PATCH V2 2/3] qemu: Decompose qemuSaveImageOpen

2025-01-30 Thread Jim Fehlig via Devel
Split the reading of libvirt's save image metadata from the opening of the fd that will be passed to QEMU. This allows improved error handling and provides more flexibility users of qemu_saveimage. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 31 +++--- src/qemu/qemu_saveimage.c |

Re: [PATCH 1/2] qemu: Decompose qemuSaveImageOpen

2025-01-30 Thread Jim Fehlig via Devel
On 1/30/25 02:09, Michal Prívozník wrote: On 1/29/25 18:35, Jim Fehlig wrote: I eventually need something like 1/2 for the mapped-ram support. Currently qemuSaveImageOpen does too much IMO. I've been working on an improvement to this series and would like to get opinions on that first. And befo

[PATCH V2 3/3] qemu: Check for valid save image format when verifying image header

2025-01-30 Thread Jim Fehlig via Devel
When attempting to restore a saved image, the check for a valid save image format does not occur until the qemu process is about to be executed. Move the check earlier in the restore process, along with the other checks that verify a valid save image header. Signed-off-by: Jim Fehlig --- src/qem

[PATCH V3 06/19] qemu: Add function to get migration params for save

2025-02-07 Thread Jim Fehlig via Devel
Introduce qemuMigrationParamsForSave() to create a qemuMigrationParams object initialized with appropriate migration capabilities and parameters for a save operation. Note that mapped-ram capability also requires the multifd capability. For now, the number of multifd channels is set to 1. Future w

[PATCH V3 05/19] qemu: Add mapped-ram migration capability

2025-02-07 Thread Jim Fehlig via Devel
Add the mapped-ram migration capability introduced in QEMU 9.0. Reviewed-by: Daniel P. Berrangé Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration_params.c | 1 + src/qemu/qemu_migration_params.h | 1 + 2 files changed, 2 insertions(+) diff --git a/src/qemu/qemu_migration_params.c b/src/qem

[PATCH V3 18/19] tools: add parallel parameter to virsh save command

2025-02-07 Thread Jim Fehlig via Devel
From: Claudio Fontana Signed-off-by: Claudio Fontana Signed-off-by: Jim Fehlig --- docs/manpages/virsh.rst | 12 +++- tools/virsh-domain.c| 42 + 2 files changed, 49 insertions(+), 5 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/m

[PATCH V3 17/19] qemu: Add support for parallel save and restore

2025-02-07 Thread Jim Fehlig via Devel
Add support for parallel save and restore by mapping libvirt's "parallel-channels" parameter to QEMU's "multifd-channels" migration parameter. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c | 32 +--- src/qemu/qemu_migration_params.c | 31 +

[PATCH V3 19/19] tools: add parallel parameter to virsh restore command

2025-02-07 Thread Jim Fehlig via Devel
From: Claudio Fontana Signed-off-by: Claudio Fontana Signed-off-by: Jim Fehlig --- docs/manpages/virsh.rst | 9 +++-- tools/virsh-domain.c| 39 +++ 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/docs/manpages/virsh.rst b/docs/manpag

[PATCH V3 10/19] qemu: Add support for mapped-ram on save

2025-02-07 Thread Jim Fehlig via Devel
Introduce support for QEMU's new mapped-ram stream format [1]. mapped-ram can be enabled by setting the 'save_image_format' setting in qemu.conf to 'sparse'. To use mapped-ram with QEMU: - The 'mapped-ram' migration capability must be set to true - The 'multifd' migration capability must be set to

[PATCH V3 09/19] qemu: Move declaration of virQEMUSaveFormat to header file

2025-02-07 Thread Jim Fehlig via Devel
Allow use of the enum outside of qemu_saveimage. Signed-off-by: Jim Fehlig --- src/qemu/qemu_saveimage.c | 20 src/qemu/qemu_saveimage.h | 20 2 files changed, 20 insertions(+), 20 deletions(-) diff --git a/src/qemu/qemu_saveimage.c b/src/qemu/qemu_save

[PATCH V3 12/19] qemu: Apply migration parameters in qemuMigrationDstRun

2025-02-07 Thread Jim Fehlig via Devel
Similar to qemuMigrationSrcRun, apply migration parameters in qemuMigrationDstRun. This allows callers to create customized migration parameters, but delegates their application to the function performing the migration. Signed-off-by: Jim Fehlig --- src/qemu/qemu_migration.c | 16 ++-

[PATCH V3 13/19] qemu: Add support for mapped-ram on restore

2025-02-07 Thread Jim Fehlig via Devel
Add support for the mapped-ram migration capability on restore. Signed-off-by: Jim Fehlig --- src/qemu/qemu_driver.c| 27 +++--- src/qemu/qemu_migration.c | 12 ++-- src/qemu/qemu_process.c | 41 --- src/qemu/qemu_process.h

  1   2   3   >