Re: [PATCH v2 03/24] block/snapshot: move drain outside of read-locked bdrv_snapshot_delete()

2025-05-23 Thread Andrey Drobyshev
On 5/20/25 1:29 PM, Fiona Ebner wrote: > This is in preparation to mark bdrv_drained_begin() as GRAPH_UNLOCKED. > > More granular draining is not trivially possible, because > bdrv_snapshot_delete() can recursively call itself. > > The return value of bdrv_all_delete_snapshot() changes from -1 to

Re: [PATCH v2 07/24] block: mark bdrv_child_change_aio_context() GRAPH_RDLOCK

2025-05-23 Thread Andrey Drobyshev
On 5/20/25 1:29 PM, Fiona Ebner wrote: > This is a small step in preparation to mark bdrv_drained_begin() as > GRAPH_UNLOCKED. More concretely, it is in preparatoin to move the > drain out of bdrv_change_aio_context() and marking that function as > GRAPH_RDLOCK. > > Signed-off-by: Fiona Ebner > -

Re: [PATCH v2 19/24] block: mark bdrv_drained_begin() as GRAPH_UNLOCKED

2025-05-23 Thread Andrey Drobyshev
On 5/20/25 1:30 PM, Fiona Ebner wrote: > bdrv_drained_begin() polls and is not allowed to be called with the > block graph lock held. Mark the function as such. > > Suggested-by: Kevin Wolf > Signed-off-by: Fiona Ebner > --- > > No changes in v2, but ordered differently (in particular, it avoid

Re: [PATCH v2 1/4] nbd: Add multi-conn option

2025-05-23 Thread Andrey Drobyshev
On 5/22/25 9:44 PM, Eric Blake wrote: > On Thu, May 22, 2025 at 08:38:34PM +0300, Andrey Drobyshev wrote: >> On 4/28/25 9:46 PM, Eric Blake wrote: >>> From: "Richard W.M. Jones" >>> >>> Add multi-conn option to the NBD client. This commit

Re: [PATCH v2 1/4] nbd: Add multi-conn option

2025-05-22 Thread Andrey Drobyshev
On 4/28/25 9:46 PM, Eric Blake wrote: > From: "Richard W.M. Jones" > > Add multi-conn option to the NBD client. This commit just adds the > option, it is not functional. > > Setting this to a value > 1 permits multiple connections to the NBD > server; a typical value might be 4. The default is

Re: [PATCH v2 4/4] nbd: Enable multi-conn using round-robin

2025-05-22 Thread Andrey Drobyshev
On 4/28/25 9:46 PM, Eric Blake wrote: > From: "Richard W.M. Jones" > > Enable NBD multi-conn by spreading operations across multiple > connections. > > (XXX) This uses a naive round-robin approach which could be improved. > For example we could look at how many requests are in flight and > assig

[PATCH 0/1] Fix racy SEGFAULT upon monitor_cleanup()

2025-05-02 Thread andrey . drobyshev
From: Andrey Drobyshev There's a race in monitor cleanup code which might result into SEGFAULT. When monitor_cleanup() is launched, qmp_dispatcher_co coroutine pointer is set to NULL (see Paolo's commit 3e6bed61 ("monitor: cleanup detection of qmp_dispatcher_co shutting down&quo

[PATCH 1/1] monitor: don't wake up qmp_dispatcher_co coroutine upon cleanup

2025-05-02 Thread andrey . drobyshev
From: Andrey Drobyshev Since the commit 3e6bed61 ("monitor: cleanup detection of qmp_dispatcher_co shutting down"), coroutine pointer qmp_dispatcher_co is set to NULL upon cleanup. If a QMP command is sent after monitor_cleanup() (e.g. after shutdown), this may lead to SEGFAULT on a

Re: [BUG, RFC] Block graph deadlock on job-dismiss

2025-04-30 Thread Andrey Drobyshev
On 4/30/25 11:47 AM, Fiona Ebner wrote: > Am 24.04.25 um 19:32 schrieb Andrey Drobyshev: >> So it looks like main thread is processing job-dismiss request and is >> holding write lock taken in block_job_remove_all_bdrv() (frame #20 >> above). At the same time iothread spa

[RFC PATCH 1/1] iotests/graph-changes-while-io: add test case with removal of lower snapshot

2025-04-28 Thread andrey . drobyshev
From: Andrey Drobyshev This case is catching potential deadlock which takes place when job-dismiss is issued when I/O requests are processed in a separate iothread. See https://mail.gnu.org/archive/html/qemu-devel/2025-04/msg04421.html Signed-off-by: Andrey Drobyshev --- .../qemu-iotests

Re: [BUG, RFC] Block graph deadlock on job-dismiss

2025-04-28 Thread Andrey Drobyshev
On 4/24/25 8:32 PM, Andrey Drobyshev wrote: > Hi all, > > There's a bug in block layer which leads to block graph deadlock. > Notably, it takes place when blockdev IO is processed within a separate > iothread. > > This was initially caught by our tests, and I

[BUG, RFC] Block graph deadlock on job-dismiss

2025-04-24 Thread Andrey Drobyshev
Hi all, There's a bug in block layer which leads to block graph deadlock. Notably, it takes place when blockdev IO is processed within a separate iothread. This was initially caught by our tests, and I was able to reduce it to a relatively simple reproducer. Such deadlocks are probably supposed

Re: [PATCH v3 05/12] iotests/common.rc: add disk_usage function

2025-04-14 Thread Andrey Drobyshev
On 4/11/25 8:48 PM, Eric Blake wrote: > On Fri, Sep 13, 2024 at 07:39:35PM +0300, Andrey Drobyshev wrote: >> Move the definition from iotests/250 to common.rc. This is used to >> detect real disk usage of sparse files. In particular, we want to use >> it for checking subcl

Re: [BUG, RFC] cpr-transfer: qxl guest driver crashes after migration

2025-03-15 Thread Andrey Drobyshev
On 3/5/25 11:19 PM, Steven Sistare wrote: > On 3/5/2025 11:50 AM, Andrey Drobyshev wrote: >> On 3/4/25 9:05 PM, Steven Sistare wrote: >>> On 2/28/2025 1:37 PM, Andrey Drobyshev wrote: >>>> On 2/28/25 8:35 PM, Andrey Drobyshev wrote: >>>>> On 2/28/25

Re: [BUG, RFC] cpr-transfer: qxl guest driver crashes after migration

2025-03-05 Thread Andrey Drobyshev
On 3/4/25 9:05 PM, Steven Sistare wrote: > On 2/28/2025 1:37 PM, Andrey Drobyshev wrote: >> On 2/28/25 8:35 PM, Andrey Drobyshev wrote: >>> On 2/28/25 8:20 PM, Steven Sistare wrote: >>>> On 2/28/2025 1:13 PM, Steven Sistare wrote: >>>>> On 2/28/2025 12

Re: [BUG, RFC] cpr-transfer: qxl guest driver crashes after migration

2025-02-28 Thread Andrey Drobyshev
On 2/28/25 8:35 PM, Andrey Drobyshev wrote: > On 2/28/25 8:20 PM, Steven Sistare wrote: >> On 2/28/2025 1:13 PM, Steven Sistare wrote: >>> On 2/28/2025 12:39 PM, Andrey Drobyshev wrote: >>>> Hi all, >>>> >>>> We've been experiment

Re: [BUG, RFC] cpr-transfer: qxl guest driver crashes after migration

2025-02-28 Thread Andrey Drobyshev
On 2/28/25 8:20 PM, Steven Sistare wrote: > On 2/28/2025 1:13 PM, Steven Sistare wrote: >> On 2/28/2025 12:39 PM, Andrey Drobyshev wrote: >>> Hi all, >>> >>> We've been experimenting with cpr-transfer migration mode recently and >>> have discov

[BUG, RFC] cpr-transfer: qxl guest driver crashes after migration

2025-02-28 Thread Andrey Drobyshev
Hi all, We've been experimenting with cpr-transfer migration mode recently and have discovered the following issue with the guest QXL driver: Run migration source: > EMULATOR=/path/to/emulator > ROOTFS=/path/to/image > QMPSOCK=/var/run/alma8qmp-src.sock > > $EMULATOR -enable-kvm \ > -machine

Re: [PATCH 4/5] migration: Activate block devices if VM is paused when migrating

2024-11-27 Thread Andrey Drobyshev
err); > +return false; > +} > +s->block_inactive = false; > +} > + > return true; > } > Hi Fabiano, Thans for the fix, I can confirm that on my setup (2 nodes with common NFS share and VM's disk on the share) this patch does solve the issue. Tested-by: Andrey Drobyshev

Re: [PATCH 4/5] migration: Activate block devices if VM is paused when migrating

2024-11-27 Thread Andrey Drobyshev
On 11/25/24 7:07 PM, Peter Xu wrote: > On Mon, Nov 25, 2024 at 11:46:11AM -0300, Fabiano Rosas wrote: >> Currently a VM that has been target of a migration using >> late-block-activate will crash at the end of a new migration (with it >> as source) when releasing ownership of the disks due to the V

Re: [PATCH 1/2] preallocate: do not allow to change BDS permission improperly

2024-10-09 Thread Andrey Drobyshev
definitely a contract violation. > > Signed-off-by: Denis V. Lunev > CC: Andrey Drobyshev > CC: Vladimir Sementsov-Ogievskiy > CC: Kevin Wolf > --- > block/preallocate.c | 12 > 1 file changed, 12 insertions(+) > > diff --git a/block/preallocate.

Re: [PATCH v3 00/12] qcow2: make subclusters discardable

2024-10-09 Thread Andrey Drobyshev
On 9/30/24 5:24 PM, Andrey Drobyshev wrote: > On 9/13/24 7:39 PM, Andrey Drobyshev wrote: >> v2: https://lists.nongnu.org/archive/html/qemu-devel/2024-05/msg02396.html >> >> v2 -> v3: >> * Added patch 12/12 "qcow2: add discard-subclusters option" whic

Re: [PATCH 2/2] block/preallocate: fix image truncation logic

2024-10-09 Thread Andrey Drobyshev
so disallow waiting inside. Thus original approach with the bottom > half is not changed. bdrv_drain_begin() schedules the operation and in > order to ensure that it has been really executed before completion of > the section increments the amount of in-flight requests. > > Signed-o

Re: [PATCH v3 00/12] qcow2: make subclusters discardable

2024-09-30 Thread Andrey Drobyshev
On 9/13/24 7:39 PM, Andrey Drobyshev wrote: > v2: https://lists.nongnu.org/archive/html/qemu-devel/2024-05/msg02396.html > > v2 -> v3: > * Added patch 12/12 "qcow2: add discard-subclusters option" which > makes subcluster-based discards optional; > * Adde

Re: [Bug Report][RFC PATCH 1/1] block: fix failing assert on paused VM migration

2024-09-30 Thread Andrey Drobyshev
On 9/30/24 12:25 PM, Vladimir Sementsov-Ogievskiy wrote: > [add migration maintainers] > > On 24.09.24 15:56, Andrey Drobyshev wrote: >> [...] > > I doubt that this a correct way to go. > > As far as I understand, "inactive" actually means that "stora

[Bug Report][RFC PATCH 0/1] block: fix failing assert on paused VM migration

2024-09-24 Thread Andrey Drobyshev
ag if it's already set is more of a kludge than a clean solution. Should we use more sophisticated logic which allows some of the nodes be in inactive state prior to the migration, and takes them into account during bdrv_inactivate_all()? Comments would be appreciated. Andrey Andrey Dro

[Bug Report][RFC PATCH 1/1] block: fix failing assert on paused VM migration

2024-09-24 Thread Andrey Drobyshev
the nodes which should be inactive at this point. Signed-off-by: Andrey Drobyshev --- block.c | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/block.c b/block.c index 7d90007cae..c1dcf906d1 100644 --- a/block.c +++ b/block.c @@ -6973,7 +6973,15 @@ static i

Re: [PATCH v3 12/12] qcow2: add discard-subclusters option

2024-09-15 Thread Andrey Drobyshev
On 9/13/24 7:39 PM, Andrey Drobyshev wrote: > Introduce Qcow2 runtime boolean option "discard-subclusters". This > option influences discard alignment value (either cluster_size or > subcluster_size) and essentially makes subcluster-based discard optional. > We disable it

[PATCH v3 04/12] block/file-posix: add trace event for fallocate() calls

2024-09-13 Thread Andrey Drobyshev
This would ease debugging of write zeroes and discard operations. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Reviewed-by: Alberto Garcia --- block/file-posix.c | 1 + block/trace-events | 1 + 2 files changed, 2 insertions(+) diff --git a/block/file-posix.c b/block/file

[PATCH v3 01/12] qcow2: make function update_refcount_discard() global

2024-09-13 Thread Andrey Drobyshev
We are going to need it for discarding separate subclusters. The function itself doesn't do anything with the refcount tables, it simply adds a discard request to the queue, so rename it to qcow2_queue_discard(). Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Reviewed-by:

[PATCH v3 07/12] qcow2: add get_sc_range_info() helper for working with subcluster ranges

2024-09-13 Thread Andrey Drobyshev
ation. Also introduce struct SubClusterRangeInfo, which would contain all the needed params. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov --- block/qcow2-cluster.c | 140 -- 1 file changed, 108 insertions(+), 32 deletions(-) diff --git a/

[PATCH v3 08/12] qcow2: zeroize the entire cluster when there're no non-zero subclusters

2024-09-13 Thread Andrey Drobyshev
When zeroizing the last non-zero subclusters within single cluster, it makes sense to go zeroize the entire cluster and go down zero_in_l2_slice() path right away. That way we'd also update the corresponding refcount table. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Rev

[PATCH v3 09/12] qcow2: make subclusters discardable

2024-09-13 Thread Andrey Drobyshev
with subclusters enabled. Also rename qcow2_cluster_discard() -> qcow2_subcluster_discard() to reflect the change. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov --- block/qcow2-cluster.c | 106 + block/qcow2-snapshot.c | 6 +-- block

[PATCH v3 10/12] qcow2: zero_l2_subclusters: fall through to discard operation when requested

2024-09-13 Thread Andrey Drobyshev
quot; will lead to actual unmap. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov --- block/qcow2-cluster.c | 26 -- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 3c134a7e80..53e04eff93

[PATCH v3 11/12] iotests/271: add test cases for subcluster-based discard/unmap

2024-09-13 Thread Andrey Drobyshev
make all discard/unmap operations enable trace point 'file_do_fallocate' so that actual fallocate() calls are visible. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov --- tests/qemu-iotests/271 | 70 +- tests/qemu-iotests/27

[PATCH v3 05/12] iotests/common.rc: add disk_usage function

2024-09-13 Thread Andrey Drobyshev
Move the definition from iotests/250 to common.rc. This is used to detect real disk usage of sparse files. In particular, we want to use it for checking subclusters-based discards. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Reviewed-by: Alberto Garcia --- tests/qemu

[PATCH v3 06/12] iotests/290: add test case to check 'discard-no-unref' option behavior

2024-09-13 Thread Andrey Drobyshev
) and that with the option enabled cluster is still marked as allocated in "qemu-img map" output. We also check that the option doesn't work with qcow2 v2 images. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Reviewed-by: Alberto Garcia --- tests/

[PATCH v3 12/12] qcow2: add discard-subclusters option

2024-09-13 Thread Andrey Drobyshev
eally test subcluster based discards. Signed-off-by: Andrey Drobyshev --- block/qcow2.c | 21 - block/qcow2.h | 2 ++ tests/qemu-iotests/271 | 10 ++ 3 files changed, 28 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index

[PATCH v3 02/12] qcow2: simplify L2 entries accounting for discard-no-unref

2024-09-13 Thread Andrey Drobyshev
atter whether we unmap or zeroize the cluster. For that OR'ing with the old entry is enough. This patch doesn't change the logic and is pure refactoring. Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov Reviewed-by: Alberto Garcia --- block/qcow2

[PATCH v3 00/12] qcow2: make subclusters discardable

2024-09-13 Thread Andrey Drobyshev
v2: https://lists.nongnu.org/archive/html/qemu-devel/2024-05/msg02396.html v2 -> v3: * Added patch 12/12 "qcow2: add discard-subclusters option" which makes subcluster-based discards optional; * Added a bunch of R-b's. Andrey Drobyshev (12): qcow2: make function updat

[PATCH v3 03/12] qcow2: put discard requests in the common queue when discard-no-unref enabled

2024-09-13 Thread Andrey Drobyshev
check for the presence of external data files for us and redirect request to underlying data_file. Here we want to do the same but avoid refcount updates, thus we perform the same checks. Suggested-by: Hanna Czenczek Signed-off-by: Andrey Drobyshev Reviewed-by: Alexander Ivanov --- block/

Re: [PATCH v3 3/3] scripts: add filev2p.py script for mapping virtual file offsets mapping

2024-08-05 Thread Andrey Drobyshev
On 8/5/24 3:29 PM, Kevin Wolf wrote: > Am 16.07.2024 um 16:41 hat Andrey Drobyshev geschrieben: >> The script is basically a wrapper around "filefrag" utility. This might >> be used to map virtual offsets within the file to the underlying block >> device offsets.

Re: [PATCH v3 2/3] iotests/298: add testcase for async writes with preallocation filter

2024-08-05 Thread Andrey Drobyshev
On 8/5/24 3:04 PM, Kevin Wolf wrote: > Am 16.07.2024 um 16:41 hat Andrey Drobyshev geschrieben: >> The testcase simply creates a 64G image with 1M clusters, generates a list >> of 1M aligned offsets and feeds aio_write commands with those offsets to >> qemu-io run with &#

[PATCH] iotests/024: exclude 'backing file format' field from the output

2024-07-30 Thread Andrey Drobyshev
m the output. This is a fixup for the commit f93e65ee51 ("iotests/{024, 271}: add testcases for qemu-img rebase"). Found-by: Thomas Huth Signed-off-by: Andrey Drobyshev --- tests/qemu-iotests/024 | 2 +- tests/qemu-iotests/024.out | 1 - 2 files changed, 1 insertion(+), 2 deletion

Re: [PULL 06/27] iotests/{024, 271}: add testcases for qemu-img rebase

2024-07-30 Thread Andrey Drobyshev
On 7/22/24 10:18 AM, Thomas Huth wrote: > [Вы нечасто получаете письма от th...@redhat.com. Узнайте, почему это > важно, по адресу https://aka.ms/LearnAboutSenderIdentification ] > > On 31/10/2023 19.58, Kevin Wolf wrote: >> From: Andrey Drobyshev >> >> As the pre

[PATCH v3 3/3] scripts: add filev2p.py script for mapping virtual file offsets mapping

2024-07-16 Thread Andrey Drobyshev
;t work with partitions built on top of LVM. Partitions on LVM would require another level of mapping. Signed-off-by: Andrey Drobyshev --- scripts/filev2p.py | 311 + 1 file changed, 311 insertions(+) create mode 100755 scripts/filev2p.py diff --gi

[PATCH v3 2/3] iotests/298: add testcase for async writes with preallocation filter

2024-07-16 Thread Andrey Drobyshev
esult into a race within the preallocation filter which would zeroize some clusters after actually writing data to them. Note: the test doesn't fail in 100% cases as there's a race involved, but the failures are pretty consistent so it should be good enough for detecting the problem. S

[PATCH v3 1/3] block: zero data data corruption using prealloc-filter

2024-07-16 Thread Andrey Drobyshev
this area and they will not proceed thanks to overlapping requests mechanics. If preallocation will fail, we will just switch to the normal expand-by-write behavior and that is not a problem except performance. Signed-off-by: Denis V. Lunev Tested-by: Andrey Drobyshev --- block/preallocate.c |

[PATCH v3 0/3] Fix data corruption within preallocation

2024-07-16 Thread Andrey Drobyshev
ets. The script was used to initially track down the data corruption problem, so it's included here. v2: https://lists.nongnu.org/archive/html/qemu-block/2024-07/msg00413.html Andrey Drobyshev (2): iotests/298: add testcase for async writes with preallocation filter scri

Re: [PATCH v2 1/2] block: zero data data corruption using prealloc-filter

2024-07-16 Thread Andrey Drobyshev
On 7/16/24 4:32 PM, Denis V. Lunev wrote: > On 7/12/24 13:55, Vladimir Sementsov-Ogievskiy wrote: >> On 12.07.24 12:46, Andrey Drobyshev wrote: >>> From: "Denis V. Lunev" >>> >>> We have observed that some clusters in the QCOW2 files are zeroed >

[PATCH v2 3/2] scripts: add filev2p.py script for mapping virtual file offsets mapping

2024-07-15 Thread Andrey Drobyshev
;t work with partitions built on top of LVM. Partitions on LVM would require another level of mapping. Signed-off-by: Andrey Drobyshev --- scripts/filev2p.py | 311 + 1 file changed, 311 insertions(+) create mode 100755 scripts/filev2p.py diff --gi

Re: [PATCH v2 00/11] qcow2: make subclusters discardable

2024-07-15 Thread Andrey Drobyshev
On 7/8/24 10:06 AM, Andrey Drobyshev wrote: > On 6/24/24 10:43 AM, Andrey Drobyshev wrote: >> On 6/17/24 9:39 AM, Andrey Drobyshev wrote: >>> On 6/10/24 11:53 AM, Andrey Drobyshev wrote: >>>> On 6/3/24 12:19 PM, Andrey Drobyshev wrote: >>>>> On 5/13/

[PATCH v2 1/2] block: zero data data corruption using prealloc-filter

2024-07-12 Thread Andrey Drobyshev
this area and they will not proceed thanks to overlapping requests mechanics. If preallocation will fail, we will just switch to the normal expand-by-write behavior and that is not a problem except performance. Signed-off-by: Denis V. Lunev Tested-by: Andrey Drobyshev --- block/preallocate.c |

[PATCH v2 0/2] Fix data corruption within preallocation

2024-07-12 Thread Andrey Drobyshev
the operations performed in other coroutines (e.g. truncate) and outside of the coroutine context (preallocate_set_perm() / preallocate_child_perm()). So the comments on this are most welcome. v1: https://lists.nongnu.org/archive/html/qemu-block/2024-07/msg00384.html Andrey Drobyshev (1): iotests/

[PATCH v2 2/2] iotests/298: add testcase for async writes with preallocation filter

2024-07-12 Thread Andrey Drobyshev
esult into a race within the preallocation filter which would zeroize some clusters after actually writing data to them. Note: the test doesn't fail in 100% cases as there's a race involved, but the failures are pretty consistent so it should be good enough for detecting the problem. S

[PATCH 0/2] Fix data corruption within preallocation

2024-07-11 Thread Andrey Drobyshev
7;re other operations which are run outside of the coroutine context and which also modify the file_end field, e.g. preallocate_set_perm() and preallocate_child_perm(). Comments on how to protect the field properly regardless of the context are welcome. Andrey Drobyshev (1): iotests/298: add testcase

[PATCH 2/2] iotests/298: add testcase for async writes with preallocation filter

2024-07-11 Thread Andrey Drobyshev
esult into a race within the preallocation filter which would zeroize some clusters after actually writing data to them. Note: the test doesn't fail in 100% cases as there's a race involved, but the failures are pretty consistent so it should be good enough for detecting the problem. S

[PATCH 1/2] block: zero data data corruption using prealloc-filter

2024-07-11 Thread Andrey Drobyshev
only once it is detected that the preallocation is really required. This is not a frequent case due to the preallocation nature thus the patch should not have performance impact. Originally-by: Denis V. Lunev Co-authored-by: Andrey Drobyshev Signed-off-by: Denis V. Lunev Signed-off-by: And

Re: [PATCH v2 00/11] qcow2: make subclusters discardable

2024-07-08 Thread Andrey Drobyshev
On 6/24/24 10:43 AM, Andrey Drobyshev wrote: > On 6/17/24 9:39 AM, Andrey Drobyshev wrote: >> On 6/10/24 11:53 AM, Andrey Drobyshev wrote: >>> On 6/3/24 12:19 PM, Andrey Drobyshev wrote: >>>> On 5/13/24 9:31 AM, Andrey Drobyshev wrote: >>>>> v1: htt

Re: [PATCH v2 00/11] qcow2: make subclusters discardable

2024-06-24 Thread Andrey Drobyshev
On 6/17/24 9:39 AM, Andrey Drobyshev wrote: > On 6/10/24 11:53 AM, Andrey Drobyshev wrote: >> On 6/3/24 12:19 PM, Andrey Drobyshev wrote: >>> On 5/13/24 9:31 AM, Andrey Drobyshev wrote: >>>> v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-10/msg07223.html

Re: [PATCH v2 00/11] qcow2: make subclusters discardable

2024-06-17 Thread Andrey Drobyshev
On 6/10/24 11:53 AM, Andrey Drobyshev wrote: > On 6/3/24 12:19 PM, Andrey Drobyshev wrote: >> On 5/13/24 9:31 AM, Andrey Drobyshev wrote: >>> v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-10/msg07223.html >>> >>> Andrey Drobyshev (11): >>>

Re: [PATCH v2 00/11] qcow2: make subclusters discardable

2024-06-10 Thread Andrey Drobyshev
On 6/3/24 12:19 PM, Andrey Drobyshev wrote: > On 5/13/24 9:31 AM, Andrey Drobyshev wrote: >> v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-10/msg07223.html >> >> Andrey Drobyshev (11): >> qcow2: make function update_refcount_discard() global >> qcow

Re: [PATCH v2 00/11] qcow2: make subclusters discardable

2024-06-03 Thread Andrey Drobyshev
On 5/13/24 9:31 AM, Andrey Drobyshev wrote: > v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-10/msg07223.html > > Andrey Drobyshev (11): > qcow2: make function update_refcount_discard() global > qcow2: simplify L2 entries accounting for discard-no-unref >

[PATCH v2 03/11] qcow2: put discard requests in the common queue when discard-no-unref enabled

2024-05-12 Thread Andrey Drobyshev
check for the presence of external data files for us and redirect request to underlying data_file. Here we want to do the same but avoid refcount updates, thus we perform the same checks. Suggested-by: Hanna Czenczek Signed-off-by: Andrey Drobyshev --- block/qcow2-cluster.c

[PATCH v2 06/11] iotests/290: add test case to check 'discard-no-unref' option behavior

2024-05-12 Thread Andrey Drobyshev
) and that with the option enabled cluster is still marked as allocated in "qemu-img map" output. We also check that the option doesn't work with qcow2 v2 images. Signed-off-by: Andrey Drobyshev --- tests/qemu-iotests/290 | 34 ++ tests/

[PATCH v2 00/11] qcow2: make subclusters discardable

2024-05-12 Thread Andrey Drobyshev
v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-10/msg07223.html Andrey Drobyshev (11): qcow2: make function update_refcount_discard() global qcow2: simplify L2 entries accounting for discard-no-unref qcow2: put discard requests in the common queue when discard-no-unref

[PATCH v2 11/11] iotests/271: add test cases for subcluster-based discard/unmap

2024-05-12 Thread Andrey Drobyshev
make all discard/unmap operations enable trace point 'file_do_fallocate' so that actual fallocate() calls are visible. Signed-off-by: Andrey Drobyshev --- tests/qemu-iotests/271 | 70 +- tests/qemu-iotests/271.out | 69 ++

[PATCH v2 07/11] qcow2: add get_sc_range_info() helper for working with subcluster ranges

2024-05-12 Thread Andrey Drobyshev
ation. Also introduce struct SubClusterRangeInfo, which would contain all the needed params. Signed-off-by: Andrey Drobyshev --- block/qcow2-cluster.c | 140 -- 1 file changed, 108 insertions(+), 32 deletions(-) diff --git a/block/qcow2-cluster.c b/block/

[PATCH v2 05/11] iotests/common.rc: add disk_usage function

2024-05-12 Thread Andrey Drobyshev
Move the definition from iotests/250 to common.rc. This is used to detect real disk usage of sparse files. In particular, we want to use it for checking subclusters-based discards. Signed-off-by: Andrey Drobyshev --- tests/qemu-iotests/250 | 5 - tests/qemu-iotests/common.rc | 6

[PATCH v2 09/11] qcow2: make subclusters discardable

2024-05-12 Thread Andrey Drobyshev
with subclusters enabled. Also rename qcow2_cluster_discard() -> qcow2_subcluster_discard() to reflect the change. Signed-off-by: Andrey Drobyshev --- block/qcow2-cluster.c | 106 + block/qcow2-snapshot.c | 6 +-- block/qcow2.c | 25 +

[PATCH v2 02/11] qcow2: simplify L2 entries accounting for discard-no-unref

2024-05-12 Thread Andrey Drobyshev
atter whether we unmap or zeroize the cluster. For that OR'ing with the old entry is enough. This patch doesn't change the logic and is pure refactoring. Signed-off-by: Andrey Drobyshev --- block/qcow2-cluster.c | 34 +++--- 1 file changed, 15 insertions(+

[PATCH v2 04/11] block/file-posix: add trace event for fallocate() calls

2024-05-12 Thread Andrey Drobyshev
This would ease debugging of write zeroes and discard operations. Signed-off-by: Andrey Drobyshev --- block/file-posix.c | 1 + block/trace-events | 1 + 2 files changed, 2 insertions(+) diff --git a/block/file-posix.c b/block/file-posix.c index 35684f7e21..45134f0eef 100644 --- a/block/file

[PATCH v2 10/11] qcow2: zero_l2_subclusters: fall through to discard operation when requested

2024-05-12 Thread Andrey Drobyshev
quot; will lead to actual unmap. Signed-off-by: Andrey Drobyshev --- block/qcow2-cluster.c | 26 -- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 3c134a7e80..53e04eff93 100644 --- a/block/qcow2-cl

[PATCH v2 01/11] qcow2: make function update_refcount_discard() global

2024-05-12 Thread Andrey Drobyshev
We are going to need it for discarding separate subclusters. The function itself doesn't do anything with the refcount tables, it simply adds a discard request to the queue, so rename it to qcow2_queue_discard(). Signed-off-by: Andrey Drobyshev Reviewed-by: Hanna Czenczek --- block/

[PATCH v2 08/11] qcow2: zeroize the entire cluster when there're no non-zero subclusters

2024-05-12 Thread Andrey Drobyshev
When zeroizing the last non-zero subclusters within single cluster, it makes sense to go zeroize the entire cluster and go down zero_in_l2_slice() path right away. That way we'd also update the corresponding refcount table. Signed-off-by: Andrey Drobyshev Reviewed-by: Hanna Czenczek ---

Re: [BUG, RFC] Base node is in RW after making external snapshot

2024-05-10 Thread Andrey Drobyshev
On 4/24/24 21:00, Andrey Drobyshev wrote: > Hi everyone, > > When making an external snapshot, we end up in a situation when 2 block > graph nodes related to the same image file (format and storage nodes) > have different RO flags set on them. > > E.g. > > # ls -la

[BUG, RFC] Base node is in RW after making external snapshot

2024-04-24 Thread Andrey Drobyshev
Hi everyone, When making an external snapshot, we end up in a situation when 2 block graph nodes related to the same image file (format and storage nodes) have different RO flags set on them. E.g. # ls -la /proc/PID/fd lrwx-- 1 root qemu 64 Apr 24 20:14 12 -> /path/to/harddisk.hdd # virsh q

Re: [PATCH 4/7] qcow2: make subclusters discardable

2024-04-16 Thread Andrey Drobyshev
On 10/27/23 14:10, Jean-Louis Dupond wrote: > [...] > > I've checked all the code paths, and as far as I see it nowhere breaks > the discard_no_unref option. > It's important that we don't introduce new code paths that can make > holes in the qcow2 image when this option is enabled :) > > If you

Re: [PATCH v4 1/7] qga: guest-get-fsinfo: add optional 'total-bytes-privileged' field

2024-04-01 Thread Andrey Drobyshev
On 3/22/24 15:17, Andrey Drobyshev wrote: > On 3/22/24 12:39, Daniel P. Berrangé wrote: >> On Wed, Mar 20, 2024 at 06:16:42PM +0200, Andrey Drobyshev wrote: >>> Since the commit 25b5ff1a86 ("qga: add mountpoint usage info to >>> GuestFilesystemInfo") we have

Re: [PATCH v4 1/7] qga: guest-get-fsinfo: add optional 'total-bytes-privileged' field

2024-03-22 Thread Andrey Drobyshev
On 3/22/24 12:39, Daniel P. Berrangé wrote: > On Wed, Mar 20, 2024 at 06:16:42PM +0200, Andrey Drobyshev wrote: >> Since the commit 25b5ff1a86 ("qga: add mountpoint usage info to >> GuestFilesystemInfo") we have 2 values reported in guest-get-fsinfo: >> use

[PATCH v4 3/7] qga/commands-posix: qmp_guest_shutdown: use ga_run_command helper

2024-03-20 Thread Andrey Drobyshev
Also remove the G_GNUC_UNUSED attribute added in the previous commit from the helper. Signed-off-by: Andrey Drobyshev Reviewed-by: Daniel P. Berrangé --- qga/commands-posix.c | 39 ++- 1 file changed, 6 insertions(+), 33 deletions(-) diff --git a/qga

[PATCH v4 1/7] qga: guest-get-fsinfo: add optional 'total-bytes-privileged' field

2024-03-20 Thread Andrey Drobyshev
eturned by statvfs(3). While here, also tweak the docs to reflect better where those values come from. Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 2 ++ qga/commands-win32.c | 1 + qga/qapi-schema.json | 7 +-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/qga

[PATCH v4 0/7] qga/commands-posix: replace code duplicating commands with a helper

2024-03-20 Thread Andrey Drobyshev
to local error declaration. v3: https://lists.nongnu.org/archive/html/qemu-devel/2024-03/msg04068.html Andrey Drobyshev (7): qga: guest-get-fsinfo: add optional 'total-bytes-privileged' field qga: introduce ga_run_command() helper for guest cmd execution qga/commands-posix: qmp_guest_shutdown

[PATCH v4 5/7] qga/commands-posix: execute_fsfreeze_hook: use ga_run_command helper

2024-03-20 Thread Andrey Drobyshev
There's no need to check for the existence of the hook executable, as the exec() call will do that for us. Signed-off-by: Andrey Drobyshev Reviewed-by: Daniel P. Berrangé --- qga/commands-posix.c | 35 +++ 1 file changed, 3 insertions(+), 32 deletions(-)

[PATCH v4 6/7] qga/commands-posix: don't do fork()/exec() when suspending via sysfs

2024-03-20 Thread Andrey Drobyshev
r/state. Let's just use g_file_set_contents() to simplify things here. Suggested-by: Daniel P. Berrangé Signed-off-by: Andrey Drobyshev Reviewed-by: Daniel P. Berrangé --- qga/commands-posix.c | 41 + 1 file changed, 5 insertions(+), 36 deletions(

[PATCH v4 4/7] qga/commands-posix: qmp_guest_set_time: use ga_run_command helper

2024-03-20 Thread Andrey Drobyshev
There's no need to check for the existence of "/sbin/hwclock", the exec() call will do that for us. Signed-off-by: Andrey Drobyshev Reviewed-by: Daniel P. Berrangé --- qga/commands-posix.c | 43 +++ 1 file changed, 3 insertions(+), 40 del

[PATCH v4 7/7] qga/commands-posix: qmp_guest_set_user_password: use ga_run_command helper

2024-03-20 Thread Andrey Drobyshev
There's no need to check for the existence of the "chpasswd", "pw" executables, as the exec() call will do that for us. Signed-off-by: Andrey Drobyshev Reviewed-by: Daniel P. Berrangé --- qga/commands-posix.c | 96 ++-- 1 fi

[PATCH v4 2/7] qga: introduce ga_run_command() helper for guest cmd execution

2024-03-20 Thread Andrey Drobyshev
story more bisectable, let's replace qmp commands implementations one by one. Also add G_GNUC_UNUSED attribute to the helper and remove it in the next commit. Originally-by: Yuri Pudgorodskiy Signed-off-by: Andrey Drobyshev Reviewed-by: Daniel P. Berrangé --- qga/command

Re: [PATCH v3 1/7] qga: guest-get-fsinfo: add optional 'total-bytes-root' field

2024-03-20 Thread Andrey Drobyshev
On 3/19/24 19:37, Daniel P. Berrangé wrote: > On Fri, Mar 15, 2024 at 02:29:40PM +0200, Andrey Drobyshev wrote: >> Since the commit 25b5ff1a86 ("qga: add mountpoint usage info to >> GuestFilesystemInfo") we have 2 values reported in guest-get-fsinfo: >> use

Re: [PATCH v3 6/7] qga/commands-posix: don't do fork()/exec() when suspending via sysfs

2024-03-20 Thread Andrey Drobyshev
On 3/19/24 20:02, Daniel P. Berrangé wrote: > On Fri, Mar 15, 2024 at 02:29:45PM +0200, Andrey Drobyshev wrote: >> Since commit 246d76eba ("qga: guest_suspend: decoupling pm-utils and sys >> logic") pm-utils logic is running in a separate child from the sysfs >>

Re: [PATCH v3 1/7] qga: guest-get-fsinfo: add optional 'total-bytes-root' field

2024-03-15 Thread Andrey Drobyshev
On 3/15/24 15:44, Markus Armbruster wrote: > [?? ??? ? ?? ?? arm...@redhat.com. ???, ?? ??? ?, > ?? ?? https://aka.ms/LearnAboutSenderIdentification ] > > Andrey Drobyshev writes: > >> Since the commit 25b5ff1a86 ("qga: ad

[PATCH v3 5/7] qga/commands-posix: execute_fsfreeze_hook: use ga_run_command helper

2024-03-15 Thread Andrey Drobyshev
There's no need to check for the existence of the hook executable, as the exec() call will do that for us. Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 35 +++ 1 file changed, 3 insertions(+), 32 deletions(-) diff --git a/qga/commands-posix.c

[PATCH v3 7/7] qga/commands-posix: qmp_guest_set_user_password: use ga_run_command helper

2024-03-15 Thread Andrey Drobyshev
There's no need to check for the existence of the "chpasswd", "pw" executables, as the exec() call will do that for us. Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 96 ++-- 1 file changed, 13 insertions(+), 83 delet

[PATCH v3 3/7] qga/commands-posix: qmp_guest_shutdown: use ga_run_command helper

2024-03-15 Thread Andrey Drobyshev
Also remove the G_GNUC_UNUSED attribute added in the previous commit from the helper. Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 39 ++- 1 file changed, 6 insertions(+), 33 deletions(-) diff --git a/qga/commands-posix.c b/qga/commands-posix.c

[PATCH v3 4/7] qga/commands-posix: qmp_guest_set_time: use ga_run_command helper

2024-03-15 Thread Andrey Drobyshev
There's no need to check for the existence of "/sbin/hwclock", the exec() call will do that for us. Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 43 +++ 1 file changed, 3 insertions(+), 40 deletions(-) diff --git a/qga/com

[PATCH v3 2/7] qga: introduce ga_run_command() helper for guest cmd execution

2024-03-15 Thread Andrey Drobyshev
story more bisectable, let's replace qmp commands implementations one by one. Also add G_GNUC_UNUSED attribute to the helper and remove it in the next commit. Originally-by: Yuri Pudgorodskiy Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 150

[PATCH v3 6/7] qga/commands-posix: don't do fork()/exec() when suspending via sysfs

2024-03-15 Thread Andrey Drobyshev
r/state. Let's just use g_file_set_contents() to simplify things here. Suggested-by: Daniel P. Berrangé Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 41 + 1 file changed, 5 insertions(+), 36 deletions(-) diff --git a/qga/commands-po

[PATCH v3 0/7] qga/commands-posix: replace code duplicating commands with a helper

2024-03-15 Thread Andrey Drobyshev
tps://lists.nongnu.org/archive/html/qemu-devel/2024-03/msg00147.html Andrey Drobyshev (7): qga: guest-get-fsinfo: add optional 'total-bytes-root' field qga: introduce ga_run_command() helper for guest cmd execution qga/commands-posix: qmp_guest_shutdown: use ga_run_command helper qga/c

[PATCH v3 1/7] qga: guest-get-fsinfo: add optional 'total-bytes-root' field

2024-03-15 Thread Andrey Drobyshev
turned by statvfs(3). While here, let's document better where those values come from in both POSIX and Windows. Signed-off-by: Andrey Drobyshev --- qga/commands-posix.c | 2 ++ qga/commands-win32.c | 1 + qga/qapi-schema.json | 12 +++- 3 files changed, 14 insertions(+), 1 deletio

Re: [PATCH v2 6/7] qga/commands-posix: use ga_run_command helper when suspending via sysfs

2024-03-15 Thread Andrey Drobyshev
On 3/5/24 20:34, Daniel P. Berrangé wrote: > [Вы нечасто получаете письма от berra...@redhat.com. Узнайте, почему это > важно, по адресу https://aka.ms/LearnAboutSenderIdentification ] > > On Fri, Mar 01, 2024 at 07:28:57PM +0200, Andrey Drobyshev wrote: >> We replace the d

  1   2   3   >