[Devel] [RFC PATCH vz9 v5 04/49] dm-ploop: move md status to use proper bitops

2024-11-17 Thread Alexander Atanasov
Fix direct bitops to use set_bit/clear_bit which are atomic - this is required since there are some places in code that do not use locking when operating on that bits. this is also a preparation to relax locking. https://virtuozzo.atlassian.net/browse/VSTOR-91820 Signed-off-by: Alexander Atanasov

[Devel] [RFC PATCH vz9 v5 03/49] dm-ploop: fsync after all pios are sent

2024-11-17 Thread Alexander Atanasov
Currently there are two workers one to handle pios, one to handle flush (via vfs_fsync). This workers are created unbound which means they are run whenever there is a free CPU. When ploop sends pios (via ploop_dispatch_pios) it checks if there are data and if there are flush pios. If both are prese

[Devel] [PATCH VZ9 0/2] dm-qcow2: add a memory pool for big buffer allocations

2024-11-17 Thread Pavel Tikhomirov
note: not yet fully tested (waiting for reproduce) https://virtuozzo.atlassian.net/browse/VSTOR-94596 Signed-off-by: Pavel Tikhomirov Pavel Tikhomirov (2): mempool: add kvmalloc/kvfree version dm-qcow2: add memory pool for compression buffers drivers/md/dm-qcow2-map.c| 18 +

[Devel] [RFC PATCH vz9 v5 09/49] dm-ploop: convert suspended_pios list to use lockless list

2024-11-17 Thread Alexander Atanasov
https://virtuozzo.atlassian.net/browse/VSTOR-91820 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-cmd.c| 13 +++-- drivers/md/dm-ploop-map.c| 2 +- drivers/md/dm-ploop-target.c | 2 +- drivers/md/dm-ploop.h| 2 +- 4 files changed, 14 insertions(+), 5 deletion

[Devel] [RFC PATCH vz9 v5 02/49] dm-ploop: Use READ_ONCE/WRITE_ONCE to access md page data

2024-11-17 Thread Alexander Atanasov
Prepare to reduce locking by using atomic 32 bit access to the fields. To ensure this we need to use the _ONCE macros. https://virtuozzo.atlassian.net/browse/VSTOR-91659 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-bat.c | 20 ++-- drivers/md/dm-ploop-cmd.c | 28

[Devel] [RFC PATCH vz9 v5 14/49] dm-ploop: move preparations of pios into the caller from worker

2024-11-17 Thread Alexander Atanasov
Prepare pios earlier in preparation to try to execute them earlier. Convert more places to use lock less lists. https://virtuozzo.atlassian.net/browse/VSTOR-91820 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-map.c | 95 --- 1 file changed, 60 inse

[Devel] [PATCH VZ9 1/2] mempool: add kvmalloc/kvfree version

2024-11-17 Thread Pavel Tikhomirov
We need it to amke a memory pool for 4M (10-th order) allocations. Due to big allocation size it seems prudent to do it via vmalloc interface. https://virtuozzo.atlassian.net/browse/VSTOR-94596 Signed-off-by: Pavel Tikhomirov --- include/linux/mempool.h | 15 +++ mm/mempool.c

[Devel] [RFC PATCH vz9 v5 46/49] dm-ploop: add pio list ids to be used when passing pios to runners

2024-11-17 Thread Alexander Atanasov
Add specific list ids for writeback and flush pios, and process them inside the runners. https://virtuozzo.atlassian.net/browse/VSTOR-91821 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-map.c | 4 drivers/md/dm-ploop.h | 3 +++ 2 files changed, 7 insertions(+) diff --git a/

[Devel] [RFC PATCH vz9 v5 01/49] dm-ploop: md_pages map all pages at creation time

2024-11-17 Thread Alexander Atanasov
md_page is always present in memory. In that case md_page->page could be always be mapped and we would not need to perform kmap_atomic/kunmap_atomic during each lookup https://virtuozzo.atlassian.net/browse/VSTOR-91659 Suggested-by: Denis V. Lunev Signed-off-by: Alexander Atanasov --- drivers/m

[Devel] [RFC PATCH vz9 v5 29/49] dm-ploop: do not use a wrapper for set_bit to make a page writeback

2024-11-17 Thread Alexander Atanasov
unify how MD_WRITEBACK bit is set across the code - one of the files is using a wrapper the rest of the code directly sets the bit. To avoid confusion remove the wrapper. https://virtuozzo.atlassian.net/browse/VSTOR-91817 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-cmd.c | 9 ++

[Devel] [RFC PATCH vz9 v5 44/49] dm-ploop: make ploop_submit_metadata_writeback return number of requests sent

2024-11-17 Thread Alexander Atanasov
We need to wait for write back to complete to issue pending flushes. To know if we have to wait return numer of pios submitted. https://virtuozzo.atlassian.net/browse/VSTOR-91821 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-map.c | 18 ++ 1 file changed, 14 insertion

[Devel] [PATCH VZ9 2/2] dm-qcow2: add memory pool for compression buffers

2024-11-17 Thread Pavel Tikhomirov
We see high order allocation warnings: kernel: order 10 >= 10, gfp 0x40c00 kernel: WARNING: CPU: 5 PID: 182 at mm/page_alloc.c:5630 __alloc_pages+0x1d7/0x3f0 kernel: process_compressed_read+0x6f/0x590 [dm_qcow2] This is because we have 1M clusters and in case of zstd compression the buffer size

Re: [Devel] [PATCH VZ9 0/2] dm-qcow2: add a memory pool for big buffer allocations

2024-11-17 Thread Pavel Tikhomirov
Oh, from the second glance, it looks like I can't use kvmalloc for the pool allocation as mempool_alloc tries to call mempool_t->alloc directly first, and only then fallbacks to elements from pool, but kvmalloc is not good for GFP_NOIO allocations... Please disregard this series, I will try to

[Devel] [RFC PATCH vz9 v5 43/49] dm-ploop: properly end pios attached to piwb pios

2024-11-17 Thread Alexander Atanasov
Fix a bug in ploop_data_pio_end. If piwb is already completed, which can happen if we attach a flush pio to it, pio is not added to the list and ploop_pio_endio is never called on it. This results in a hang since pio is never completed. Avoid this by calling ploop_pio_endio directly in case piwb

[Devel] [RFC PATCH vz9 v5 16/49] dm-ploop: do not flush after metadata writes

2024-11-17 Thread Alexander Atanasov
From: Andrey Zhadchenko Flushed should be explicitly called by the writer if he really wants it. Drop ploop_md_write_endio() and ploop_md_fsync_endio(). https://virtuozzo.atlassian.net/browse/VSTOR-91817 Signed-off-by: Andrey Zhadchenko --- drivers/md/dm-ploop-map.c | 31 +++---

[Devel] [RFC PATCH vz9 v5 34/49] dm-ploop: convert high_prio to status

2024-11-17 Thread Alexander Atanasov
convert bool high prio to status bit - reduce size and gain atomicity https://virtuozzo.atlassian.net/browse/VSTOR-91817 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-bat.c | 1 - drivers/md/dm-ploop-map.c | 9 - drivers/md/dm-ploop.h | 2 +- 3 files changed, 5 insertions

[Devel] [RFC PATCH vz9 v5 30/49] dm-ploop: BAT use only one list for writeback

2024-11-17 Thread Alexander Atanasov
remove the wb_batch_list_prio, use only wb_batch_list and use the md high_prio flag for immediate submission. https://virtuozzo.atlassian.net/browse/VSTOR-91817 Signed-off-by: Alexander Atanasov --- drivers/md/dm-ploop-map.c| 70 +++- drivers/md/dm-ploop-targe

[Devel] [RFC PATCH vz9 v5 20/49] dm-ploop: reduce BAT accesses on discard completion

2024-11-17 Thread Alexander Atanasov
From: Andrey Zhadchenko Drop extra ploop_cluster_is_in_top_delta() as we are planning to access BAT anyway https://virtuozzo.atlassian.net/browse/VSTOR-91817 Signed-off-by: Andrey Zhadchenko --- drivers/md/dm-ploop-map.c | 28 1 file changed, 12 insertions(+), 16 d

[Devel] [RFC PATCH vz9 v5 24/49] dm-ploop: drop ploop_cluster_is_in_top_delta()

2024-11-17 Thread Alexander Atanasov
From: Andrey Zhadchenko as all it's users are gone or reworked https://virtuozzo.atlassian.net/browse/VSTOR-91817 Signed-off-by: Andrey Zhadchenko --- drivers/md/dm-ploop.h | 14 -- 1 file changed, 14 deletions(-) diff --git a/drivers/md/dm-ploop.h b/drivers/md/dm-ploop.h index 2f