Re: [PATCH v2] dm: error: Add support for zoned block devices

2023-10-25 Thread Damien Le Moal
On 10/26/23 08:50, Mike Snitzer wrote: > On Wed, Oct 25 2023 at 5:34P -0400, > Damien Le Moal wrote: > >> On 10/26/23 00:03, Mike Snitzer wrote: >>> On Wed, Oct 25 2023 at 3:23P -0400, >>> Damien Le Moal wrote: >>> >>>> dm-error is us

[PATCH v3] dm: error: Add support for zoned block devices

2023-10-25 Thread Damien Le Moal
with a NULL pointer dereference in set_dax_synchronous dax_dev is NULL. This is consistent with the current behavior because dm_table_supports_dax() always returned fals for targets that do not define the iterate_devices method. Signed-off-by: Damien Le Moal --- drivers/md/dm-table.c | 23

Re: [PATCH v3] dm: error: Add support for zoned block devices

2023-10-29 Thread Damien Le Moal
/linux-dm.git/commit/?h=dm-6.7&id=c85b4fb8b8edbd915283f6ab3537f2c3b95e7c85 Looks good. Thanks ! -- Damien Le Moal Western Digital Research

[PATCH] dm: Validate zoned model of devices built with zoned targets

2023-11-30 Thread Damien Le Moal
. With this change, dm-linear and dm-error devices built using conventional zones are now exposed as regular block devices. Signed-off-by: Damien Le Moal --- drivers/md/dm-table.c | 13 + drivers/md/dm-zone.c | 63 +++ drivers/md/dm.h | 1 + 3

Re: [PATCH] dm: Validate zoned model of devices built with zoned targets

2023-12-03 Thread Damien Le Moal
On 12/1/23 20:09, Johannes Thumshirn wrote: > On 01.12.23 03:03, Damien Le Moal wrote: >> +/* Count conventional and sequential zones */ >> +noio_flag = memalloc_noio_save(); >> +ret = dm_blk_do_report_zon

Re: [PATCH 3/5] block: remove support for the host aware zone model

2023-12-17 Thread Damien Le Moal
&zoned)) >> return false; >> } else if (!dm_target_supports_mixed_zoned_model(ti- >>> type)) { >> -if (zoned_model == BLK_ZONED_HM) >> +if (zoned) >>

Re: [PATCH 3/5] block: remove support for the host aware zone model

2023-12-18 Thread Damien Le Moal
On 12/18/23 17:21, Ed Tsai (蔡宗軒) wrote: > On Mon, 2023-12-18 at 15:53 +0900, Damien Le Moal wrote: >> On 2023/12/18 15:15, Ed Tsai (蔡宗軒) wrote: >>> Hi Christoph, >>> >>> some minor suggestions: >>> >>> On Sun, 2023-12-17 at 17:53 +0100, Chri

Re: [PATCH 1/5] virtio_blk: cleanup zoned device probing

2023-12-18 Thread Damien Le Moal
e model reading and sharing between builds with > and without CONFIG_BLK_DEV_ZONED, and improve it for the > !CONFIG_BLK_DEV_ZONED case. > > Signed-off-by: Christoph Hellwig Looks good. Reviewed-by: Damien Le Moal > --- > drivers/block/virtio_blk.c | 50 +--

Re: [PATCH 2/5] virtio_blk: remove the broken zone revalidation support

2023-12-18 Thread Damien Le Moal
uot;) > Fixes: f1ba4e674feb ("virtio-blk: fix to match virtio spec") > Signed-off-by: Christoph Hellwig Not ideal... But I think this is OK for now given that as you say, the upper layer will not be able to handle zone changes anyway. Reviewed-by: Damien Le Moal > --- > dr

Re: [PATCH 3/5] block: remove support for the host aware zone model

2023-12-18 Thread Damien Le Moal
= 1 << 4; /* zoned field = 01b */ I think we should keep everything related to HA in scsi debug as that is an easy way to test the block layer and scsi. no ? Other than this, very nice cleanup ! -- Damien Le Moal Western Digital Research

Re: [PATCH 4/5] block: simplify disk_set_zoned

2023-12-18 Thread Damien Le Moal
ph Hellwig Looks good. Reviewed-by: Damien Le Moal -- Damien Le Moal Western Digital Research

Re: [PATCH 5/5] sd: only call disk_clear_zoned when needed

2023-12-18 Thread Damien Le Moal
queue freezing when revalidating > conventional devices. > > Signed-off-by: Christoph Hellwig Looks good. Reviewed-by: Damien Le Moal -- Damien Le Moal Western Digital Research

Re: [PATCH] dm: Validate zoned model of devices built with zoned targets

2023-12-18 Thread Damien Le Moal
On 2023/12/01 11:03, Damien Le Moal wrote: > Using dm-linear or dm-error, a user can craft a DM device using > host-managed SMR disks by mapping only conventional zones. Currently, > such DM device will be reported also as being host-managed, despite the > fact that this zoned mo

Re: [PATCH 3/5] block: remove support for the host aware zone model

2023-12-19 Thread Damien Le Moal
+, Ed Tsai (蔡宗軒) wrote: >>> On Mon, 2023-12-18 at 15:53 +0900, Damien Le Moal wrote: >>>> On 2023/12/18 15:15, Ed Tsai (蔡宗軒) wrote: >>>>> Hi Christoph, >>>>> >>>>> some minor suggestions: >>>>> >>>>> O

Re: [PATCH 1/5] zonefs: pass GFP_KERNEL to blkdev_zone_mgmt() call

2024-01-23 Thread Damien Le Moal
n recurses into the filesystem and attempts to take >> i_truncate_mutex as well? >> >> i.e. GFP_KERNEL -> iomap_do_writepage -> zonefs_write_map_blocks -> >> zonefs_write_iomap_begin -> mutex_lock(&zi->i_truncate_mutex) > > zonefs doesn't have a ->writepage method, so writeback can't be > called from memory reclaim like this. And also, buffered writes are allowed only for conventional zone files, for which we never do zone management. For sequential zone files which may have there zone managed with blkdev_zone_mgmt(), only direct writes are allowed. > > -Dave. -- Damien Le Moal Western Digital Research

Re: [PATCH] softirq: fix memory corruption when freeing tasklet_struct

2024-01-25 Thread Damien Le Moal
looks like code bloat to me) That would force removing dm-multipath, which is I think the only DM driver using requests. But given how widespread the use of dm-multipath is, killing it would likely make a lot of people unhappy... -- Damien Le Moal Western Digital Research

Re: [PATCH v3 1/5] zonefs: pass GFP_KERNEL to blkdev_zone_mgmt() call

2024-01-29 Thread Damien Le Moal
"Zone management operation %s at %llu failed %d\n", I think this is OK but I will need a little more time to fully convince myself. So let me look again at the code to check all the calls contexts. -- Damien Le Moal Western Digital Research

Re: [PATCH v3 2/5] dm: dm-zoned: guard blkdev_zone_mgmt with noio scope

2024-01-29 Thread Damien Le Moal
On 1/29/24 16:52, Johannes Thumshirn wrote: > Guard the calls to blkdev_zone_mgmt() with a memalloc_noio scope. > This helps us getting rid of the GFP_NOIO argument to blkdev_zone_mgmt(); > > Signed-off-by: Johannes Thumshirn Reviewed-by: Damien Le Moal > --- > drivers/md/d

Re: [PATCH v3 3/5] btrfs: zoned: call blkdev_zone_mgmt in nofs scope

2024-01-29 Thread Damien Le Moal
On 1/29/24 16:52, Johannes Thumshirn wrote: > Add a memalloc_nofs scope around all calls to blkdev_zone_mgmt(). This > allows us to further get rid of the GFP_NOFS argument for > blkdev_zone_mgmt(). > > Signed-off-by: Johannes Thumshirn Looks OK to me. Reviewed-by: Damien Le Moa

Re: [PATCH v3 4/5] f2fs: guard blkdev_zone_mgmt with nofs scope

2024-01-29 Thread Damien Le Moal
On 1/29/24 16:52, Johannes Thumshirn wrote: > Guard the calls to blkdev_zone_mgmt() with a memalloc_nofs scope. > This helps us getting rid of the GFP_NOFS argument to blkdev_zone_mgmt(); > > Signed-off-by: Johannes Thumshirn Reviewed-by: Damien Le Moal -- Damien Le Moal Wes

Re: [PATCH v3 5/5] block: remove gfp_flags from blkdev_zone_mgmt

2024-01-29 Thread Damien Le Moal
et_all() and blkdev_zone_reset_all_emulated(). > > Signed-off-by: Johannes Thumshirn Looks good to me. Reviewed-by: Damien Le Moal But let me check zonefs (patch 1). -- Damien Le Moal Western Digital Research

[PATCH 00/26] Zone write plugging

2024-02-01 Thread Damien Le Moal
realistic use of the device by the FS, ZWP significantly improves SMR HDD performance thanks to the more efficient zone append emulation compared to ZWL. Damien Le Moal (26): block: Restore sector of flush requests block: Remove req_bio_endio() block: Introduce bio_straddle_zones() and bio_of

[PATCH 01/26] block: Restore sector of flush requests

2024-02-01 Thread Damien Le Moal
flush BIO was submitted. Signed-off-by: Damien Le Moal --- block/blk-flush.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/blk-flush.c b/block/blk-flush.c index b0f314f4bc14..2f58ae018464 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -130,6 +130,7 @@ static void

[PATCH 02/26] block: Remove req_bio_endio()

2024-02-01 Thread Damien Le Moal
using the request sector for these operations can be moved directly before the call to bio_endio(). Signed-off-by: Damien Le Moal --- block/blk-mq.c | 66 -- 1 file changed, 31 insertions(+), 35 deletions(-) diff --git a/block/blk-mq.c b/block/blk

[PATCH 06/26] block: Introduce zone write plugging

2024-02-01 Thread Damien Le Moal
sed for zoned disks, the function disk_free_zone_bitmaps() is renamed to the more generic disk_free_zone_resources(). This commit contains contributions from Christoph Hellwig . Signed-off-by: Damien Le Moal --- block/bio.c | 7 + block/blk-merge.c | 11 + block/blk-mq.

[PATCH 09/26] block: Allow BIO-based drivers to use blk_revalidate_disk_zones()

2024-02-01 Thread Damien Le Moal
-off-by: Damien Le Moal --- block/blk-zoned.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 929c28796c41..8bf6821735f3 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -1316,8 +1316,8 @@ static int blk_revalidate_zone_cb

[PATCH 10/26] dm: Use the block layer zone append emulation

2024-02-01 Thread Damien Le Moal
functions __map_bio() and dm_zone_endio(). This later function now only needs to deal with completions of real zone append operations for targets that support it. Signed-off-by: Damien Le Moal --- drivers/md/dm-core.h | 11 +- drivers/md/dm-zone.c | 470

[PATCH 13/26] null_blk: Do not request ELEVATOR_F_ZBD_SEQ_WRITE elevator feature

2024-02-01 Thread Damien Le Moal
this allows using a zoned null_blk device with any scheduler, including "none". Signed-off-by: Damien Le Moal --- drivers/block/null_blk/zoned.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 6f5e0994862e..f2

[PATCH 15/26] null_blk: Introduce fua attribute

2024-02-01 Thread Damien Le Moal
device with a write cache but without FUA support. This is convenient to test the block layer flush machinery. Signed-off-by: Damien Le Moal --- drivers/block/null_blk/main.c | 12 ++-- drivers/block/null_blk/null_blk.h | 1 + 2 files changed, 11 insertions(+), 2 deletions(-) diff

[PATCH 17/26] block: Remove BLK_STS_ZONE_RESOURCE

2024-02-01 Thread Damien Le Moal
The zone append emulation of the scsi disk driver was the only driver using BLK_STS_ZONE_RESOURCE. With this code removed, BLK_STS_ZONE_RESOURCE is now unused. Remove this macro definition and simplify blk_mq_dispatch_rq_list() where this status code was handled. Signed-off-by: Damien Le Moal

[PATCH 07/26] block: Allow zero value of max_zone_append_sectors queue limit

2024-02-01 Thread Damien Le Moal
0 limit. Signed-off-by: Damien Le Moal --- block/blk-core.c | 2 +- block/blk-settings.c | 35 +++ block/blk-sysfs.c | 2 +- block/blk-zoned.c | 2 +- include/linux/blkdev.h | 20 +--- 5 files changed, 43 insertions(+), 18

[PATCH 03/26] block: Introduce bio_straddle_zones() and bio_offset_from_zone_start()

2024-02-01 Thread Damien Le Moal
Implement the inline helper functions bio_straddle_zones() and bio_offset_from_zone_start() to respectively test if a BIO crosses a zone boundary (the start sector and last sector belong to different zones) and to obtain the oofset from a zone starting sector of a BIO. Signed-off-by: Damien Le

[PATCH 12/26] ublk_drv: Do not request ELEVATOR_F_ZBD_SEQ_WRITE elevator feature

2024-02-01 Thread Damien Le Moal
this allows using a zoned ublk device with any scheduler, including "none". Signed-off-by: Damien Le Moal --- drivers/block/ublk_drv.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 1dfb2e77898b..35fb9cc739eb 100644 ---

[PATCH 04/26] block: Introduce blk_zone_complete_request_bio()

2024-02-01 Thread Damien Le Moal
blk_zone_complete_request_bio() to avoid duplicating this BIO update for zone append requests, and to compile out this helper call when CONFIG_BLK_DEV_ZONED is not enabled. Signed-off-by: Damien Le Moal --- block/blk-mq.c | 11 +-- block/blk.h| 19 ++- 2 files changed, 23

[PATCH 05/26] block: Allow using bio_attempt_back_merge() internally

2024-02-01 Thread Damien Le Moal
Remove the static definition of bio_attempt_back_merge() to allow using this function internally from other block layer files. Add the definition of enum bio_merge_status and the declaration of bio_attempt_back_merge() to block/blk.h. Signed-off-by: Damien Le Moal --- block/blk-merge.c | 8

[PATCH 08/26] block: Implement zone append emulation

2024-02-01 Thread Damien Le Moal
. This commit contains contributions from Christoph Hellwig . Signed-off-by: Damien Le Moal --- block/blk-mq.c| 2 +- block/blk-zoned.c | 457 -- block/blk.h | 14 +- include/linux/blk_types.h | 1 + include/linux/blkdev.h

[PATCH 11/26] scsi: sd: Use the block layer zone append emulation

2024-02-01 Thread Damien Le Moal
the zone append emulation code implemented in sd_zbc.c can be removed. Signed-off-by: Damien Le Moal --- drivers/scsi/sd.c | 8 - drivers/scsi/sd.h | 19 --- drivers/scsi/sd_zbc.c | 335 ++ 3 files changed, 10 insertions(+), 352 deletions

[PATCH 14/26] null_blk: Introduce zone_append_max_sectors attribute

2024-02-01 Thread Damien Le Moal
() is modified to use blk_zone_write_plug_bio() to handle zone append emulation if that is enabled. Signed-off-by: Damien Le Moal --- drivers/block/null_blk/main.c | 40 +-- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/zoned.c| 31

[PATCH 16/26] nvmet: zns: Do not reference the gendisk conv_zones_bitmap

2024-02-01 Thread Damien Le Moal
The gendisk conventional zone bitmap is going away. So to check for the presence of conventional zones on a zoned target device, always use report zones. Signed-off-by: Damien Le Moal --- drivers/nvme/target/zns.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a

[PATCH 18/26] block: Simplify blk_revalidate_disk_zones() interface

2024-02-01 Thread Damien Le Moal
. there is no gendisk ->revalidate method). Signed-off-by: Damien Le Moal --- block/blk-zoned.c | 16 +--- drivers/block/null_blk/zoned.c | 2 +- drivers/block/ublk_drv.c | 2 +- drivers/block/virtio_blk.c | 2 +- drivers/md/dm-zone.c | 2 +- driv

[PATCH 19/26] block: mq-deadline: Remove support for zone write locking

2024-02-01 Thread Damien Le Moal
dispatching of write requests using zone write locking. Remove all the code that implement this control in the mq-deadline scheduler and remove advertizing support for the ELEVATOR_F_ZBD_SEQ_WRITE elevator feature. Signed-off-by: Damien Le Moal --- block/mq-deadline.c | 176

[PATCH 20/26] block: Remove elevator required features

2024-02-01 Thread Damien Le Moal
required features. Signed-off-by: Damien Le Moal --- block/blk-settings.c | 16 --- block/elevator.c | 46 +- block/elevator.h | 1 - include/linux/blkdev.h | 10 - 4 files changed, 5 insertions(+), 68 deletions(-) diff --git

[PATCH 21/26] block: Do not check zone type in blk_check_zone_append()

2024-02-01 Thread Damien Le Moal
zone. In preparation for the removal of zone write locking and its conventional zone bitmap (used by bio_zone_is_seq()), remove the bio_zone_is_seq() call from blk_check_zone_append(). Signed-off-by: Damien Le Moal --- block/blk-core.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions

[PATCH 22/26] block: Move zone related debugfs attribute to blk-zoned.c

2024-02-01 Thread Damien Le Moal
block/blk-mq-debugfs-zone.c contains a single debugfs attribute function. Defining this outside of block/blk-zoned.c does not really help in any way, so move this zone related debugfs attribute to block/blk-zoned.c and delete block/blk-mq-debugfs-zone.c. Signed-off-by: Damien Le Moal --- block

[PATCH 23/26] block: Remove zone write locking

2024-02-01 Thread Damien Le Moal
as listing zones that are write-locked is replaced with the zone_plugged_wplugs entry which lists the number of zones that have a zone write plug throttling write operations. Signed-off-by: Damien Le Moal --- block/blk-mq-debugfs.c| 3 +- block/blk-mq-debugfs.h| 4 +- block/b

[PATCH 24/26] block: Do not special-case plugging of zone write operations

2024-02-01 Thread Damien Le Moal
ged requests will be harmless. We can thus remove the special casing for write operations to zones and have these requests plugged as well. This allows removing the function blk_mq_plug and instead directly using current->plug where needed. Signed-off-by: Damien Le Moal --- block/blk-core.c

[PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-01 Thread Damien Le Moal
block device for zone write plugs is roughly reduced by a factor of 4. E.g. for a 28 TB SMR hard disk, memory usage is reduce to about 1.6 MB. This commit contains contributions from Christoph Hellwig . Signed-off-by: Damien Le Moal --- block/blk-core.c | 2 + block/blk-zoned.c | 293

[PATCH 26/26] block: Add zone_active_wplugs debugfs entry

2024-02-01 Thread Damien Le Moal
device user. Signed-off-by: Damien Le Moal --- block/blk-mq-debugfs.c | 1 + block/blk-mq-debugfs.h | 5 + block/blk-zoned.c | 27 +++ 3 files changed, 33 insertions(+) diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index b803f5b370e9..5390526f2ab0

Re: [PATCH 00/26] Zone write plugging

2024-02-01 Thread Damien Le Moal
On 2/2/24 16:30, Damien Le Moal wrote: > The patch series introduces zone write plugging (ZWP) as the new > mechanism to control the ordering of writes to zoned block devices. > ZWP replaces zone write locking (ZWL) which is implemented only by > mq-deadline today. ZWP also allows em

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-04 Thread Damien Le Moal
On 2/4/24 12:56, Ming Lei wrote: > On Fri, Feb 02, 2024 at 04:30:44PM +0900, Damien Le Moal wrote: >> +/* >> + * Zone write plug flags bits: >> + * - BLK_ZONE_WPLUG_CONV: Indicate that the zone is a conventional one. >> Writes >> + *

Re: [PATCH 08/26] block: Implement zone append emulation

2024-02-04 Thread Damien Le Moal
On 2/4/24 21:24, Hannes Reinecke wrote: > On 2/2/24 15:30, Damien Le Moal wrote: >> +/* >> + * Set a zone write plug write pointer offset to either 0 (zone reset case) >> + * or to the zone size (zone finish case). This aborts all plugged BIOs, >> which >> + * is

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-04 Thread Damien Le Moal
, because queue freeze can't prevent new > write zoned bio from being submitted any more given only percpu_ref_get() is > called for all write zoned bios. New BIOs (BIOS that have never been plugged yet) will go through the normal blk_queue_enter() in blk_mq_submit_bio(), so they will be stopped there if another context asked for a queue freeze. I do not think there is any issue with how things are currently done (we tested that *a lot* with many different drives and drive configs with DM etc). Reference counting as it is is OK, even though it most likely be simplified. I am looking at that now. -- Damien Le Moal Western Digital Research

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-04 Thread Damien Le Moal
On 2/5/24 14:11, Christoph Hellwig wrote: > On Mon, Feb 05, 2024 at 11:41:04AM +0900, Damien Le Moal wrote: >>> I think only queue re-configuration(blk_revalidate_zone) requires the >>> queue usage counter. Otherwise, bdev open()/close() should work just >>> fine.

Re: [PATCH 10/26] dm: Use the block layer zone append emulation

2024-02-04 Thread Damien Le Moal
) { return md->emulate_zone_append && blk_zone_write_plug_bio(bio, 0); } These 2 helpers define to "return false" for !CONFIG_BLK_DEV_ZONED. I hope this works for you. Otherwise, I will drop your review tag when posting V2. -- Damien Le Moal Western Digital Research

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-04 Thread Damien Le Moal
On 2/5/24 14:50, Christoph Hellwig wrote: > On Mon, Feb 05, 2024 at 02:37:41PM +0900, Damien Le Moal wrote: >> OK. So I think that Hannes'idea to get/put the queue usage counter reference >> based on a zone BIO plug becoming not empty (get ref) and becoming empty (put >>

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-05 Thread Damien Le Moal
On 2/5/24 19:06, Ming Lei wrote: > On Mon, Feb 05, 2024 at 11:41:04AM +0900, Damien Le Moal wrote: >> On 2/5/24 11:19, Ming Lei wrote: >>>>>> +static inline void blk_zone_wplug_add_bio(struct blk_zone_wplug *zwplug, >>>>>> +

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-05 Thread Damien Le Moal
On 2/5/24 21:20, Damien Le Moal wrote: > On 2/5/24 19:06, Ming Lei wrote: >> On Mon, Feb 05, 2024 at 11:41:04AM +0900, Damien Le Moal wrote: >>> On 2/5/24 11:19, Ming Lei wrote: >>>>>>> +static inline void blk_zone_wplug_add_bio

Re: [PATCH 10/26] dm: Use the block layer zone append emulation

2024-02-05 Thread Damien Le Moal
On 2/6/24 05:33, Mike Snitzer wrote: > On Mon, Feb 05 2024 at 12:38P -0500, > Damien Le Moal wrote: > >> On 2/4/24 02:58, Mike Snitzer wrote: >>> Love the overall improvement to the DM core code and the broader block >>> layer by switching to this bio-based

Re: [PATCH 00/26] Zone write plugging

2024-02-05 Thread Damien Le Moal
On 2/6/24 02:21, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> The patch series introduces zone write plugging (ZWP) as the new >> mechanism to control the ordering of writes to zoned block devices. >> ZWP replaces zone write locking (ZWL) which is impl

Re: [PATCH 01/26] block: Restore sector of flush requests

2024-02-05 Thread Damien Le Moal
On 2/6/24 02:22, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> diff --git a/block/blk-flush.c b/block/blk-flush.c >> index b0f314f4bc14..2f58ae018464 100644 >> --- a/block/blk-flush.c >> +++ b/block/blk-flush.c >> @@ -130,6 +130,7 @@ stat

Re: [PATCH 02/26] block: Remove req_bio_endio()

2024-02-05 Thread Damien Le Moal
On 2/6/24 02:28, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> @@ -916,9 +888,8 @@ bool blk_update_request(struct request *req, >> blk_status_t error, >> if (blk_crypto_rq_has_keyslot(req) && nr_bytes >= blk_rq_bytes(req)) >>

Re: [PATCH 06/26] block: Introduce zone write plugging

2024-02-05 Thread Damien Le Moal
On 2/6/24 02:48, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> The next plugged BIO is unplugged and issued once the write request >> completes. > > So this patch series is orthogonal to my patch series that implements zoned > write p

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-05 Thread Damien Le Moal
On 2/6/24 02:51, Bart Van Assche wrote: > On 2/4/24 04:42, Hannes Reinecke wrote: >> On 2/2/24 15:31, Damien Le Moal wrote: >>> With this mechanism, the amount of memory used per block device for zone >>> write plugs is roughly reduced by a factor of 4. E.g. for a 28 T

Re: [PATCH 08/26] block: Implement zone append emulation

2024-02-05 Thread Damien Le Moal
On 2/6/24 02:58, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> diff --git a/block/blk-zoned.c b/block/blk-zoned.c >> index 661ef61ca3b1..929c28796c41 100644 >> --- a/block/blk-zoned.c >> +++ b/block/blk-zoned.c >> @@ -42,6 +42,8 @@ struct blk_

Re: [PATCH 00/26] Zone write plugging

2024-02-05 Thread Damien Le Moal
On 2/6/24 03:18, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> - Zone write plugging operates on BIOs instead of requests. Plugged >> BIOs waiting for execution thus do not hold scheduling tags and thus >> do not prevent other BIOs from being su

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-08 Thread Damien Le Moal
On 2/7/24 06:20, Bart Van Assche wrote: > On 2/5/24 15:55, Damien Le Moal wrote: >> The array of struct blk_zone_wplug for the disk is sized for the total >> number of >> zones of the drive. The reason for that is that we want to retain the >> wp_offset >> valu

Re: [PATCH 00/26] Zone write plugging

2024-02-08 Thread Damien Le Moal
On 2/6/24 10:25, Bart Van Assche wrote: > On 2/5/24 16:07, Damien Le Moal wrote: >> On 2/6/24 03:18, Bart Van Assche wrote: >>> Are there numbers available about the performance differences (bandwidth >>> and latency) between plugging zoned write bios and zoned

Re: [PATCH 00/26] Zone write plugging

2024-02-08 Thread Damien Le Moal
that you removed the mq-deadline insert optimization ? Is that in purpose or did I mess up something ? -- Damien Le Moal Western Digital Research

Re: [PATCH 02/26] block: Remove req_bio_endio()

2024-02-08 Thread Damien Le Moal
On 2/6/24 02:28, Bart Van Assche wrote: > On 2/1/24 23:30, Damien Le Moal wrote: >> @@ -916,9 +888,8 @@ bool blk_update_request(struct request *req, blk_status_t >> error, >>   if (blk_crypto_rq_has_keyslot(req) && nr_bytes >= blk_rq_bytes(req)) >>    

Re: [PATCH v3 1/5] zonefs: pass GFP_KERNEL to blkdev_zone_mgmt() call

2024-02-08 Thread Damien Le Moal
ck for reclaim. So there should be no risk of re-entering the FS on reclaim with GFP_KERNEL. So: Acked-by: Damien Le Moal -- Damien Le Moal Western Digital Research

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-09 Thread Damien Le Moal
On 2/10/24 04:36, Bart Van Assche wrote: > On 2/8/24 19:58, Damien Le Moal wrote: >> We still need to keep in memory the write pointer offset of zones that are >> not >> being actively written to but have been previously partially written. So I do >> not see how exc

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-11 Thread Damien Le Moal
On 2/11/24 12:40, Bart Van Assche wrote: > On 2/9/24 16:06, Damien Le Moal wrote: >> On 2/10/24 04:36, Bart Van Assche wrote: >>> written zones is typically less than 10. Hence, tracking the partially >>> written >> >> That is far from guaranteed, especiall

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-12 Thread Damien Le Moal
On 2/11/24 12:40, Bart Van Assche wrote: > On 2/9/24 16:06, Damien Le Moal wrote: >> On 2/10/24 04:36, Bart Van Assche wrote: >>> written zones is typically less than 10. Hence, tracking the partially >>> written >> >> That is far from guaranteed, especiall

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-12 Thread Damien Le Moal
On 2/12/24 17:23, Damien Le Moal wrote: > On 2/11/24 12:40, Bart Van Assche wrote: >> On 2/9/24 16:06, Damien Le Moal wrote: >>> On 2/10/24 04:36, Bart Van Assche wrote: >>>> written zones is typically less than 10. Hence, tracking the partially >>>> w

Re: [PATCH 25/26] block: Reduce zone write plugging memory usage

2024-02-12 Thread Damien Le Moal
On 2/13/24 03:40, Bart Van Assche wrote: > On 2/12/24 00:47, Damien Le Moal wrote: >> Replying to myself as I had an idea: >> 1) Store the zone capacity in a separate array: 4B * nr_zones needed. Storing >> "0" as a value for a zone in that array would indicate that

[PATCH v2 04/28] block: Introduce bio_straddle_zones() and bio_offset_from_zone_start()

2024-03-25 Thread Damien Le Moal
: Damien Le Moal Reviewed-by: Hannes Reinecke --- include/linux/blkdev.h | 13 + 1 file changed, 13 insertions(+) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index c3e8f7cf96be..93f93f352b54 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -853,6

[PATCH v2 03/28] block: Introduce blk_zone_update_request_bio()

2024-03-25 Thread Damien Le Moal
blk_zone_update_request_bio() to avoid duplicating this BIO update for zone append requests, and to compile out this helper call when CONFIG_BLK_DEV_ZONED is not enabled. Signed-off-by: Damien Le Moal --- block/blk-mq.c | 11 +-- block/blk.h| 19 ++- 2 files changed, 23

[PATCH v2 02/28] block: Remove req_bio_endio()

2024-03-25 Thread Damien Le Moal
using the request sector for these operations can be moved directly before the call to bio_endio(). Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- block/blk-mq.c | 66 -- 1 file changed, 31 insertions(+), 35 deletions(-) diff --git a

[PATCH v2 05/28] block: Allow using bio_attempt_back_merge() internally

2024-03-25 Thread Damien Le Moal
Remove "static" from the definition of bio_attempt_back_merge() and declare this function in block/blk.h to allow using it internally from other block layer files. The definition of enum bio_merge_status is also moved to block/blk.h. Signed-off-by: Damien Le Moal Reviewed-by: Hanne

[PATCH v2 07/28] block: Introduce zone write plugging

2024-03-25 Thread Damien Le Moal
allocated. This commit contains contributions from Christoph Hellwig . Signed-off-by: Damien Le Moal --- block/bio.c |7 + block/blk-core.c |2 + block/blk-merge.c | 11 + block/blk-mq.c| 38 +- block/blk-zoned.c | 1034 +

[PATCH v2 10/28] block: Allow zero value of max_zone_append_sectors queue limit

2024-03-25 Thread Damien Le Moal
function as well. Stacking drivers that require zone append emulation, e.g. dm-crypt, can still request this feature by calling blk_queue_max_zone_append_sectors() with a 0 limit. Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- block/blk-core.c | 2 +- block/blk-settings.c

[PATCH v2 13/28] dm: Use the block layer zone append emulation

2024-03-25 Thread Damien Le Moal
functions __map_bio() and dm_zone_endio(). This later function now only needs to deal with completions of real zone append operations for targets that support it. Signed-off-by: Damien Le Moal Reviewed-by: Mike Snitzer Reviewed-by: Hannes Reinecke --- drivers/md/dm-core.h | 2 +- drivers/md/dm

[PATCH v2 12/28] block: Allow BIO-based drivers to use blk_revalidate_disk_zones()

2024-03-25 Thread Damien Le Moal
bitmap are allocated and initialized only for mq devices and for BIO-based devices that require zone append emulation. Signed-off-by: Damien Le Moal --- block/blk-zoned.c | 26 -- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/block/blk-zoned.c b/block/blk

[PATCH v2 11/28] block: Implement zone append emulation

2024-03-25 Thread Damien Le Moal
blk_update_request() and blk_zone_complete_request_bio() are modified to use bio_is_zone_append() to execute blk_zone_update_request_bio() for both native and emulated zone append operations. This commit contains contributions from Christoph Hellwig . Signed-off-by: Damien Le Moal --- block/blk-mq.c

[PATCH v2 15/28] ublk_drv: Do not request ELEVATOR_F_ZBD_SEQ_WRITE elevator feature

2024-03-25 Thread Damien Le Moal
this allows using a zoned ublk device with any scheduler, including "none". Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- drivers/block/ublk_drv.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_d

[PATCH v2 18/28] null_blk: Introduce fua attribute

2024-03-25 Thread Damien Le Moal
device with a write cache but without FUA support. This is convenient to test the block layer flush machinery. Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- drivers/block/null_blk/main.c | 12 ++-- drivers/block/null_blk/null_blk.h | 1 + 2 files changed, 11

[PATCH v2 16/28] null_blk: Do not request ELEVATOR_F_ZBD_SEQ_WRITE elevator feature

2024-03-25 Thread Damien Le Moal
this allows using a zoned null_blk device with any scheduler, including "none". Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- drivers/block/null_blk/zoned.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zon

[PATCH v2 20/28] block: Remove BLK_STS_ZONE_RESOURCE

2024-03-25 Thread Damien Le Moal
The zone append emulation of the scsi disk driver was the only driver using BLK_STS_ZONE_RESOURCE. With this code removed, BLK_STS_ZONE_RESOURCE is now unused. Remove this macro definition and simplify blk_mq_dispatch_rq_list() where this status code was handled. Signed-off-by: Damien Le Moal

[PATCH v2 00/28] Zone write plugging

2024-03-25 Thread Damien Le Moal
of patch 8 and 9. - Rebased everything on 6.9.0-rc1 - Added review tags for patches that did not change Damien Le Moal (28): block: Restore sector of flush requests block: Remove req_bio_endio() block: Introduce blk_zone_update_request_bio() block: Introduce bio_straddle_zones() and bio_of

[PATCH v2 21/28] block: Simplify blk_revalidate_disk_zones() interface

2024-03-25 Thread Damien Le Moal
. there is no gendisk ->revalidate method). Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- block/blk-zoned.c | 16 +--- drivers/block/null_blk/zoned.c | 2 +- drivers/block/ublk_drv.c | 2 +- drivers/block/virtio_blk.c | 2 +- drivers/md

[PATCH v2 27/28] block: Do not force select mq-deadline with CONFIG_BLK_DEV_ZONED

2024-03-25 Thread Damien Le Moal
Now that zone block device write ordering control does not depend anymore on mq-deadline and zone write locking, there is no need to force select the mq-deadline scheduler when CONFIG_BLK_DEV_ZONED is enabled. Signed-off-by: Damien Le Moal --- block/Kconfig | 1 - 1 file changed, 1 deletion

[PATCH v2 26/28] block: Remove zone write locking

2024-03-25 Thread Damien Le Moal
aiting for execution in the zone write plug BIO list. Signed-off-by: Damien Le Moal --- block/blk-mq-debugfs.c| 3 +- block/blk-mq-debugfs.h| 4 +- block/blk-zoned.c | 92 ++- include/linux/blk-mq.h| 83 --- inc

[PATCH v2 01/28] block: Restore sector of flush requests

2024-03-25 Thread Damien Le Moal
flush BIO was submitted. Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- block/blk-flush.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/blk-flush.c b/block/blk-flush.c index b0f314f4bc14..2f58ae018464 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -130,6

[PATCH v2 25/28] block: Move zone related debugfs attribute to blk-zoned.c

2024-03-25 Thread Damien Le Moal
block/blk-mq-debugfs-zone.c contains a single debugfs attribute function. Defining this outside of block/blk-zoned.c does not really help in any way, so move this zone related debugfs attribute to block/blk-zoned.c and delete block/blk-mq-debugfs-zone.c. Signed-off-by: Damien Le Moal Reviewed-by

[PATCH v2 28/28] block: Do not special-case plugging of zone write operations

2024-03-25 Thread Damien Le Moal
ged requests will be harmless. We can thus remove the special casing for write operations to zones and have these requests plugged as well. This allows removing the function blk_mq_plug and instead directly using current->plug where needed. Signed-off-by: Damien Le Moal Reviewed-by: Hannes Re

[PATCH v2 09/28] block: Fake max open zones limit when there is no limit

2024-03-25 Thread Damien Le Moal
performance penalty that may happen when writing simultaneously to more zones than the mempool size. Signed-off-by: Damien Le Moal --- block/blk-zoned.c | 22 -- include/linux/blkdev.h | 1 + 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/block/blk-zoned.c b

[PATCH v2 14/28] scsi: sd: Use the block layer zone append emulation

2024-03-25 Thread Damien Le Moal
the zone append emulation code implemented in sd_zbc.c can be removed. Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke Reviewed-by: Martin K. Petersen --- drivers/scsi/sd.c | 8 - drivers/scsi/sd.h | 19 --- drivers/scsi/sd_zbc.c | 335

[PATCH v2 22/28] block: mq-deadline: Remove support for zone write locking

2024-03-25 Thread Damien Le Moal
dispatching of write requests using zone write locking. Remove all the code that implement this control in the mq-deadline scheduler and remove advertizing support for the ELEVATOR_F_ZBD_SEQ_WRITE elevator feature. Signed-off-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- block/mq-deadline.c | 176

[PATCH v2 06/28] block: Remember zone capacity when revalidating zones

2024-03-25 Thread Damien Le Moal
all known NVMe ZNS devices have the same zone capacity for all zones blk_revalidate_disk_zones() returns an error if different capacities are detected for different zones. Signed-off-by: Damien Le Moal --- block/blk-zoned.c | 15 +++ include/linux/blkdev.h | 1 + 2 files changed

[PATCH v2 17/28] null_blk: Introduce zone_append_max_sectors attribute

2024-03-25 Thread Damien Le Moal
: Damien Le Moal --- drivers/block/null_blk/main.c | 12 ++-- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/zoned.c| 22 +++--- 3 files changed, 30 insertions(+), 5 deletions(-) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk

  1   2   3   4   5   6   >